Guardium Insights – installation cookbook

The central reporting of activity in DAM systems is measured against the problem of huge amounts of data and the necessity of long data retention enforced by regulations, as well as correct identification of anomalies in user behavior through quantitative analysis.
Guardium Insights (GI) is a response to these needs and is designed as a container service implementable in both private and public clouds.
The following procedure brings together the various installation and configuration steps required to install Guardium Insights. The whole is divided into 5 main tasks:

1. System requirements

GI requires an IBM Cloud Private environment running on the OpenShift platform.
Minimum hardware requirements are 4 physical or virtual nodes with the following specifications:

  • Master node:
    – 16 cores, 64 GB RAM, 300 GB HDD

2019-12-30_13-46-02

  • Data node:
    – 16 cores, 64 GB RAM, 300+300 GB HDD (OS+GlusterFS) + 0.5 TB or more storage space for gathered data

2019-12-30_14-06-13

  • 2 auxiliary nodes:
    – 16 cores, 32 GB RAM, 300+300 GB HDD (OS+GlusterFS)

2019-12-30_14-23-04

In my case, I use an ESX server with 100 cores, 240 GB RAM and 40 TB HDD

From my point of view huge root partitions on auxiliary nodes are not important and standard 20-40 GB root filesystem should be enough.

Software requirements:

  • RedHat 7 with access to OpenShift repository
  • IBM Cloud Private 3.2.1 (latest patch preferred)
  • Security Guardium Insights 2.0
  • DNS resolver

2. RedHat setup

OS installation2019-12-30_13-53-47

I suggest Minimal Install with some standard administration tools. Access to infrastructure servers GUI is not required.

Disk setup

Install OS on 300 GB disk and use manual partitioning. Other disks will be configured later.2019-12-30_14-17-55

Switch to suggested configuration2019-12-30_13-55-38

Remove the /home partition from suggested configuration2019-12-30_13-57-21

Set root partition size to maximum available space2019-12-30_13-58-31

In my example the root partition will be used as docker storage (overlayFS) hence its size.
You can also install the docker before installing OpenShift and configure the storage differently, for example as a thin pool and then the root partition can be 50 GB.

Networking

I suggest static ip configuration of your GI infrastructure. All names have to be resolvable using DNS.2019-12-30_14-00-29

2019-12-30_15-19-10

My nodes are located in this same VLAN so there is no needs to setup firewall rules. However you should check connectivity between them in case of multi-VLAN architecture. The nature of container networking setup in the OpenShift requires that communication between nodes should be generally open.

Additionally create DNS GI subdomain – for example insights – and add the redirection to master node

*.<subdomain>.<domain> <ip_address>

2019-12-30_15-27-23

Set DNS to resolve correctly all your node names.

RedHat subscription and rpm’s setup (execute on all cluster nodes)

Register nodes

subscription-manager register --username <user>  --password <password>

Attach subscription with access to RedHat and OpenShift repos

subscription-manager attach --pool=<pool>

Check attached subscriptions

subscription-manager list --consumed

You should have access to the marked functionalities2019-12-30_16-02-31

Add required installation repo’s

subscription-manager repos --enable="rhel-7-server-rpms" --enable="rhel-7-server-extras-rpms" --enable="rhel-7-server-ose-3.11-rpms" --enable="rhel-7-server-ansible-2.6-rpms"

Update system

yum update

Finally restart system

shutdown -r now

3. OpenShift installation (all steps executed on master node)

My installation assumes that root account is OpenShift instance owner
I refer to node names, IP addresses in my lab – they should be modified accordingly
GI 2.0 supports OpenShift 3.11 only

RedHat configuration

Generate ssh key without passphrase

ssh-keygen

2019-12-30_16-40-41

Then copy key to other nodes to eliminate password authentication between nodes

for h in gimaster gidb giaux1 giaux2; do ssh-copy-id -i ~/.ssh/id_rsa.pub $h; done

Install openshift-ansible package on master

yum install openshift-ansible

Add glusterfs client repo

for h in gimaster gidb giaux1 giaux2; do ssh $h subscription-manager repos --enable=rh-gluster-3-client-for-rhel-7-server-rpms; done

Install gluster-fuse rpm

for h in gimaster gidb giaux1 giaux2; do ssh $h yum -y install glusterfs-fuse; done

Inventory file review

Our OpenShift configuration must meet IBM Cloud Private and Guardium Insights requirements for glusterfs support and designation of the master node as a worker.

Here my inventory.ini file with some important remarks:

[masters]
gimaster.guardium.notes
[etcd]
gimaster.guardium.notes
[nodes]
gidb.guardium.notes openshift_node_group_name="configmap-0"
giaux1.guardium.notes openshift_node_group_name="configmap-1"
giaux2.guardium.notes openshift_node_group_name="configmap-2"
gimaster.guardium.notes openshift_node_group_name="configmap-3"
[glusterfs]
gidb.guardium.notes glusterfs_ip=192.168.250.160 glusterfs_devices='["/dev/sdb"]'
giaux1.guardium.notes glusterfs_ip=192.168.250.170 glusterfs_devices='["/dev/sdb"]'
giaux2.guardium.notes glusterfs_ip=192.168.250.180 glusterfs_devices='["/dev/sdb"]'
[glusterfs_registry]
gidb.guardium.notes glusterfs_ip=192.168.250.160 glusterfs_devices='["/dev/sdb"]'
giaux1.guardium.notes glusterfs_ip=192.168.250.170 glusterfs_devices='["/dev/sdb"]'
giaux2.guardium.notes glusterfs_ip=192.168.250.180 glusterfs_devices='["/dev/sdb"]'
[OSEv3:children]
masters
nodes
etcd
glusterfs
glusterfs_registry
[OSEv3:vars]
openshift_node_groups=[{'name': 'configmap-0', 'labels': ['node-role.kubernetes.io/compute=true'], 'edits': [{ 'key': 'nodeIP','value': '192.168.250.160'}]},{'name': 'configmap-1', 'labels': ['node-role.kubernetes.io/compute=true'], 'edits': [{ 'key': 'nodeIP','value': '192.168.250.170'}]},{'name': 'configmap-2', 'labels': ['node-role.kubernetes.io/compute=true'], 'edits': [{ 'key': 'nodeIP','value': '192.168.250.180'}]},{'name': 'configmap-3', 'labels': ['node-role.kubernetes.io/master=true','node-role.kubernetes.io/infra=true','node-role.kubernetes.io/compute=true'], 'edits': [{ 'key': 'nodeIP','value': '192.168.250.150'}]}]
ansible_user=root
ansible_become=false
ansible_ssh_user=root
host_key_checking=False
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
openshift_deployment_type=openshift-enterprise
openshift_release=v3.11
openshift_master_default_subdomain=insights.guardium.notes
openshift_disable_check=docker_storage,docker_image_availability,package_version
openshift_storage_glusterfs_image=registry.access.redhat.com/rhgs3/rhgs-server-rhel7:v3.11
openshift_storage_glusterfs_block_image=registry.access.redhat.com/rhgs3/rhgs-gluster-block-prov-rhel7:v3.11
openshift_storage_glusterfs_heketi_image=registry.access.redhat.com/rhgs3/rhgs-volmanager-rhel7:v3.11
openshift_storage_glusterfs_timeout=900
openshift_storage_glusterfs_namespace=glusterfs
openshift_storage_glusterfs_storageclass=true
openshift_storage_glusterfs_storageclass_default=false
openshift_storage_glusterfs_block_deploy=true
openshift_storage_glusterfs_block_host_vol_size=50
openshift_storage_glusterfs_block_storageclass=true
openshift_storage_glusterfs_block_storageclass_default=true
debug_level=2
openshift_docker_selinux_enabled=true
openshift_docker_options="--selinux-enabled --signature-verification=false --insecure-registry=['192.168.250/24','172.30.0.0/16'] --log-opt max-size=1M --log-opt max-file=3 --disable-legacy-registry=true"
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_master_htpasswd_users={'ocadmin': '$apr1$e4i8sseH$zN5tmrKq3WjCNnIVUPmKM.'}
openshift_master_api_port=7443
openshift_master_console_port=7443
openshift_hostname_check=false
os_sdn_network_plugin_name=redhat/openshift-ovs-networkpolicy
oreg_url=registry.access.redhat.com/openshift3/ose-${component}:${version}
oreg_auth_user=""
oreg_auth_password=""
openshift_examples_modify_imagestreams=true
openshift_hosted_registry_cert_expire_days=3650
openshift_ca_cert_expire_days=3650
openshift_node_cert_expire_days=3650
openshift_master_cert_expire_days=3650
etcd_ca_default_days=3650

[OSEv3:children] – our OpenShift deployment contains glusterfs and glusterfs_registry support
glusterfs_devices – this parameter points the block device of the additional disk available on all nodes except master, check using fdisk -l the correct device name, do not point here an additional 0.5TB+ disk on storage node (gidb machine in my case)
openshift_node_groups – it marks all nodes as compute (‘labels’: [‘node-role.kubernetes.io/compute=true’]) and additionally defines that master node will have also master and infra designation (‘labels’: [‘node-role.kubernetes.io/master=true’,’node-role.kubernetes.io/infra=true’,’node-role.kubernetes.io/compute=true’])
Some browsers (IE) incorrectly displays my example, here full line:
openshift_node_groups=[{‘name’: ‘configmap-0’, ‘labels’: [‘node-role.kubernetes.io/compute=true’], ‘edits’: [{ ‘key’: ‘nodeIP’,’value’: ‘192.168.250.160’}]},{‘name’: ‘configmap-1’, ‘labels’: [‘node-role.kubernetes.io/compute=true’], ‘edits’: [{ ‘key’: ‘nodeIP’,’value’: ‘192.168.250.170’}]},{‘name’: ‘configmap-2’, ‘labels’: [‘node-role.kubernetes.io/compute=true’], ‘edits’: [{ ‘key’: ‘nodeIP’,’value’: ‘192.168.250.180’}]},{‘name’: ‘configmap-3’, ‘labels’: [‘node-role.kubernetes.io/master=true’,’node-role.kubernetes.io/infra=true’,’node-role.kubernetes.io/compute=true’], ‘edits’: [{ ‘key’: ‘nodeIP’,’value’: ‘192.168.250.150’}]}]
openshift_master_default_subdomain – subdomain reference for cluster services, should correspond to DNS name defined before (insights.guardium.notes in my case)
openshift_docker_options – provide corrector network subnet definition for your nodes in the parameter –insecure-registry, value 172.30.0.0/16 is default virtual ip addressing for OpenShift cluster ([‘192.168.250/24‘,’172.30.0.0/16’] – value 192.168.250/24 points my node’s public addresses
openshift_master_htpasswd_users – defines user and password for OpenShift default admin, password must be provided in encrypted form (use htpasswd)
openshift_master_api_port and openshift_master_console_port – OpenShift portal port
oreg_auth_user and oreg_auth_password – user and password for RHN account

htpasswd command to get encrypted user password

htpasswd -nb <user> <password>

Inventory file creation

Create inventory.ini file (base on proposed above) in root home directory on master and adapt it to your settings

Prerequisites playbook execution

Execute prerequisites.yml playbook

ansible-playbook -i inventory.ini /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml

After a few minutes, operations on all nodes should be completed.

Sometimes the playbook can hang during the installation of the docker. If the installation progress output does not change the status on this task for a few minutes, restart the playbook.

2019-12-30_20-20-57

Cluster deploy playbook execution

Execute deploy_cluster.yml playbook

ansible-playbook -i inventory.ini /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml

2019-12-30_22-08-34

After several minutes, the installation process should be completed and display summary

The command below displays information about node designation and its status

oc get nodes

2019-12-31_16-41-55

I suggest execute these commands to assign to ocadmin user the cluster-admin role

oc login -u system:admin
oc adm policy add-cluster-role-to-user cluster-admin ocadmin

Finally we can login to OpenShift portal https://gimaster.guardium.notes:7443 or https://console.insights.guardium.notes (ingress  route)2019-12-31_16-48-00Switch to Console2019-12-31_16-49-21and review cluster status2020-01-02_19-44-14

4. IBM Cloud Private installation (all steps on executed on master node)

Latest version of ICP download

Guardium Insights 2.0 supports installation on IBM Cloud Private 3.2.1. The latest version is available on IBM Fix Central2020-01-01_10-39-09

In my case the latest fix pack has subversion 1911. Please download version applicable for OpenShift installation – ibm-cloud-private-rhos prefix

Then upload archive to your master node

Installation preparation

To upload ICP docker images to OpenShift execute:

tar xf ibm-cloud-private-rhos-3.2.1.1911.tar.gz -O | docker load

your gzip archive name can differ from mine2020-01-02_12-21-07

This process takes several minutes

ICP archive can be removed from file system to save the storage space.

Then create ICP installation directory and switch to it:

mkdir /opt/icp-install; cd /opt/icp-install

Installation files are located in the one of just uploaded images – ibmcom/inception

To copy installation files to /opt/icp-install execute command:

docker run --rm -v $(pwd):/data:z -e LICENSE=accept --security-opt label:disable `docker images| grep ibmcom/icp-inception | grep -v grep | awk '{print($1":"$2)}'` cp -r cluster /data

Move cli session to cluster directory and copy OpenShift configuration to it:

cd cluster; cp /etc/origin/master/admin.kubeconfig kubeconfig

We need also update the /opt/icp-install/cluster/hosts file to these settings:

[master]
gimaster.guardium.notes

[etcd]
gimaster.guardium.notes

[nodes]
gimaster.guardium.notes
gidb.guardium.notes
giaux1.guardium.notes
giaux2.guardium.notes

it is node function assignment of our OpenShift cluster (use your node names).

Then we must set /opt/icp-install/cluster/config.yaml file (use your node and domain names)
Place the indentations properly in the yaml document

storage_class: glusterfs-storage-block

openshift:
 console:
  host: gimaster.guardium.notes
  port: 7443
 router:
  cluster_host: icp-console.insights.guardium.notes
  proxy_host: icp-proxy.insights.guardium.notes

ingress_http_port: 3080
ingress_https_port: 3443

password_rules:
- '(.*)'

management_services:
 vulnerability-advisor: disabled
 storage-glusterfs: disabled
 storage-minio: disabled
 istio: disabled
 monitoring: enabled
 metering: enabled
 logging: enabled
 custom-metrics-adapter: disabled
 platform-pod-security: enabled
 multicluster-hub: disabled

cluster_nodes:
 master:
 - gimaster.guardium.notes
 proxy:
 - gimaster.guardium.notes
 management:
 - gimaster.guardium.notes

default_admin_user: icpadmin
default_admin_password:

storage_class: glusterfs-storage-block
preferred cluster storage class for Guardium Insights
console:
 host: gimaster.guardium.notes
 port: 7443
OpenShift console URL (not routed one)
router:
 cluster_host: icp-console.insights.guardium.notes
 proxy_host: icp-proxy.insights.guardium.notes
ICP console and router URL’s – use openshift_master_default_subdomain defined during OpenShift installation
ingress_http_port: 3080
ingress_https_port: 3443
OpenShift ingress ports (80, 443) cannot overlap ICPs’, so define another
cluster_nodes:
 master:
 – gimaster.guardium.notes
 proxy:
 – gimaster.guardium.notes
 management:
 – gimaster.guardium.notes
OpenShift master node will get also 3 main roles for ICP instance (master, proxy and management)
default_admin_user: icpadmin
default_admin_password:
set ICP portal user name and credential (I suggest not to use the same username for OpenShift and ICP because these are different identities in our infrastructure)

ICP installation

ICP installation command:

docker run -t --net=host -e LICENSE=accept -v $(pwd):/installer/cluster:z -v /var/run:/var/run:z -v /etc/docker:/etc/docker:z --security-opt label:disable `docker images| grep ibmcom/icp-inception | grep -v grep | awk '{print($1":"$2)}'` install-with-openshift

It takes several minutes but only absorbs master node.

2020-01-02_14-12-49

Post installation tasks

Check nodes assignment and correct security context for ICP:

kubectl --kubeconfig /etc/origin/master/admin.kubeconfig get nodes
kubectl --kubeconfig /etc/origin/master/admin.kubeconfig patch scc icp-scc -p '{"allowPrivilegedContainer": true} --type=merge'

2020-01-02_14-30-29

Now we can log to ICP console (https://icp-console.insights.guardium.notes defined in config.yaml)2020-01-02_14-44-00Select Overview from menu and confirm that all deployments are healthy2020-01-02_14-58-35

CLI tools installation

GI installation script requires access to cloudctl and helm command line tools which are also useful for ICP administration tasks.

As a root user copy tools from ICP console (use your ICP name) into root home directory:

curl -kLo cloudctl https://icp-console.insights.guardium.notes/api/cli/cloudctl-linux-amd64
curl -kLo helm.tar.gz https://icp-console.insights.guardium.notes/api/cli/helm-linux-amd64.tar.gz

Move binaries to /usr/local/bin

chmod +x cloudctl; mv cloudctl /usr/local/bin
tar xvf helm.tar.gz *helm; mv linux-amd64/helm /usr/local/bin;rm -rf linux-amd64/ helm.tar.gz

2020-01-02_15-34-28

5. Guardium Insights installation

DB2 DataWarehouse storage preparation (execute on db node)

Now we must configure the separate storage for Guardium Insights where audited events will be stored.

I will use the third disk device on my gidb.guardium.notes node

Login to your DB node and use fdisk to create new partition (/dev/sdc in my example)

fdisk /dev/sdc

2020-01-02_15-39-20Create on a new partition XFS file system

mkfs.xfs /dev/sdc1

2020-01-02_15-49-05Create mount point (for example /gi):

mkdir /gi

Configure automatic FS mounting at system startup – edit /etc/fstab and add line:

/dev/sdc1     /gi     xfs    defaults    0  0

Mount new file system

mount -a

2020-01-02_15-51-53

GI software package (master node)

Download Guardium Insights installer from Passport Advantage. The software package archive should contain installation gzip file – SCRTY_GUARDIUM_INSIGHTS_V_2.0_IMG.tar.gz. Copy it to master node and unpack

cd; tar zxf SCRTY_GUARDIUM_INSIGHTS_V_2.0_IMG.tar.gz; rm SCRTY_GUARDIUM_INSIGHTS_V_2.0_IMG.tar.gz

Two GI installation subdirectories should appear in your root home directory:2020-01-02_16-08-00

values.yaml modifications (master node)

These commands should be applied only for initial version of GI to cover some problems:
– incorrect ssh-service redirection
– too high memory and disk requirements according to minimal demands

Execute these commands (put correct master IP address, in my case 192.168.250.150):

sed -i 's/IP: 10.x.x.x/IP: 192.168.250.150/' insights_charts/values.yaml
sed -i 's/memory: 50Gi/memory: 24Gi/' insights_charts/values.yaml
sed -i 's/cpu: 10000m/cpu: 8000m/' insights_charts/values.yaml

GI installation preparation (on master)

Create new project (gi in my case)

oc create namespace gi

oc project gi

Login to ICP and setup helm

cloudctl login -a https://icp-console.insights.guardium.notes -u icpadmin

2020-01-02_16-28-42

Login to local OpenShift docker repository. Docker login requires token authentication which can be extruded from oc whoami command:

oc whoami -t
docker login -u ocadmin -p <token> docker-registry-default.insights.guardium.notes

2020-01-02_16-38-37

Load GI images to OpenShift repository (it takes a while)

cd insights_images/; ./dockerLoad.sh

2020-01-03_09-59-07

Push images to project, takes next few minutes (in my case it is gi namespace):

./dockerPush.sh docker-registry-default.insights.guardium.notes/gi

2020-01-02_17-15-56

Switch to insights_charts directory and execute prepare.sh script

cd ../insights_charts; ./prepare.sh -u root -w gidb.guardium.notes -d /gi/insights

where:
-w – points DB node
-d – directory on an additional data filesystem2020-01-02_17-17-30

GI installation script execution (master node)

Execute install.sh with some options

./installer.sh -s gidb.guardium.notes -d /gi/insights -n gi -c 2Ti -h gi-console.insights.guardium.notes -l true -a icpadmin -e https://icp-console.insights.guardium.notes -i docker-registry.default.svc:5000/gi

where:
-s – DB node specification (why -w defines this same parameter in prepare.sh script?)
-d – directory on an additional data filesystem
-n – OpenShift project name
-c – size of DB storage (2Ti – means 2TB)
-h – the virtual name of Guardium Insights portal
-l – license agreement acceptance
-a – ICP admin user
-e – URL of ICP portal
-i – local project registry – put here private cluster docker URL (docker-registry.default.svc:5000/<namespace>)

Script asks for ICP password:

Missing global.insights.icp.authPassword. ICP admin users password. Enter the global.insights.icp.authPassword:

And requests confirmation of helm command execution:

Do you wish to execute the above helm command with the shown values ?(y/n)

Process takes several minutes. You can monitor it in OpenShift console or wait for deployment of tenant-postdeploy-ready ConfigMap

oc get cm | grep tenant-postdeploy-ready

2020-01-02_17-38-31Finally we can login to Guardium Insights using ICP admin credentials (https://gi-console.insights.guardium.notes):2020-01-02_17-49-32

2020-01-02_17-51-05and configure GI feeders.

Thanks a lot to Devan Shah, I wouldn’t have moved on this article without him.

This is the first version of Guardium Insights still not all options have been documented, so this cookbook should be treated as a supporting resource.
For my part, I will update it in case of important changes or inaccuracies.

6 methods of data access monitoring

Time and again I am asked what and how can be monitored by Guardium? I will try to present in a concise way possible methods and indicate where they are usually used and  pros and cons all of them.

Let’s start with the basic question of what is behind the monitoring?
The purpose of DAM solutions is to identify and analyze every possible vector of access to the data engine in order to:

  • comply with the audit requirements set by the regulations
  • detect unwanted data access activity related to security breaches, attempted theft, destruction or fraudulent actions
  • providing information on threat and risk analysis through the context of access to data
  • prevention of undesirable and harmful effects of un- and intentional activity
  • ensuring accountability of access to data for investigation and corrective actions

These requirements force the DAM solution to access all sessions initiated both remotely and locally using all data engine supported protocols and communication encryption methods.

policy1-1

Access vectors for self-owned and IaaS environments

The diversity of data engines technologies and IT environment setup presents us with a very difficult task and a uniform approach is currently impossible. Therefore, in Guardium it is possible to approach the problem of data monitoring in many ways, which I describe in more detail below.

1. Agent based (S-TAP, Z-TAP)

policy1-2

Agent-based monitoring is the most popular method of collecting information about data activity on the monitored system.

This method requires installation at the operating system level of an agent that has the ability to intercept sessions. In the case of Unix and Linux this is done at the kernel level. Drivers are used on Windows platform. Monitoring of z/OS uses a combination of Query Monitor and analysis of memory areas in DB2 or IMS space. The i5 platform (AS/400) uses a unique method of capturing communication between processes and no need to use journals.
Additionally, an agent on Unix, Linux and Windows platforms allows to track I/O operations in order to monitor file access. Similar functionality is also available on z/OS in the area of monitoring access to various types of data sets.

For most supported data engines it is possible to monitor communication outside the TCP stack: pipe, ipc, shared memory protocols.
Agent implementation does not force termination of encrypted connections in order to capture the context of the session, although not all databases allow this approach (for example MySQL, MariaDB).

All intercepted traffic is sent to the collectors and further processed there (except z/OS and i5 where preprocessing in managed on monitored system level).
The agent allows for selective monitoring by excluding connections (at the IP level) or sessions (dynamically).
Additionally, working at a low level of the operating system allows to block the sessions inline.

Pros

  • Local sessions visibility – only this method allows us to see local sessions with SoD, one of the most important DAM use case.
  • Segregation of Duties (SoD)  – data engine administrator has no influence on monitoring.
  • Very difficult to avoid to be monitored – only database reconfiguration which is changing the available access vectors can lead to miss the activity interception although this kind of situation can be identified and alerted, in some cases Guardium can automatically reconfigure itself even (inspection engine management).
  • Inline blocking – blocking access or preventing specific behavior can be implemented easily but should not be used in the application stream.
  • Easy implemented interception of the encrypted sessions – visibility of encrypted sessions does not require access to certificates and connections termination

Cons

  • Software installation on production systems – can be used only on systems where access to OS is possible (self-owned platform or IaaS), IT departments can block this approach sometimes (especially for outsourced systems), data engine vendor or partner responsible for implementation can try to avoid monitoring this way
  • Influence on the monitored system performance – Guardium agent are very reliable and efficient, CPU load usually does not exceed 2 percent even at peak activity levels. However this aspect can be stopper sometimes on the critical systems or systems where customer has no enough skills (legacy ones) and afraid any change.
  • Strictly defined support of OS platforms – Guardium supports the vast majority of commercial operating systems (Windows, AIX, Solaris, HP-UX, Linux (RedHat/CentOS, Suse, Ubuntu/Debian) – in the case of Linux also for various processor platforms). Support for new OS version usually appears no later than a quarter after release. However, we may encounter niche operating systems or long since not supported versions of OS, which may imply a different method of monitoring.
  • OS and Data Engine administration procedures modification – standard non-encrypted traffic monitoring can be completely transparent for OS and data engine administrators. However, in the case of Linux, where changes in the microkernel are frequent, it is necessary to develop a proper system update process to ensure continuous monitoring. Guardium provides a few scenarios for that which are described here and here.
    Access to encrypted traffic is usually associated with ATAP configuration which requires an additional procedure to update the agent and database. Both of these aspects can block implementation from the IT side sometimes.

Monitoring with S-TAP can be done in several ways:
– KTAP – session capture at the kernel subsystems level
– ATAP – access to decrypted traffic and some protocols through interaction with the data engine
– PCAP – collecting only network packets (outdated)
– EXIT – consumption of the existing data engine API interface for DAM (available for DB2, Informix and Teradata)

The preferred method is EXIT but keep in mind that the data engine administrator has control over its configuration and in this case you need additional configuration change control, for example Guardium CAS.

2. Agentless – SPANpolicy1-3

In this method of monitoring, all sessions are copied at the network level using the port mirroring capability of network switches to the broadcast subnet in which the collector is listening.
Only unencrypted traffic can be analyzed this way.
The configuration of network devices must ensure correct redirection of every possible path to the monitored data engine in order to duplicate session packets.
A single collector provides up to two network interfaces to support this type of event collection.

Technical it is possible to combine agent and agentless methods together, where remote sessions are monitored by SPAN and local ones are intercepted by S-TAP, but it doesn’t make much sense thanks to the very good performance of the agent.

All the time, the choice of a method other than S-TAP agent should be based on technical capabilities, expected benefits, implementation complexity and limitations.

Pros

  • Segregation of Duties – data engine administrator has no influence on monitoring
  • No influence on monitored system – monitoring is completely transparent
  • OS platform independence – OS base of monitored engine is not relevant

Cons

  • No local sessions visibility – monitoring at the network level excludes the possibility of monitoring of local sessions, so the method is used only where access to the operating system is impossible.
  • Additional costs and security controls – ensuring non-repudiation and audit quality requires an additional control layer. In large networks with complex physical and logical infrastructure, redirecting and duplicating traffic involves a large investment in additional network equipment, ensuring high availability and configuration change monitoring. In the case of a public cloud, a more effective solution is to use External STAP, as shown below.
  • Unsupported encryption communication – in this method it is not possible to decrypt traffic, which limits its use (especially in the cloud).
  • No prevention features – is based on passive monitoring and thus no possibility of active session blocking.

3. External S-TAP (E-TAP)policy1-4

The adoption of cloud solutions is inevitable. PaaS and SaaS models are becoming a standard not only for most startups but also for larger and larger organizations.
Migrating critical data beyond the bounds of direct control of its owner or administrator is a challenge that must also be met by Guardium.

External S-TAP allows you to capture sessions inline by redirecting them to it using a load balancer. The intercepted traffic is then sent to the collector.
E-STAP is a docker based container service. Therefore, it can be quickly implemented, scaled and administered using Kubernetes or other tools available to manage dockers on cloud site. The solution itself is not limited to public cloud monitoring only. It can be used effectively in a private cloud or for standard data engines.

Pros

  • Segregation of Duties – data engine administrator has no influence on monitoring
  • OS platform independence – OS base of monitored engine is not relevant
  • Simple implementation on cloud – containerization allows fast and easy management of cloud monitoring.
  • Encrypted traffic visibility and prevention capabilities – despite the illusory similarity to port mirror monitoring, we are dealing with active tracking that implements traffic decryption and preventive actions (probably available from version 11.1)
  • Real alternative to native cloud streams – the critical aspect of the loss of control over the infrastructure is also reflected in the control layer, which is the lack of independent audit of cloud vendor security services. E-STAP allows us to build an independent control mechanism and estimate the quality of services offered by the cloud provider. It is also obvious that monitoring needs to be unified when multiple cloud providers are selected (diversification), where this method can be more convenient than simply streaming events through a cloud data engine.

Cons

  • No local sessions visibility – monitoring at the network level excludes the possibility of monitoring of local sessions but in PaaS and SaaS cases it is not relevant (similar to cloud data engine streaming).
  • Additional costs and security controls – additional use of load balancer and dockered microservice has an impact on the total cost of the monitoring solution. Network configuration control aspects similar to SPAN monitoring.

4. Direct Streaming

policy1-5

The simplest and agentless method of monitoring consisting in direct sending logs of data engine activity to Guardium.
This is the preferred approach by cloud providers but in many cases is not satisfactory due to the lack of segregation of the monitored entity from the monitoring mechanism.
It seems interesting to combine this method with ETAP to verify the quality of the native log :).

Pros

  • Simple and cheap implementation on cloud – usually it is enough to enable the option and specify the recipient of the data, in some solutions free of charge if the events are not sent outside the internal network.
  • All traffic visible (customer access) – a properly structured activity audit should deliver the whole activity on demand (how to prove it?). Another issue is the possibility of its filtering at this stage, which is usually not possible and forces the analysis of the entire traffic on Guardium level even when it is not necessary.

Cons

  • Lack segregation of duties – choosing a cloud solution is always associated with limitations of control and their transfer to the provider. However, wherever possible, we should apply independent security monitoring mechanisms. A good example is the analysis of leakage of personal data protected by law (GDPR or CCPA) and the issue of the responsibility of the controller and data operator. Under most contracts, the cloud provider does not automatically become their operator and it is still necessary to assess the risks and appropriate control measures – whether the non-repudiation, adequacy and all the information is available to us? I leave this question for you to think about 🙂
  • Log absorption – the format of the information provided usually changes much more often than the native engine language, which may affect the quality of the data or its importability to Guardium (however, a close cooperation between IBM and supported cloud vendors should be assumed).
  • No prevention – logging means passive monitoring without the possibility of preventive actions.

5. Logging with STAP

policy1-6

In this approach, the native data engine logs are consumed by the S-TAP and not sent directly to the collector as was the case with the direct stream.
They are usually available as replacement solutions for S-TAP where event capture is complex or preventive actions are not required.
An example is a method of monitoring Oracle engine through communication with Oracle Connection Manager or Cloudera where logs are taken from Kafka node instead of installing an agent on each data node.

Usually S-TAP does not have to be installed directly on the environment where logs are stored, but only in the place where network access to log is possible.

Pros

  • Simple implementation – eliminates technical and administrative problems related to monitoring through the use of native audits.
  • All traffic visible – as in the case of streaming, the log should contain all activities but also enable its filtration.

Cons

  • Lack segregation of duties – the same problems as with the streaming.
  • Additional costs and security controls – the event logging function may be associated with additional licenses, architecture or infrastructure changes.
  • Influence on performance – native logging very often is associated with a large impact on the performance of the data engine which forces the filtration of events to the necessary minimum.
  • No prevention – logging means passive monitoring without the possibility of preventive actions.

6. Universal feed, custom tables

policy1-7

Forgotten and potentially very useful method of supplying Guardium with events from unsupported data engines.
Guardium provides several data absorption methods, including native S-TAP (protobuff) agent communication protocol.
Unfortunately, poor documentation in this area blocks the development of these techniques.
Here is a link to a more detailed description.

Pros

  • Support of unsupported – the ability to integrate analysis and reporting of data access for niche and industry-specific solutions is a value that cannot be overestimated.

Cons

  • Difficult implementation – the aforementioned lack of full API documentation blocks the development of this functionality, but a change in this area is expected, about which a few words at the end of this article.
  • No prevention – Universal Feed implements one-way communication so we are talking about simple log streaming or consumption.

Looking to the future

Will the consolidation of events from the collectors in GBDI or in just announced Guardium Insights bring the significant changes in the techniques of event interception?
Probably not because the “data lake” is only a place where standardized information is processed.
At most, we can expect the possibility of direct transmission of events without the use of collectors rather than new mysterious methods of collecting and delivering them.
From a different perspective, data standardization and ease of extending their schemes in JSON structures, REST-based communication and containerization may facilitate the extension of supported platforms through communities, business partners or Guardium’s clients themselves.

The mechanisms presented are, in my opinion, proof of the good preparation of Guardium for the revolution that is taking place, known as the ‘journey to the cloud’*.

* 😉

 

GDE installation guide

Guardium Data Encryption installation cookbook.

– Software Access – 00:12
– Appliance setup – 04:30
– Failover setup – 10:29
– User management – 14:58
– Agent installation – 20:54
– Web certificates management – 28:57
– Backup & Restore – 32:42
– Host assignment – 38:38
– DSM upgrade – 42:42
– Agent update – 47:56
– Agent deinstallation – 50:21

Published on GuardiumNotes Youtube channel – GDE installation guide

 

Guardium enhancements review 10.1.3-10.6

In addition to simplifying the process of defining report content and its appearance, we can immediately build a datamart and place it in the indicated dashboard.
New Report builder is also available to manage reports based on Custom domains.

The rule setup screen scared every new user. Identifying the correct field and understanding what was defined required a lot of experience.
Simplifying the process of rule creation is the main value of changes in the new version of Policy Builder. The division of criteria into three categories (session, SQL and other) and displaying only active ones allows you to understand the logic of the rule in the blink of an eye.

Edit Rule screen

The management of the order of rules and their interaction has also been simplified. In one window we can manipulate, copy and import rules in the policy and quickly determine whether it is necessary to analyze a given situation by several roles.

Rules in policy

Group Builder (10.5)

Managing groups is one of the basic administrative tasks that allow you to properly monitor and report activity on protected resources. In the new Group Builder, we can find a specific group in a transparent way and identify its meaning in our installation (where the group is used, source of collected data, how many records it contains).

Group list

The most anticipated improvement in group management is the ability to easily filter and modify content. Especially after the Policy Builder version 10.6 upgrade, we get one coherent interface in which managing reference data is a simple task.

Group content

Agent Management by GIM (10.1.4)

The new GIM management interface has been described in my K-TAP video and significantly increases the agent management effectiveness.

In version 10.6 we will find some improvements like preview of installed modules, identification of lack of communication with the agent during configuration of the module, update of progress information.


GIM is now a great tool for managing hundreds of agents without need to get direct access to managed system.

Simple Agent Deployment (10.1.3)

The ability to pre-install the GIM agent in large environments (Listener mode) is extremely valuable. The new “Deploy Monitor Agents” feature also reduces the time it takes to install agents.

Simple Agent Deployment

It is useful tool, however, has a limitation related to the inability to indicate the SSL certificates if we decide to use our own. 😦

I assume that we will have possibility to install other modules (CAS, FAM) and create configuration templates soon.

Simple Compliance Monitoring (10.1.3)

This dashboard provides possibility to setup and present results in the unified view the status of our datasets against our compliance requirements (like GDPR, PCI, SOX).

Compliance Monitoring

Sound good but especially for compliance the polices, reports and audit have to be customized case by case.

The configuration requires customization of classification process, monitoring policy and reports to be real compliance dashboard. I like it but still improvements needed to have the possibility to:

  • assign custom policy instead of the modification permanently assigned
  • assign custom classification process instead the modification the assigned one
  • define list of reports assigned to compliance dashboard

This dashboard assumes that compliance settings have global character for all customer data sources what cannot be true in many cases.

Guardium Application Ecosystem

The ability to create additional functionalities for the system by users or business partners without the need to integrate with the vendor’s development process is a genius idea. The success of the App Exchange platform in QRadar (IBM SIEM solution) resulted in its implementation in other ones, including the Guardium.

App Exchange Portal

The developed application is isolated from the system itself within the container (Docker). The programming platform is Flask – Python based web microframework.
Communication with the system takes place through a rich Guardium REST API that implements most of the functions available in the standard API available through cli.

Application management in the Guardium itself amounts to installing (file import), determining access for roles and launching the developed application.
The entire ecosystem can be fully controlled from the Central Manager level.

Application Management on CM

Creating an application does not require a lot of programming experience. Technologies used in the solution allow you quickly and efficiently implement “missing” system functions.
However, it should be remembered that each application operates in an independent container, therefore using them imposes more requirements on the appliance – a minimum 32 GB of RAM (if Quick Search is switched on).

Here my first Guardium application screen (I am working on separate article focused on this only)

GN Tools Application

Cloud support

DAM solutions appeared on the market when the idea of ​​data storage in the cloud was only the subject of scientific dissertations. Within a dozen or so years, the situation has changed dramatically and the requirement to support processing outside local data centers becomes a requirement and not an addition to the solution.
Each subsequent version of Guardium brought something new in this matter.
Guardium architecture allows to implement it inside IaaS services where the user has access to the operating system and it is possible to install S-TAP. However, data transmission from the cloud to local appliance was ineffective so now we have access to pre-installed appliances in the Amazon, Azure and Softlayer clouds (10.1.3).
However the real challenge for monitoring solutions is the SaaS infrastructure where the service provider administers the data silo without the possibility of installing additional services.
The solution can be the consumption the native audit logs provided by the service or engine by DAM system. What has been introduced for AWS RDS Oracle 11 and 12 in Guardium 10.1.4. However it stresses another problem of support a completely new format of session information for something which works perfectly by parsing SQL syntax. Logs format, scope of information can be changed by provider without possibility to control this stream and elevates huge problem for any DAM vendor.

So in the latest Guardium release we have got the External S-TAP. The proxy solution which assumes that sessions to unmanaged data nodes will be redirected to new Guardium service by load balancer. This approach simplifies implementation because we do not need analyze logs and base on standard session interception.

The External S-TAP is distributed as a Docker solution and can be downloaded from dockerhub

2018-12-20_16-00-43

The customer can decide how will be it implemented:

  • External S-TAP on premise and reroutes all request to cloud service
  • External S-TAP as cloud service and reroutes request inside cloud infrastructure
  • External S-TAP on premise intercepts traffic to local databases without necessity to install S-TAP on monitored system

This kind of approach opens possibility to use it in different situation not only for cloud services but also as a platform to extend functionality for new data silo based on self-developed parsers.

This initial release supports MSSQL and Oracle on premise and AWS Oracle and Azure MSSQL for cloud. I believe that very soon this technology will be opened to support more platforms and armored by SDK.

Platform support enhancement

  • New releases brought support for new versions of Oracle, MSSQL, Teradata, MongoDB, Cassandra, MariaDB, MemSQL
  • Simplification of Cloudera Navigator monitoring with auditing events directly from Kafka node
  • Vulnerability Assessment supports Cloudera now
  • The FAM is MS Office documents aware and reduces noise related to I/O operation related to them
  • SharePoint support – 3rd party agent for SharePoint monitoring and data classification
  • NAS support – 3rd party agent to classify and monitor access to data on Hitachi, NetApp, EMC, Dell EMC devices
  • Simplification of ATAP management (including possibility to avoid service restart in case of STAP upgrade)
  • Teradata monitoring based on EXIT

Other enhancements

VA multi-threading (10.6)

Now many instances of classification and vulnerability assessment tasks can be executed at this same time – however be aware that it requires a lot of resources. So I suggest setup the separated aggregator which will be dedicated to this job.

CLI and public key authentication (10.5)

Finally we can login to cli using certificates instead of simple password based authentication

Session-level policies (10.6)

Standard policy evaluation requires SQL body identification (command, objects) to be available for condition inside rule even the decision process does not require this kind of analysis because the session information only is sufficient (user, ip address, network protocol).

To speed up policy evaluation in these situations now we have completely new type of polices – session-level.Session Level PolicyWe can create policy in GUI or cli using special “Session Level Rule Language Grammar”.

The policy focuses only on session information

Session-Level policy rule

and uses completely new set of actions including information transformation (can be very tricky).

Session-level rule actions

The session-level policy is still evaluated on collector (by sniffer) but the decision about logging, switching to open mode from close one can be returned much more faster.

There is possibility to install together standard and session-level policy.

New Firewall  mode (10.6)

The new FIREWALL_DEFAULT_STATE=2 value integrates the decision flow with WATCH and UNWATCH flags from Session-Level policies. So the decision about excluding of application sessions from blocking analysis approach can be made faster and smarter way.

Enterprise Load Balancer

ELB update allows to create failover groups to switch STAP only in the defined scope of appliances. It provides solution for large environments with geographically scattered Guardium domains.

Summary:


Last four Guardium releases introduce a lot of new functionalities to make this system simpler in use, be prepared for customer transition into a cloud and create system open to extend in the directions stressed by their users.

 

GDP Policies – Part One

Guardium Data Protection Policy tutorial.

Not only basic stuff inside, should be helpful also for experienced Guardium administrators.

In Part One:

  • 00′:21” – Guardium Policy basics  – selective policy, rules, conditions and actions
  • 04′:46” – Simple policy – create, install, check
  • 08′:50” – Access control – technical account using inappropriate access vector
  • 13′:40” – Access control – identification access from unapproved workstation
  • 18′:30” – Access control – alerting the access of unknown user and DDL execution
  • 22′:43” – What exactly DB Name is?
  • 26′:25” – Access control – alerting DML’s in the application schema
  • 30′:48” – Masking sensitive information in the reports and alerts

The tutorial published on Guardiumnotes youtube channel  – GDE Policies – Part One

Public Key Authentication with SSH – PuTTY

Guardium 10.5 allows authenticate on cli accounts by public keys.

The configuration is simple but is not well described at this moment in the standard documentation so I have decided to publish this short post.

I present here the most popular case where the SSH access is based on PuTTy.

Step 1 – PuTTy configuration

I suggest use the puttygen.exe to create SSH keys (can be downloaded as a supporting tool from PuTTy home page – link)

I push Generate button to create keys. The default settings points the RSA algorithm and 2048 bits key length. For production purposes I suggested use the longer keys.

2018-05-04_16-58-47

Now we need to save the private key in the place available for PuTTY. I strongly suggest provide the key passphrase to secure private key, it will be inserted for any session opened with the generated here the keys.2018-05-04_17-01-51

Then we need also save somewhere the public key.2018-05-04_17-04-24

Step 2 – Collector setup

This actions have to be repeated on each appliance which will support the authentication using the public key infrastructure.

The appliance configuration requires the setup of appliance keys and import public keys for all Guardium administrators which are allowed to login on cli account. Of course the clue of the public key infrastructure is the keys uniqueness per administrator what does allow us to control access even for shared accounts.

Action 1 – Appliance keys generation

From cli (still logged using password) execute command:

show system public key cli

The output will inform that there is no keys on the appliance and they will be generated.2018-05-04_15-44-51

The message will also display just generated public key. In case of PuTTy configuration we do not need to copy it.

There is also possible deletion of existing keys using command:

store system public key reset

The appliance keys removal will stop access to system using public key infrastructure for all registered users. To restore configuration after appliance keys deletion we need execute again the command:

show system public key cli

Action 2 – Client public key import

The import of user public keys is possible by use command:

store system public key authorized

The command will expect the client public key in Open SSH format inserted in one line:

ssh-rsa <key> <comment label>

but the exported public key from Step 1 has been stored by puttygen in the standard format and should be reformatted to supported by Guardium one.

2018-05-04_17-44-59 So in this case the command which registers my PuTTy client on the appliance looks like that:

2018-05-04_17-59-35

We can review the list of registered client using command:

show system public key authorized

To remove particular client access we can use command:

delete system public key authorized

Step 3 – Putty session configuration

Now we can configure our PuTTY to use the generated keys. I have created new session (MySSHPKI) to login on appliance as cli user.

2018-05-04_18-31-47

and I set the location of my private key inside Connection/SSH/Auth configuration view and saved the session settings.

2018-05-04_19-02-06

Step 4 – Connection test

The ssh connection asked me for my private key passphrase and I will able finally login to the appliance without Guardium password.

2018-05-04_19-06-51

I suggest this kind of configuration for all production systems. It allows control access to system and quickly remove access to Guardium infrastructure by removing the public key from the list of accepted on the appliance.

Still configuration has to be managed on each appliance separately and there is not internal audit trail for key used during cli connections but I believe that these improvements will be implemented soon.