GDE installation guide

Guardium Data Encryption installation cookbook.

– Software Access – 00:12
– Appliance setup – 04:30
– Failover setup – 10:29
– User management – 14:58
– Agent installation – 20:54
– Web certificates management – 28:57
– Backup & Restore – 32:42
– Host assignment – 38:38
– DSM upgrade – 42:42
– Agent update – 47:56
– Agent deinstallation – 50:21

Published on GuardiumNotes Youtube channel – GDE installation guide

 

Advertisements

Guardium enhancements review 10.1.3-10.6

In addition to simplifying the process of defining report content and its appearance, we can immediately build a datamart and place it in the indicated dashboard.
New Report builder is also available to manage reports based on Custom domains.

The rule setup screen scared every new user. Identifying the correct field and understanding what was defined required a lot of experience.
Simplifying the process of rule creation is the main value of changes in the new version of Policy Builder. The division of criteria into three categories (session, SQL and other) and displaying only active ones allows you to understand the logic of the rule in the blink of an eye.

Edit Rule screen

The management of the order of rules and their interaction has also been simplified. In one window we can manipulate, copy and import rules in the policy and quickly determine whether it is necessary to analyze a given situation by several roles.

Rules in policy

Group Builder (10.5)

Managing groups is one of the basic administrative tasks that allow you to properly monitor and report activity on protected resources. In the new Group Builder, we can find a specific group in a transparent way and identify its meaning in our installation (where the group is used, source of collected data, how many records it contains).

Group list

The most anticipated improvement in group management is the ability to easily filter and modify content. Especially after the Policy Builder version 10.6 upgrade, we get one coherent interface in which managing reference data is a simple task.

Group content

Agent Management by GIM (10.1.4)

The new GIM management interface has been described in my K-TAP video and significantly increases the agent management effectiveness.

In version 10.6 we will find some improvements like preview of installed modules, identification of lack of communication with the agent during configuration of the module, update of progress information.


GIM is now a great tool for managing hundreds of agents without need to get direct access to managed system.

Simple Agent Deployment (10.1.3)

The ability to pre-install the GIM agent in large environments (Listener mode) is extremely valuable. The new “Deploy Monitor Agents” feature also reduces the time it takes to install agents.

Simple Agent Deployment

It is useful tool, however, has a limitation related to the inability to indicate the SSL certificates if we decide to use our own. 😦

I assume that we will have possibility to install other modules (CAS, FAM) and create configuration templates soon.

Simple Compliance Monitoring (10.1.3)

This dashboard provides possibility to setup and present results in the unified view the status of our datasets against our compliance requirements (like GDPR, PCI, SOX).

Compliance Monitoring

Sound good but especially for compliance the polices, reports and audit have to be customized case by case.

The configuration requires customization of classification process, monitoring policy and reports to be real compliance dashboard. I like it but still improvements needed to have the possibility to:

  • assign custom policy instead of the modification permanently assigned
  • assign custom classification process instead the modification the assigned one
  • define list of reports assigned to compliance dashboard

This dashboard assumes that compliance settings have global character for all customer data sources what cannot be true in many cases.

Guardium Application Ecosystem

The ability to create additional functionalities for the system by users or business partners without the need to integrate with the vendor’s development process is a genius idea. The success of the App Exchange platform in QRadar (IBM SIEM solution) resulted in its implementation in other ones, including the Guardium.

App Exchange Portal

The developed application is isolated from the system itself within the container (Docker). The programming platform is Flask – Python based web microframework.
Communication with the system takes place through a rich Guardium REST API that implements most of the functions available in the standard API available through cli.

Application management in the Guardium itself amounts to installing (file import), determining access for roles and launching the developed application.
The entire ecosystem can be fully controlled from the Central Manager level.

Application Management on CM

Creating an application does not require a lot of programming experience. Technologies used in the solution allow you quickly and efficiently implement “missing” system functions.
However, it should be remembered that each application operates in an independent container, therefore using them imposes more requirements on the appliance – a minimum 32 GB of RAM (if Quick Search is switched on).

Here my first Guardium application screen (I am working on separate article focused on this only)

GN Tools Application

Cloud support

DAM solutions appeared on the market when the idea of ​​data storage in the cloud was only the subject of scientific dissertations. Within a dozen or so years, the situation has changed dramatically and the requirement to support processing outside local data centers becomes a requirement and not an addition to the solution.
Each subsequent version of Guardium brought something new in this matter.
Guardium architecture allows to implement it inside IaaS services where the user has access to the operating system and it is possible to install S-TAP. However, data transmission from the cloud to local appliance was ineffective so now we have access to pre-installed appliances in the Amazon, Azure and Softlayer clouds (10.1.3).
However the real challenge for monitoring solutions is the SaaS infrastructure where the service provider administers the data silo without the possibility of installing additional services.
The solution can be the consumption the native audit logs provided by the service or engine by DAM system. What has been introduced for AWS RDS Oracle 11 and 12 in Guardium 10.1.4. However it stresses another problem of support a completely new format of session information for something which works perfectly by parsing SQL syntax. Logs format, scope of information can be changed by provider without possibility to control this stream and elevates huge problem for any DAM vendor.

So in the latest Guardium release we have got the External S-TAP. The proxy solution which assumes that sessions to unmanaged data nodes will be redirected to new Guardium service by load balancer. This approach simplifies implementation because we do not need analyze logs and base on standard session interception.

The External S-TAP is distributed as a Docker solution and can be downloaded from dockerhub

2018-12-20_16-00-43

The customer can decide how will be it implemented:

  • External S-TAP on premise and reroutes all request to cloud service
  • External S-TAP as cloud service and reroutes request inside cloud infrastructure
  • External S-TAP on premise intercepts traffic to local databases without necessity to install S-TAP on monitored system

This kind of approach opens possibility to use it in different situation not only for cloud services but also as a platform to extend functionality for new data silo based on self-developed parsers.

This initial release supports MSSQL and Oracle on premise and AWS Oracle and Azure MSSQL for cloud. I believe that very soon this technology will be opened to support more platforms and armored by SDK.

Platform support enhancement

  • New releases brought support for new versions of Oracle, MSSQL, Teradata, MongoDB, Cassandra, MariaDB, MemSQL
  • Simplification of Cloudera Navigator monitoring with auditing events directly from Kafka node
  • Vulnerability Assessment supports Cloudera now
  • The FAM is MS Office documents aware and reduces noise related to I/O operation related to them
  • SharePoint support – 3rd party agent for SharePoint monitoring and data classification
  • NAS support – 3rd party agent to classify and monitor access to data on Hitachi, NetApp, EMC, Dell EMC devices
  • Simplification of ATAP management (including possibility to avoid service restart in case of STAP upgrade)
  • Teradata monitoring based on EXIT

Other enhancements

VA multi-threading (10.6)

Now many instances of classification and vulnerability assessment tasks can be executed at this same time – however be aware that it requires a lot of resources. So I suggest setup the separated aggregator which will be dedicated to this job.

CLI and public key authentication (10.5)

Finally we can login to cli using certificates instead of simple password based authentication

Session-level policies (10.6)

Standard policy evaluation requires SQL body identification (command, objects) to be available for condition inside rule even the decision process does not require this kind of analysis because the session information only is sufficient (user, ip address, network protocol).

To speed up policy evaluation in these situations now we have completely new type of polices – session-level.Session Level PolicyWe can create policy in GUI or cli using special “Session Level Rule Language Grammar”.

The policy focuses only on session information

Session-Level policy rule

and uses completely new set of actions including information transformation (can be very tricky).

Session-level rule actions

The session-level policy is still evaluated on collector (by sniffer) but the decision about logging, switching to open mode from close one can be returned much more faster.

There is possibility to install together standard and session-level policy.

New Firewall  mode (10.6)

The new FIREWALL_DEFAULT_STATE=2 value integrates the decision flow with WATCH and UNWATCH flags from Session-Level policies. So the decision about excluding of application sessions from blocking analysis approach can be made faster and smarter way.

Enterprise Load Balancer

ELB update allows to create failover groups to switch STAP only in the defined scope of appliances. It provides solution for large environments with geographically scattered Guardium domains.

Summary:


Last four Guardium releases introduce a lot of new functionalities to make this system simpler in use, be prepared for customer transition into a cloud and create system open to extend in the directions stressed by their users.

 

GDP Policies – Part One

Guardium Data Protection Policy tutorial.

Not only basic stuff inside, should be helpful also for experienced Guardium administrators.

In Part One:

  • 00′:21” – Guardium Policy basics  – selective policy, rules, conditions and actions
  • 04′:46” – Simple policy – create, install, check
  • 08′:50” – Access control – technical account using inappropriate access vector
  • 13′:40” – Access control – identification access from unapproved workstation
  • 18′:30” – Access control – alerting the access of unknown user and DDL execution
  • 22′:43” – What exactly DB Name is?
  • 26′:25” – Access control – alerting DML’s in the application schema
  • 30′:48” – Masking sensitive information in the reports and alerts

The tutorial published on Guardiumnotes youtube channel  – GDE Policies – Part One

Public Key Authentication with SSH – PuTTY

Guardium 10.5 allows authenticate on cli accounts by public keys.

The configuration is simple but is not well described at this moment in the standard documentation so I have decided to publish this short post.

I present here the most popular case where the SSH access is based on PuTTy.

Step 1 – PuTTy configuration

I suggest use the puttygen.exe to create SSH keys (can be downloaded as a supporting tool from PuTTy home page – link)

I push Generate button to create keys. The default settings points the RSA algorithm and 2048 bits key length. For production purposes I suggested use the longer keys.

2018-05-04_16-58-47

Now we need to save the private key in the place available for PuTTY. I strongly suggest provide the key passphrase to secure private key, it will be inserted for any session opened with the generated here the keys.2018-05-04_17-01-51

Then we need also save somewhere the public key.2018-05-04_17-04-24

Step 2 – Collector setup

This actions have to be repeated on each appliance which will support the authentication using the public key infrastructure.

The appliance configuration requires the setup of appliance keys and import public keys for all Guardium administrators which are allowed to login on cli account. Of course the clue of the public key infrastructure is the keys uniqueness per administrator what does allow us to control access even for shared accounts.

Action 1 – Appliance keys generation

From cli (still logged using password) execute command:

show system public key cli

The output will inform that there is no keys on the appliance and they will be generated.2018-05-04_15-44-51

The message will also display just generated public key. In case of PuTTy configuration we do not need to copy it.

There is also possible deletion of existing keys using command:

store system public key reset

The appliance keys removal will stop access to system using public key infrastructure for all registered users. To restore configuration after appliance keys deletion we need execute again the command:

show system public key cli

Action 2 – Client public key import

The import of user public keys is possible by use command:

store system public key authorized

The command will expect the client public key in Open SSH format inserted in one line:

ssh-rsa <key> <comment label>

but the exported public key from Step 1 has been stored by puttygen in the standard format and should be reformatted to supported by Guardium one.

2018-05-04_17-44-59 So in this case the command which registers my PuTTy client on the appliance looks like that:

2018-05-04_17-59-35

We can review the list of registered client using command:

show system public key authorized

To remove particular client access we can use command:

delete system public key authorized

Step 3 – Putty session configuration

Now we can configure our PuTTY to use the generated keys. I have created new session (MySSHPKI) to login on appliance as cli user.

2018-05-04_18-31-47

and I set the location of my private key inside Connection/SSH/Auth configuration view and saved the session settings.

2018-05-04_19-02-06

Step 4 – Connection test

The ssh connection asked me for my private key passphrase and I will able finally login to the appliance without Guardium password.

2018-05-04_19-06-51

I suggest this kind of configuration for all production systems. It allows control access to system and quickly remove access to Guardium infrastructure by removing the public key from the list of accepted on the appliance.

Still configuration has to be managed on each appliance separately and there is not internal audit trail for key used during cli connections but I believe that these improvements will be implemented soon.

KTAP installation on Linux – video

This video covers most KTAP installation challenges on Linux platform.

Chapters timeline:

  1. Introduction 0’00”
  2. STAP installation in new GIM “Setup by Client” application 0’30”
  3. KTAP initialization problems identification 3’08”
  4. Local KTAP compilation 5’58”
  5. Installation in Combo mode 8’54”
  6. KTAP installation flow 11’01”
  7. TEST & PROD scenario – STAP installation 11’58”
  8. TEST & PROD scenario – STAP upgrade 15’15”
  9. TEST & PROD scenario – Linux kernel upgrade 16’55”

Link: https://www.youtube.com/watch?v=77QQT7Rjlc0

Summary:

KTAP initialization on Linux is challenging task for new Guardium beginners.
The new portal update in GIM 10.1.4 make these tasks more clear and simple.

Video presents the most common situations related to KTAP in STAP life-cycle management on Linux platform.

Remarks:

Please notice that custom modules can be installed on machines where GIM client flag GIM_ALLOW_CUSTOMED_BUNDLES is set 1.

So in all scenarios where 8XX modules are applies please assume this setting.

 

 

Guardium Reports Platform understanding (2)

Full SQL and SQL monitoring, deeper view on Access domain

For better understanding SQL entity we need to describe a little bit deeper the logging actions in Guardium policy.
My audit policy (selective audit trail) contains two rules.
2017-10-09_12-33-50It will log activity of syntaxuser1 using LOG action (LOG ONLY) and other traffic will be audited with details – LOG FULL DETAILS action.
I connected to postgreSQL database two times as a test and syntaxuser1
2017-10-09_13-08-34and these sessions are visible (right) but report based on Full SQL entity does not contain syntaxuser1 activity (left).
The reason is simple and understanding of this is very important to create accurate database monitoring policy and reports. The LOG ONLY action logs SQL constructs and does not audit full SQL body executed inside session.

So what does exactly LOG ONLY log?!

The LOG ONLY (it is also default action for non selective audit trail policies!) removes SQL parameter values from SQL body. For instance 3 SQL’s:

SELECT * FROM table WHERE columnX='value1'
SELECT * FROM table WHERE columnX='value2'
SELECT * FROM table WHERE columnX='value3'

are described as a one SQL construct

SELECT * FROM table WHERE columnX='?'

so audited activity based on LOG ONLY action allows identify syntax but it is not possible to present full body (if SQL contains parameters).
The main purpose of LOG ONLY action use is the meaningful decrease of disk space consumption by audited traffic because we do not need store each SQL and put only reference (Construct ID) to known by collector SQL constructs stored in SQL entity.

It is a good time to introduce very important entity – Access Period. Independently to FULL SQL flow (described in part 1) Guardium stores audited activity inside the hourly based sets named periods – we can visualize them as data partitions. Now we can discuss sense of this kind approach but it was historic decision (more that 10 years ago) based among others on cost of storage and CPU utilization.

Periods describe the all audited traffic on hour basis and simplify data partitioning and point executed in this timeframe SQL’s by Instance ID and Construct ID keys . So we can present data inside entities this way
gn23Data flow in 5 main entities:

  • policy makes decision to log activity (LOG ONLY or LOG FULL DETAILS)
  • if SQL is related to new session – new Session ID is registered and Access ID is attached to it or new connection profile is registered
  • system checks – is Session ID registered in the current period (current hour)?
    • False – new Instance ID is created in Access Period entity
  • if LOG ONLY action is used
    • the SQL is “anonymized” – parameter values are replaced by question mark
    • system checks existence of SQL construct in SQL Entity
      • False – new Construct ID is registered in SQL Entity
    • new record (access) is attached to current period (partition) in the Access Period entity – Access ID, relations to SQL (Construct ID) and Session (Session ID)
  • if LOG FULL DETAILS action is used
    • the SQL is registered in FULL SQL with reference to Session ID
    • SQL is “anonymized” and registered in Access Period (Instance ID) this same way like described for LOG ONLY action
    • Record in FULL SQL stores reference to Instance ID in Access Period

We should be aware some limitations if LOG ONLY action is used to audit session:

  • data are stored without parameter values
  • we cannot identify exact time of SQL execution, we can estimate time by reference to:
    • Session timestamps – between Session Start and Session End
    • Access Period – SQL execution inside partition, between Period Start and Period End
    • Access Period Timestamp – last execution of particular SQL construct inside period instance
  • SQL’s from one session can be located in many periods (partitions) if session involves many hours
  • Both logging actions can be used inside policy to audit activity from this same session (it is powerful) but it can lead to incorrect conclusions if we base on FULL SQL report only
  • The Period Start is used as a timestamp for Access Period Entity (the Timestamp field has not important value)

It should be also stressed that LOG FULL DETAILS action stores SQL in both entities Full SQL and SQL

So, from theory to practice 🙂

Example 1 – No space on disk, no data in reports

It happens when we use LOG ONLY action in our policies and we try to review activity in the report based on FULL SQL entity
2017-10-09_18-28-05
SQL counter identifies thousands constructs (report based on SQL entity) but SQL syntax report is empty (based on FULL SQL) – my audit policy uses LOG ONLY action. If you log events using LOG ONLY action somewhere you should not report data based on FULL SQL to report them.

Example 2 – Where is timestamp for SQL entity?

I created a query based on main entity – SQL
2017-10-09_18-48-09
and you should notice that there is no timestamp inside the available in SQL entity fields. What does it mean? Can we create report base on it and use time period specification for results?
Yes, we can, because query gathers timestamp from the closest (direct) relation if it does not exists in the main entity.
The direct relation for SQL entity is Access Period which use Period Start field as timestamp. It has a big influence on result 🙂

I connected to database two times and executed simple query in each session
2017-10-09_19-08-54then I tried to display my activity related to second session only (base on Start Date after 19:18:00)
2017-10-09_19-10-35
and both sessions are visible. The explanation is simple – timestamp for this report bases on Period Start field what for the input value 19:18 defines the period 19:00-20:00 (7-8 am) where two Test user sessions happened.
There is no possibility to granular time different way (more in Example 5) because it is main entity dependent. How to display SQL’s belonging only to the second session? – we can use Session ID for example as a filter
2017-10-09_19-26-31I put Session ID of interesting me session in the added filter and “voila”2017-10-09_19-28-12

Example 3 – I do not see part of my SQL’s – situation 1

This time I executed 6 SQL’s but only two of them are displayed in my report based on SQL entity.
2017-10-09_22-16-45What I pointed before, the SQL entity stores constructs instead of FULL SQL body so SELECT 1, SELECT 2 and SELECT 3 are visible here as SELECT ? and 3 executions of SELECT now() also point only one SQL construct. Please notice also that Timestamp in Access Period entity points the time closest to the last execution of particular SQL construct.
Does it mean that we cannot identify the exact number of executed SQL’s if LOG ONLY action is used? – of course, we can 🙂 The number of occurrences of constructs are stored in Access Period and we can refer to it from SQL entity using entity counter (Add Count)
2017-10-09_22-29-07Now we have full information that my session contained two SQL constructs and each of them was executed three times.
The SQL entity does not store execution timestamp so order of executed constructs is unknown.

Example 4 – I do not see part of my SQL’s – situation 2

This time I executed 8 SQL’s inside session
2017-10-10_09-39-26and only four appear in the report. You should notice that my session lifetime covers 2 periods (08:00-09:00 and 09:00-10:00) but report time range refers only to the second one. In Session Start column we have information when session started and my period reference has to point to it if I would like to receive full session statistics
2017-10-10_09-51-07

Example 5 – One hour granularity is not foxy

Guardium allows decrease the default access period time granularity from one hour to 1 minute even.
2017-10-21_10-55-31Do not forget Apply changes and Restart IE’s before. Here the Logging Granularity has been set to 10 minutes what is visible in the report below
2017-10-21_11-04-43

Example 6 – SQL or Access Period as main entity?

The SQL entity as a main entity is used only when information about construct body is needed.  If we focus on quantitative analysis and interesting in the user behave the Access Period domain is much more efficient. Access Period is also the timestamp reference entity for very useful entities like Command and Object (I will focus on them in the next article about Guardium reporting)

Example 7 – How to see all SQL’s

It is common situation that Guardium policy mixes LOG ONLY and LOG FULL DETAILS actions inside rules. So only part of activity can be reported using FULL SQL entity. We do not need create complicated reports to summarize and analyze this diversified type of auditing because each fully monitored SQL is also stored inside SQL entity.
2017-10-21_11-28-35It should be now clear that FULL SQL refers to Access Period using Instance ID key and indirectly we can identify executed Construct ID.

Please remember that any audited activity in Guardium is always visible inside Access Period and SQL entities.
Any quantitative analysis should relies on them especially when not only LOG FULL DETAILS action is used inside policy rules.
The report data extraction works much more efficient if we use Access Period instead of heavy queries on FULL SQL entity.