Configuring Profiling Settings¶

In addition to using your Rule Set to determine the inventory of what to profile, a Profiling job uses Expressions to identify your sensitive data. You can add regular expressions to be used by Profiler Sets to the Profiler Settings.

To display the Profiler Settings, click on the Settings tab and select Profiler on the left-hand side of the page.

The Profiler Settings screen displays Expressions along with their Domain, Expression text, Expression Name, Owner, and Expression profiling Level.

Note

Column and data level expressions are case insensitive.

To add an Expression¶

Click Add Expression at the top of the Profiler screen.

Select a Domain from the Domain dropdown.
- Domains are used by Profiling jobs to determine the masking Algorithm to apply to your sensitive data. When an Expression is matched, the Profiling job will associate the specified Domain to the sensitive data. The Masking Engine comes out of the box with over 30 pre-defined Domains. Domains can be added, edited, and deleted from the Settings Domains screen.
Enter the following information for the Expression:
- Expression Name— The name used to select this expression as part of a Profiler Set.
- Expression Text— The regular expression used to identify sensitive data.
Select an Expression Level for the Expression:
- Column Level— To identify sensitive data based on column names.
- Data Level— To identify sensitive data based on data values, not column names.
When you are finished, click Save.

To edit a saved Expression, click the Edit icon to the right of the Expression.

To delete an Expression¶

Click the Delete icon to the far right of the name.

Profile Type Expressions¶

Matches with column level expressions can be limited by data type using Type Expressions. Type expressions are currently only available through the API. To create a type expression, use the profile-type-expressions API endpoint. Type Expressions consist of a user-chosen name, a data type, an optional minimum field length and a domain to which the constraint applies. Supported data types are: String, Number, Date, Binary. Each type represents a number of native datatypes in the database. For example, VARCHAR2, NVARCHAR and TEXT fields will all be recognized as being String types for the purposes of profiling. The minDataLength field limits matches to fields that are at least as long as the minDataLength. Finally, the domainName field specifies which domain the constraint applies to.

For example, if we want to ensure that column level profiling only identifies a column with the FIRST_NAME domain if the column is a string type and has a capacity of at least 5 characters, we could add the type constraint shown below.

{
  "domainName": "FIRST_NAME",
  "expressionName": "FirstNameType",
  "dataType": "String",
  "minDataLength": 5
}

If there are more than one type expressions assigned to a domain then a column will match for the domain if the regular expression matches and at least one of the type expressions match. For example, dates of birth are often stored in string types instead of dates, so you might have a String type expression and a Date type expression assigned to the Date of Birth domain to allow columns of either type to match. Two type expressions of the same type can't be assigned to the same domain in the same profile set. If there are no type expressions assigned to a domain, then the profile expression alone will determine matching without regard to data type. Like Profile Expressions, Profile Type Expressions must be part of a profile set in order to be effective. Profile Type Expressions have no effect on Data Level Profiling.

Note

Profile Type Expressions are only supported for database profiling. They have no effect on profiling of file data.

Note

Currently only Oracle and MS SQL Server are fully supported. On other platforms, type expressions may result in unexpected matches.

String¶

All character types supported by the database, such as VARCHAR, NVARCHAR, CLOB and NCLOB are considered String types by the profiling logic. The minLength parameter considers the length specification of the column type, which may be characters or bytes. For example, Oracle supports VARCHAR2 fields measuring in either characters or in bytes. A VARCHAR2(20) column can hold 20 characters whereas a VARCHAR2(20 BYTE) column can hold 20 bytes, which may be fewer than 20 characters if multibyte characters are present. A type expression with a minLength of 20 will match to both.

Number¶

All numeric types are considered Number types by the profiling logic, including INTEGER, FLOAT, BIG_INTEGER, etc. The minLength parameter considers the number of base-10 digits supported by the type. For floating point values, minLength refers to the integral part of the number.

Date¶

The Date type includes all calendar date and date/time types, such as DATE and LOCAL_DATE_TIME types. The minLength parameter is not permitted for Date Type Expressions.

Binary¶

The Binary type includes large object types such as BLOB and BINARY. The minLength parameter considers the maximum storage size of the column in bytes.

Note

Length constraints are not applied to large object types such as CLOBs and BLOBs.

To add a Type Expression¶

In the examples below, the apiHostInfo, login and helpers modules are described in Delphix Masking Apis

Adding a Profile Type Expression can be accomplished through the API as shown here:

#!/bin/bash

source apiHostInfo
eval $(cat loginCredentials)
source helpers

login

curl $SSL_CERT -X POST -H ''"$AUTH_HEADER"'' -H 'Content-Type: application/json' -H 'Accept: application/json' --data @- $MASKING_ENGINE/profile-type-expressions <<EOF
{
  "domainName": "FIRST_NAME",
  "expressionName": "FirstNameType",
  "dataType": "String",
  "minDataLength": 5
}
EOF

echo

As mentioned above, to be effective, a Profile Type Expression has to be part of a profile set. A type expression can be added to a profile set with the profile-sets endpoint. For example, if some Profile Type Expressions were created and have ids 57 and 48, we can use the PUT method on the profile-set endpoint to update an existing profile set so that it includes the new profile type expression. This is shown below, where the profile set has id 42.

#!/bin/bash

source apiHostInfo
eval $(cat loginCredentials)
source helpers

login

curl $SSL_CERT -X PUT -H ''"$AUTH_HEADER"'' -H 'Content-Type: application/json' -H 'Accept: application/json' --data @- $MASKING_ENGINE/profile-sets/42 <<EOF
{
  "profileSetName": "FINDS_ALL_SENSITIVE_DATA",
  "profileExpressionIds": [
    4,
    8,
    12,
    13,
    27
  ],
  "profileTypeExpressionIds": [
    57,
    58
  ]
}
EOF

To delete a Type Expression¶

Deleting a type expression is done using the DELETE method on the profile-type-expression endpoint. The expression must be removed from any profile sets it's a part of before it can be deleted.

#!/bin/bash

source apiHostInfo
eval $(cat loginCredentials)
source helpers

login

echo "* creating application 'App123'..."
curl $SSL_CERT -X DELETE -H ''"$AUTH_HEADER"'' -H 'Content-Type: application/json' -H 'Accept: application/json' --data @- $MASKING_ENGINE/profile-type-expressions/57

echo

Profiler Sets¶

Profiling jobs use Profiler Sets to determine the set of Expressions to use in identifying sensitive data in an Inventory. A Profiler Set is a grouping of Expressions for a particular purpose. For instance, First Name, Last Name, Address, Credit Card, SSN, and Bank Account Number Expressions could constitute a Financial Profiler Set.

The Masking Engine comes with two predefined Profiler Sets: Financial and Healthcare vertical. A Delphix Masking Engine administrator (a user with the appropriate role privileges) can create/add/update/delete these Profiler Sets.

If you want to edit or add a Profiler set, click Profiler Set at the top of the Profiler Settings screen. The Profiler Set dialog appears, listing the Profiler Sets along with their Purpose, Owner, and Date Created.

To add a Profiler Set¶

Click Add Set at the top of dialog window.
Enter a Profiler Set Name.
Optionally, enter a Purpose for this Profiler Set.
Enter or select which Expressions to include in this set.
When you are finished, click Submit.

To edit an existing Profiler Set, click the Edit icon to the right of the Profiler Set name.

To delete a Profiler Set¶

Click the Delete icon to the right of the Profiler Set name.