Secure Lookup is the most commonly used type of algorithm. It is easy to generate and works with different languages. When this algorithm replaces real, sensitive data with fictional data, it is possible that it will create repeating data patterns, known as “collisions.” For example, the names “Tom” and “Peter” could both be masked as “Matt”. Because names and addresses naturally recur in real data, this mimics an actual data set. However, if you want the Masking Engine to mask all data into unique outputs, you should use Character Mapping.
Starting in version 18.104.22.168, we introduced a builtin Extensible Secure Lookup Algorithm Framework, co-existing with the legacy one. The new framework uses SHA256 hashing method and allows case configurations for input and output (i.e. masked) values.
Creating a Secure Lookup Algorithm via UI¶
In the upper right-hand corner of the Algorithm tab, click Add Algorithm.
Choose Secure Lookup Algorithm. The Create SL Algorithm pane appears.
Choose the type of SL algorithm framework to use. For Legacy one click on
Legacybutton at the top. The default version is the newer Extensible SL Framework, allowing configuration of the following options:
Output (Masked) Case Case Sensitive Lookup
which are disabled for Legacy SL Framework type.
Enter an Algorithm Name.
This MUST be unique.
Enter a Description.
Choose the Output (Masked) Case configuration. This option is available only for the Extensible SL Framework type. It is explained with the examples in the information popup window, which may be opened by clicking on the blue question sign on the above Create SL Algorithm window:
Choose the Case Sensitive Lookup configuration. This option is available only for the Extensible SL Framework type.
If Case Sensitive Lookup box is marked then the same input of different cases will be masked to the different values. For example:
Peter -> John peter -> Andrew
If that setting is not marked (which is a default option), then lookup would be case insensitive, for example:
Peter -> John peter -> John
Specify a Lookup File.
This file is a single list of values. It does not require a header. Every line of the Lookup File might be used as a masked value. The Lookup File must be ASCII or UTF-8 encoding compatible. The following is sample file content:
Smallville Clarkville Farmville Townville Cityname Citytown Towneaster
When you are finished, click Save.
Before you can use the algorithm in a profiling job, you must add it to a domain.
For Legacy Framework type only: If the lookup file contains foreign alphabet characters, the file must be saved in UTF-8 format with no BOM (Byte Order Marker) for the Masking Engine to read the Unicode text correctly. Some applications, e.g. Notepad on Windows, write a BOM (Byte Order Marker) at the beginning of the Unicode file. This character will be included as part of the first replacement value, potentially leading to SQL update or insert errors on databases where this character is not allowed in VARCHAR fields - when trying to run a masking job that applies a Secure Lookup algorithm that has been created based on a UTF-8 file that included a BOM.
Extensible builtin SL Framework filters the BOM automatically without causing the mentioned errors.
For information on creating Secure Lookup algorithms through the API, see API Calls for Creating Algorithms - Secure Lookup.