Skip to content

Data Cleansing

A data cleansing algorithm does not perform any masking. Instead, it standardizes varied spellings, misspellings, and abbreviations for the same name. For example, “Ariz,” “Az,” and “Arizona” can all be cleansed to “AZ.” Use this algorithm if the target data needs to be in a standard format prior to masking.

Creating a Data Cleansing Algorithm via UI

  1. Enter an Algorithm Name.

    Info

    This MUST be unique.

  2. Enter a Description.

  3. Select Lookup File location.

  4. Specify a Delimiter (key and value separator). The default delimiter is =. You can change this to match the lookup file.

  5. Click Save.

Below is an example of a lookup input file. It does not require a header. Make sure there are no spaces or returns at the end of the last line in the file. The following is sample file content:

NYC=NY
NY City=NY
New York=NY
Manhattan=NY

For information on creating Data Cleansing algorithms through the API, see API Calls for Creating Algorithms - Data Cleansing.