Skip to content

Managing Algorithm Usage

Overview

The Masking Engine provides API endpoints to view and modify the usage of algorithms globally. These operations span all usage of an algorithm, including: database column and file format assignments across all environments, usage in domains, and references from other algorithms.

Viewing Algorithm Usage

The following API endpoint retrieves all usage of the algorithm specified in the request path:

algorithm   GET algorithm/{algorithmName}/usage

This endpoint supports the following option in the query parameters:

  • includeAssignmentDetail (optional, default=false)

    boolean
    Enabling this option causes the API response to include a list of human-readable assignment detail objects, one for each individual usage of the algorithm in inventory. This can result in a very large response if the algorithm is heavily used. Algorithm usage in file formats will be reported once for each application of the file format to a file in inventory.

  • environmentFilter (optional)

    String
    Report only usage occurring within the specified environment(s). This query option may be included multiple times, in which case usage for all matching environments will be reported. When the algorithm is used in a file format, all usage of that file format is reported so long as it is referenced by any environment matching the filter; this may include environments other than those selected by the filter. Filtering by environment excludes all domain and algorithm reference usage, as such usage is global rather than part of any particular environment.

  • rulesetFilter (optional)

    String
    Report only usage occurring within the specified ruleset(s). This query option may be included multiple times, in which case usage for all matching rulesets will be reported. When the algorithm is used in a file format, all usage of that file format is reported so long as it is referenced by any ruleset matching the filter; this may include rulesets other than those selected by the filter. Filtering by ruleset excludes all domain and algorithm reference usage, as such usage is global rather than part of any particular ruleset.

Updating Algorithm Usage

The following API endpoint updates all usage of the algorithm specified in the request path to use the new algorithm name supplied as a query parameter:

algorithm   PUT algorithm/{algorithmName}/usage

This endpoint supports the following option in the query:

  • replacementAlgorithmName (required, no default)

    String
    The name of the algorithm that should replace the existing algorithm in all usage across the entire Masking Engine.

  • ignoreIncompatibleTypes (optional, default=false)

    boolean
    Setting this option to true will allow some algorithm replacements that would normally fail due to incompatible types to succeed. This may result in job failures if type conversions don't exist to convert the underlying data type to the type expected by the new algorithm.

  • environmentFilter (optional)

    String
    This options functions just like the environmentFilter option for the GET operation described above. Only usage matching the filter is updated. The update will fail if any file format referencing the algorithm is used from an environment that does not match the filter. Domain and algorithm reference usage is never updated when an environment filter is applied.

  • rulesetFilter (optional)

    String
    This options functions just like the rulesetFilter option for the GET operation described above. Only usage matching the filter is updated. The update will fail if any file format referencing the algorithm is used from a ruleset that does not match the filter. Domain and algorithm reference usage is never updated when a ruleset filter is applied.

The response body from the PUT request details all usage that was updated by the operation.

WARNING

Globally updating algorithm usage can produce many inventory changes across multiple environments, and is not easily reversible when the replacement algorithm is already in use on the engine.

Delphix recommends performing the following steps before any update to algorithm usage via this API endpoint:

  1. Perform the GET usage operation (described above) for both the existing and replacement algorithms. Carefully review the results and save them for future reference.

  2. Export the engine's global settings and all affected environments prior to changing algorithm usage.

WARNING

Algorithm compatibility checking of usage changes may still allow some replacements that could result in job failures using the new algorithm. Careful consideration should be given to whether the new algorithm can handle the data types and inputs for all usage of the algorithm being replaced.

Examples

Getting usage for an algorithm with one column assignment:

REQUEST

curl -X GET --header 'Accept: application/json' --header 'Authorization: d46db68d-59f1-41e0-a128-c01bc920da30'
'http://masking-engine.example.com/masking/api/v5.1.10/algorithms/alg_6EBH8EGK/usage?includeAssignmentDetail=false'

RESPONSE

{
  "algorithmName": "alg_6EBH8EGK",
  "columnMetadataIds": [
    11
  ],
  "fileFieldMetadataIds": [],
  "mainframeDatasetFieldMetadataIds": [],
  "domainNames": [
    "domain_6GXKQP60"
  ],
  "algorithmReferences": []
}

The same request with additional detail requested:

REQUEST

curl -X GET --header 'Accept: application/json' --header 'Authorization: d46db68d-59f1-41e0-a128-c01bc920da30'
'http://masking-engine.example.com/masking/api/v5.1.10/algorithms/alg_6EBH8EGK/usage?includeAssignmentDetail=true'

RESPONSE

{
  "algorithmName": "alg_6EBH8EGK",
  "columnMetadataIds": [
    11
  ],
  "fileFieldMetadataIds": [],
  "mainframeDatasetFieldMetadataIds": [],
  "domainNames": [
    "domain_6GXKQP60"
  ],
  "algorithmReferences": [],
  "assignmentDetails": [
    {
      "assignmentType": "DATABASE_COLUMN",
      "environmentName": "env_ZBQ0XKO9",
      "databaseRulesetName": "rule_POQRBZ44",
      "databaseTableName": "profile",
      "databaseColumnName": "last_name"
    }
  ]
}

Updating all usage of algorithm named alg_6EBH8EGK to alg_82U5GUZB:

REQUEST

curl -X PUT --header 'Content-Type: application/json' --header 'Accept: application/json' 
--header 'Authorization: d46db68d-59f1-41e0-a128-c01bc920da30'
'http://masking-engine.example.com/masking/api/v5.1.10/algorithms/alg_6EBH8EGK/usage?replacementAlgorithmName=alg_82U5GUZB&ignoreIncompatibleTypes=false'

RESPONSE

{
  "columnMetadataIds": [
    11
  ],
  "fileFieldMetadataIds": [],
  "mainframeDatasetFieldMetadataIds": [],
  "domainNames": [
    "domain_6GXKQP60"
  ],
  "algorithmReferences": [],
  "assignmentDetails": []
}