Skip to content

Creating Masking Job

This section describes how users can create a masking job.

Creating New Jobs

In the Environment Overview screen, select one of the jobs icons to create the corresponding job:

  • Profile

  • Mask

Creating a New Masking Job

To create a new masking job:

  1. Click Mask. The Create Masking Job window appears.

  2. You will be prompted for the following information:

    1. Job Name — A free-form name for the job you are creating. Must be unique across the entire application.

    2. Masking Method — Select either In-Place or On-The-Fly. In-Place jobs update the source environment with the masked values. On-The-Fly jobs read unmasked data from the source environment and writes the masked data to the target environment.

      INFO: On-The-Fly Masking Jobs.

      Only certain combinations of connector types are supported. On-The-Fly jobs where the source and target connectors are of the same type (e.g. Oracle to Oracle, delimited file to delimited file), and jobs with a database source (e.g. Oracle, MS SQL) and the target is delimited files are supported.

      The target tables or files must be created in advance and the names must match the names of the source tables or files. In the case of a database to delimited file job, the file names should match the table names.

    3. Multi Tenant — Check box if the job is for a multi-tenant database.

      INFO: Provisioning Masked VDBs.

      A job must be Multi-Tenant to use it when creating a masked virtual database (VDB). This option allows existing rulesets to be reused to mask identical schemas via different connectors. The connector can be selected at job execution time.

    4. Rule Set — Select a rule set that this job will execute against.

    5. Source Environment (only for On-The-Fly Masking Method) - Select the Source Environment that this job will get the data from.

    6. Source Connector (only for On-The-Fly Masking Method) - Select the Source Connector that provides the connection to chosen Source Environment.

    7. No. of Streams—The number of parallel streams to use when running the jobs. For example, you can select two streams to run two tables in the Rule Set concurrently in the job instead of one table at a time.

    8. Min Memory (MB) — (optional) Minimum amount of memory to allocate for the job, in megabytes.

    9. Max Memory (MB) — (optional) Maximum amount of memory to allocate for the job, in megabytes.

    10. Update Threads — The number of update threads to run in parallel to update the target database.

      Info

      Multiple threads should not be used if the masking job contains any table without an index. Multi-threaded masking jobs can lead to deadlocks on the database engine. Multiple threads can cause database engine deadlocks for databases using T-SQL If masking jobs fail and a deadlock error exists on the database engine, then reduce the number of threads.

    11. If Nonconforming Data is encountered

      • Stop job on first occurrence - (optional) To abort a job on first occurrence of nonconforming data. The default is for this check box to be clear.

      Info

      The job behavior depends on the setting specified in the If Nonconforming data is encountered field on the Algorithm Settings page. If Mark job as Failed is selected then the job would be aborted on first occurrence of nonconforming data. If Mark job as Succeeded is selected then the job will not be aborted.

    12. Commit Size — (optional) The number of rows to process before issuing a commit to the database.

    13. Feedback Size — (optional) The number of rows to process before writing a message to the logs. Set this parameter to the appropriate level of detail required for monitoring your job. For example, if you set this number significantly higher than the actual number of rows in a job, the progress for that job will only show 0 or 100%.

    14. Disable Constraint — (optional) Whether to automatically disable database constraints. The default is for this check box to be clear and therefore not perform automatic disabling of constraints. For more information about database constraints see Enabling and Disabling Database Constraints.

    15. Batch Update — (optional) Enable or disable use of a batch for updates. A job's statements can either be executed individually, or can be put in a batch file and executed at once, which is faster.

    16. Disable Trigger — (optional) Whether to automatically disable database triggers. The default is for this check box to be clear and therefore not perform automatic disabling of triggers.

    17. Drop Indexes — (optional) Whether to automatically drop indexes on columns which are being masked and automatically re-create the index when the masking job is completed. The default is for this check box to be clear and therefore not perform automatic dropping of indexes.

    18. Prescript — (optional) Specify the full pathname of a file that contains SQL statements to be run before the job starts, or click Browse to specify a file. If you are editing the job and a prescript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a prescript file was already specified.) For information about creating your own prescript files.

    19. Postscript — (optional) Specify the full pathname of a file that contains SQL statements to be run after the job finishes, or click Browse to specify a file. If you are editing the job and a postscript file is already specified, you can click the Delete button to remove the file. (The Delete button only appears if a postscript file was already specified.) For information about creating your own postscript files see Creating SQL Statements to Run Before and After Jobs

    20. Comments — (optional) Add comments related to this masking job.

    21. Email — (optional) Add e-mail address(es) to which to send status messages.

  3. When you are finished, click Save.

Enabling and Disabling Database Constraints

Depending on the type of target database you are using, the Delphix Engine can automatically enable and disable database constraints.

The ability to enable and disable constraints ensures that the Delphix Engine can update columns that have primary key or foreign key relationships. You can set Delphix to handle constraints automatically by enabling the Disable Constraint checkbox on a Masking job.

Note

Delphix does not support the enable/disable constraints feature for all databases. To see which databases are supported, see the Data Source Support page.

Creating SQL Statements to Run Before and After Jobs

When you create a masking job or a certification job, you can specify standard, static SQL statements to run before (prescript) you run a job and/or after (postscript) the job has completed. For example, if you want to mask a column that has a foreign key constraint to another table, you could use a prescript to disable the constraint and a postscript to re-enable the constraint.

You create prescripts and postscripts by creating a text document with the SQL statement(s) to execute. If the text file contains more than one SQL statement, each statement must be separated by a semicolon [;]. For example to remove records with date_column before December 12th, 2017 before masking a table (owner.table), one would create a prescript file containing the following and associate the prescript file to the masking job that includes the table in its ruleset:

DELETE FROM owner.table WHERE date_column < ‘20171207’;

Database-specific, SQL programming extensions (such as PL/SQL and Transact-SQL) and dynamic SQL statements are not supported in prescripts and postscripts. However, you can create procedures and functions using your database tooling of choice and call them using standard SQL statements from a prescript or postscript.