Skip to content

Making an Algorithm Configurable

As described in the introduction, there are a couple of requirements for making an algorithm configurable, so that it will appear as a framework on the Delphix Masking Engine:

  1. The getAllowFurtherInstances method in the algorithm Class must return true.
  2. One or more data members in the algorithm class must be marked public, and must be annotated with the @JsonProperty (specifically com.fasterxml.jackson.annotation.JsonProperty) annotation.

In order to assure that JSON document and schema interpretation is consistent, most JSON handling is done by the Masking Plugin API implementation, rather than the plugins themselves. For each configurable algorithm, the SDK or Delphix Masking Engine will examine the annotations in the class to determine which values are configurable. Whenever a new instance is created, an attempt is made to apply the user-supplied JSON to the object of the framework class. This includes some validation that the supplied JSON matches the expected schema implied by the set of fields marked configurable, however there are some limitations to this validation, as described below.

Example Configurable Algorithm Explained

The concept of configurability can be illustrated using one of the sample algorithms from the SDK as an example - StringRedaction.java in this case:

package sample.masking.algorithm.redaction;
...

public class StringRedaction implements MaskingAlgorithm<String> {
    private String name = "StringRedaction";

    @JsonProperty(value = "redactionCharacter", required = true)
    public String redactionCharacter = "specified";

    @Override
    public String getName() {
        return name;
    }

    @Override
    public Collection<MaskingComponent> getDefaultInstances() {
        StringRedaction instanceX = new StringRedaction();
        instanceX.name = "Redaction X";
        instanceX.redactionCharacter = "X";
        return Arrays.asList(instanceX);
    }

    @Override
    public boolean getAllowFurtherInstances() {
        return true;
    }

    @Override
    public String getDescription() {
        return String.format(
                "Redact String by overwriting with '%s' character", redactionCharacter);
    }

    @Override
    public String mask(@Nullable String input) throws MaskingException {
        if (input == null) {
            return null;
        }
        StringBuilder returnVal = new StringBuilder();

        for (int i = 0; i < input.length(); i++) {
            returnVal.append(redactionCharacter);
        }
        return returnVal.toString();
    }

    @Override
    public void validate() throws ComponentConfigurationException {
        if (redactionCharacter == null || redactionCharacter.length() != 1) {
            throw new ComponentConfigurationException(
                    "redactionCharacter must be a single character");
        }
    }
}

This algorithm does simple redaction of the input String, but the redaction character may be configured by creating additional instances with custom values. How this works:

  • The Class has a public field redactionCharacter annotated with @JsonProperty. A default value has been provided so that the getDescription method will return a suitable description in both the framework and instance cases.

  • The Class's getDefaultInstances method defines a single instance, with redaction characters 'X'. This is accomplished by simply returning a list of correctly configured objects. The API framework extracts the object configuration as JSON, and store it for use whenever an instance of "Redaction X" algorithm is needed.

  • The Class's getAllowFurtherInstances method returns true, making it possible to create additional instances of this algorithm after the plugin is loaded on the Masking Engine using the Masking API via the API client.

  • The Class implements a validate method to ensure that the supplied configuration value is usable. In this case, the length of the redactionCharacter String is restricted to a single character.

Frameworks, Instances, and Configuration Injection

When used as a framework, the algorithm class is instantiated and used without any configuration injection. In the example above, that means that the getDescription method will return "Redact String by overwriting with 'specified' character" when the algorithm framework is evaluated. Similarly, getName will return "StringRedaction", the name of the framework.

When a runnable algorithm instance is needed, the algorithm class is instantiated, and all saved configuration is injected before any methods are called. This configuration is gathered in one of two ways:

  • For statically provided instances embedded in the plugin, the configurable fields of each object returned by the getDefaultInstances method are serialized to JSON and saved. Again, only the values of public fields marked with the @JsonProperty annotation are extracted this way.
  • When the user creates a new algorithm instance using the Masking Web API, the contents of the algorithmExtension field of the POST or PUT request is validated and saved for future injection whenever that particular algorithm instance is needed in the future.

Using the above example again, when algorithm instance "Redaction X" is created, the saved values will be injected, so redactionCharacter will have the value 'X'.

Validation of Configuration Values

For what the author can only presume to be performance considerations, the major JSON handling libraries perform only minimal validation when objects are deserialized. The practical effect of this is that several aspects of the @JsonProperty annotation are not enforced. For example, a property might be marked as required, but an object will be successfully deserialized even when that property is missing from the input JSON. While libraries are available that would allow us to expand the degree to which JSON is validated by the framework, this would make defining the exact set of validations done by the API framework vs. what must be validated in the component's validate method even more complex. For these reasons, only minimal input validation is performed by the framework. Plugin authors should validate all aspects of the object's configuration, especially the presence (that is, non-null, non-empty value) of required fields, in the validate method implementation.

However, this is not to say that the unenforced properties of the @JsonProperty annotation should be omitted. These values are visible in the auto-generated schema for each framework, which is visible using the SDK's maskApp, as well as the algorithm/framework endpoint in the Masking API Client, and may be useful for UI generation in the future.

Default Interface Implementations

The Masking Plugin API defines default implementations of getDefaultInstances and getAllowFurtherInstances as follows:

default Collection<ComponentInstanceDescription> getDefaultInstances() {
        return Collections.singletonList(this));
    }

default boolean getAllowFurtherInstances() {
        return getDefaultInstances() == null || getDefaultInstances().isEmpty();
    }

This means that if neither of these methods is overridden by the masking algorithm class, a single instance capturing whatever default values exist for configurable fields is created by default.

Only algorithms classes that define getAllowFurtherInstances to return true appear as Algorithm Frameworks on the Masking Engine.

Build Dependencies for Configurable Algorithms

When the maskScript init sub-command is used to create a new project, the initial build files will may not include the dependencies required for the Jackson @JsonProperty annotation. This an be corrected by adding this line to proj_root/gradle.properties:

jacksonVer=2.9.5

And this line to the dependencies* section at the end of proj_root***/build.gradle:

compileOnly ('com.fasterxml.jackson.core:jackson-annotations:' + jacksonVer)

The set of Jackson annotations tested and supported for use in algorithm plugin classes are:

  • @JsonProperty
  • @JsonPropertyDescription
  • @JsonFormat (Useful in specifying formats for Date fields)