Version: 7.x-36.0.0

This is a technical preview. Technical preview features are not fully supported, may not be functionally complete, and are not suitable for deployment in production. We encourage you to try them out and provide your feedback, good and bad, on the Search Guard forum. This will help us improve and add any features you might be missing.

Transformations

A transformation is a script that

  • has access to the runtime data
  • performs one or more painless statements
  • replaces the runtime data in the target context

As opposed to Calculations, Transformation scripts have a return statement and need to define the target context where the transformed values are written back to.

If the target context already exists, it is overwritten. If not, a new one is created.

Transformations can be used

  • in the checks section of a watch definition
    • the transformation is executed before any actions are executed. Changes to the execution context runtime data are visible for all subsequent steps and for all actions
  • in the checks section of any action
    • the transformation is executed before the action is executed. Changes to the execution context runtime data are only applied for that specific action.

Using inline scripts

A transformation painless script can be defined as inline script within the transformation definition.

For example, the next transformation accesses a runtime context that has stored an Elasticsearch query result. It will replace the context data with only the hits of the search result, discarding all other data like total hits or execution time

{
  "type": "transform",
  "name": "extract_search_hits",
  "target": "mysearchresult"
  "source": "return data.logs.hits.hits;"
}
Name Description
type transform, defines this script as transformation. Mandatory.
name name of this transformation. Can be chosen freely. Mandatory.
target Under which context name to store the result of the transformation in the runtime data. If the context already exists, it is replaced. If it does not exist, a new contect is created. If omitted, the top-level context will be used.
source The script to execute. Mandatory
lang The scripting language to be used. Optional, defaults to painless. Other scripting languages may be provided by Elasticsearch plugins.

Using stored scripts

{
  "type": "transform",
  "name": "extract_search_hits",
  "target": "mysearchresult"
  "script_id": "extract_hits"
}
Name Description
type transform, defines this script as transformation. Mandatory.
name name of this transformation. Can be chosen freely. Mandatory.
target Under which context name to store the result of the transformation in the runtime data. If the context already exists, it is replaced. If it does not exist, a new contect is created. If omitted, the top-level context will be used.
script_id The ID of the stored script. Mandatory.

Accessing the runtime data

All scripts have full access to the runtime data. The data in the execution context is available via the data prefix.

Using transformations with actions

Transformations can also be used with actions. Each action can define it’s own chain of checks, including transformation.

The next example runs a transformation that extracts the hits from an Elasticsearch result set prior to writing it back to another Elasticsearch index via an Index Action.

{ 
  "trigger":{},
  "checks":[],
  "actions":[ 
    {
      "type": "index",
      "name": "store_cleaned_data",
      "checks": [
        {
          "type": "transform",
          "name": "extract_search_hits",
          "target": "mysearchresult"
          "script_id": "extract_hits"
        }
      ],
      "index": "signals-data"
    }
  ]
}

Not what you were looking for? Try the search.