Input Data Transformation

If structured Input Data are not in JSON format necessary for UI Agent processing, data transformation can be used to bring them to this format and related data structures.

Configuring default data transformation for UI Agent

Default data transformer can be configured for the UI Agent.

JSON data are expected by default if not configured.

Configuring data transformation for data type

Data transformer can be configured per InputData.type, so it has to be defined to use transformation.

Example of the yaml config:

data_types:
  movie.detail:
    data_transformer : yaml
    ...

OOTB transformers

Few OOTB transformers are provided in the UI Agent Core package

JSON transformer

Transformer name: json

Default transformer used for JSON data.

YAML transformer

Transformer name: yaml

As YAML is another form how to express the same data structures as JSON, conversion is very straighforward.

CSV transformers

Transformer name: csv-comma, csv-semicolon, csv-tab

This transformer takes CSV formatted text with delimiter indicated in the transformer name. " character is used as quotation mark in the case delimiter or new line is present in the CSV value. First row is used as field names, other rows are converted into array of objects, where field names from the first row are used. Field names are sanitized so JSONPath can work with them easily.

Field values are trimmed from leading/trailing white spaces, and converted from String to Boolean or Number if possible.

Fixed Width Columns Table transformer

Transformer name: fwctable

Similar to the CSV transformer, produces array of objects, but expects that columns of the table have fixed width in number of characters. First row is used as field names. It expects at least two consecutive white characters as a column separator on the first row, so one white character can be used in the column label/field name.

Field names sanitization and values handling are the same as in case of the CSV transformer.

Example of data in "Fixed Width Column Table" format:

Name     Age  Birth city
John Doe 30   New York
Jane Ei  25   Boston

Noop transformer

Transformer name: noop

This transformer keeps input data as a string, not as a parsed JSON tree. Usefull if large chunk of unformatted text must be passed to the UI component.

For Hand-Build Components, text is outputted directly in the data field of their JSON data.

If LLM processing has to be applied, text is shortened to 1000 characters to save LLM's context and wrapped into JSON object with one field.

Writing own transformer

UI Agent core package allows to add new data transformers. Stevedore framework is used, so you only have to implement your own python module, and install it to the UI Agent.

To implement transformer, you have to:

Add next-gen-ui-agent dependency to your python module
Implement class extending next_gen_ui_agent.types.InputDataTransformerBase. Be sure object structure returned from the transformation matches defined rules for values access by jsonpath_ng and JSON serialization by Pydantic model_dump_json(). Implement correct error handling, write unit tests. You can find examples of transformers and their unit tests in UI Agent core source code.
Register your transformer using Stevedore under next_gen_ui.agent.input_data_transformer_factory namespace in your python module. Use unique transformer name.

   entry_points={
        'next_gen_ui.agent.input_data_transformer_factory': [
            'my_transformer_name = transformer_example:MyTransformer'
        ],
    },