Augmenting Datasources via “CreateContent.lua”

Manipulation of  incoming data is possible through the execution of a script right after EasyCatalog retrieves data from a data source, but before any subsequent processing takes place. The script, named ‘createcontent.lua‘, when placed in the data source’s scripts folder, is triggered as the last stage of the synchronise operation. Pressing the info button on a data panel, and then the script Icon will automatically create this file. It receives a ‘records’ variable, which is a RECORDSET object representing the unprocessed data obtained from the data source.

This script has the ability to add new records, remove fields, modify field names, and assign field content. This functionality allows for the integration of data from diverse sources into a unified dataset, as well as the retrieval of language-specific fields based on options provided by the data source.

For example:

simple example.xls

field 1field 2field 3
1AA
2BB
3CC
4DD
5EE

When loaded into EasyCatalog, with field 1 as a key, it looks like:

Using the following ‘createcontent.lua’ script:

The result after synchronizing is:

Make sure the field type of “Tabular Field” is set to be a tabular field, with “Command Script” as the source.

Notice the indicators on the fields which didn’t originate from the original data. Fields created in this way are treated as regular fields, with the exception of tabular fields which are stored in binary format within the snapshot file.

The benefit of employing this technique instead of using ‘createformattedcontent.lua’ is that data manipulation occurs prior to loading, resulting in actual changes to the content within a field. Furthermore, it has no impact on the rebuilding of formatted content, as only takes place during data synchronization. Additionally, this approach allows for the indication of content changes, including updates and deletions.


Examples:

Duplicating records using a delimited category field

This particular sample is suitable for situations where a single field determines the categories in which a record should appear. In order to effectively group the panel, EasyCatalog necessitates having one record per category. Hence, this script duplicates a record for each value found in its ‘Category’ field, where values are separated by a semi-colon. Since each record requires a unique identifier (referred to as the key), this script appends ‘-1’, ‘-2’, ‘-3’, and so on to the end of the key field for each duplicated record. In this example, the key field is named ‘Product number,’ and the field containing the list of categories is ‘Category.’

Transforming XML field content using XSL

This script is designed to transform XML data using XSLT stylesheets and store the transformed results in a field named “Tranformed” for each record. It requires a version of EasyCatalog 2024 or the beta version to work correctly.

Creating a configurable option to select data

This example introduces a custom ‘locale’ parameter for the data source. This parameter enables the extraction of data from JSON and assigns it to a field based on the selected locale. You can modify the locale by accessing the Information Dialog in the data panel and clicking the ‘Update’ button. The JSON data is processed using JMESPath

Defining a custom sort order field

This example creates a custom sorting field based on the sizes of T-Shirts.

Parsing JSON data using a cursor

The following example shows breaking down a JSON field using the jsoncursor method. Only available in the 2024 version:

Augmenting data from another data source

Creates a set of HEADING and VALUE fields as described in an external data source:

Selectively Collecting Field Content

Collects fields without the given prefix into a delimited field for use as a Tabular Field.