Overview
Through optional modules EasyCatalog provides a number of data providers (ODBC, XML, Excel, CSV, Sqlite). The Enterprise Data Provider also provides additional connections to a number of third party systems. For connections to other systems, such as JSON or REST APIs, the Enterprise Data Provider enables the ability to create custom data providers.
Creating a Custom Data Provider
Custom data providers are implement via a single Lua script. The script is parsed each time the File->New->EasyCatalog Panel menu is accessed. The script should be called <name>.lua where <name> is the menu name shown on the File menu. Templates should be saved into the ‘EasyCatalog Data Source Templates’ folder in the EasyCatalog workspace folder.
Implementing the Script
The script should provide a number of specific functions, which are called during data retrieval and update. The script is stateless, so cannot persist global variables. To persist data a single Lua table is used to stores data source settings in the data.xml configuration file.
Script Functions
The examples below show connection to the REST API of an Airtable data source. The entire script can be downloaded here.
Initial Data Retrieval
Initialize()
This method is called immediately after selecting File->New->EasyCatalog Panel->New <name> Data Source. The purpose is to return a table of default settings for the data source.
1 2 3 4 5 6 7 8 9 | function Initialize() -- create a new table config = {}; -- setup some default setting config.name = "New Name"; config.apiKey = "keyeW488zRndxDvWy"; config.initialized = false; return config; end |
ConfigureUI(config)
In the Desktop version of InDesign, this is called after Initialize or when the data source is edited. It’s not used on InDesign Server. Its purpose is to present UI in order to specify credentials and criteria for data selection and return an updated configuration table. If a table is returned, this must contain a name entry, which will be the name of the new data source. If it returns nothing, then the user cancelled the operation. In the case of editting the data source, the name cannot change.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | function ConfigureUI(config) configdialog = DIALOG.new( { title = "New Airtable Data Source", validate = validatemethod} ); enable_name = false; if config.initialized == false then enable_name = true end; configdialog:addwidget( { { type = "statictext", title = "Name", align = "left", left = 20, top = 20, right = 100, bottom = 40, }, { type = "editbox", id = "Name", left = 100, top = 20, right = 450, bottom = 40, content = config.name, enable = enable_name }, { type = "statictext", title = "API Key2:", align = "left", left = 20, top = 20+25, right = 100, bottom = 40+25 }, { type = "editbox", id = "ID", left = 100, top = 20+25, right = 450, bottom = 40+25, content = config.apiKey, enabled = true }, { type = "button", id = "testbutton", title = "Show JSON Result", left = 100, top = 20+25+25, right = 250, bottom = 40+25+25, onchange = testbuttonpress}, { type = "custom", id = "preview", multiselect = true, left = 10, top = 20+25+25+25, right = 450, bottom = 40+25+25+560, vscroll = true, stepamount = 20}, } ); if configdialog:open() then config.name = configdialog:getwidget("Name").content; config.apiKey = configdialog:getwidget("ID").content; config.initialized = true; return config; end end |
Synchronize(config, DATASOURCE)
Once the configuration table has been created, Synchronize is called to create a new RECORDSET, which should be returned. Parameters are the data sources configuration setting and the DATASOURCE object. This should return the result of a RECORDSET.new command to create a complete record set of field definitions and record data, or nil for failure.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | function Synchronize(config, datasource) offset = ""; getmore = true; fields = { { name = "airtable_id", key = "true"} } records = {} unique_field_names = {}; unique_field_count = 0; -- Unfortunately there's no way in Airtable to determine the number of data pages local newbar = PROGRESSBAR.new(); newbar:show("Downloading Data From Airtable", 100, true) page = 1; while getmore do newbar:update("Downloading Page #" .. page); url = "https://api.airtable.com/v0/appBFZg5x97vChmPw/Imported%20table?api_key=" .. config.apiKey; if offset ~= "" then url = url .. "&offset=" .. offset; end err, body = HTTP.get(url) if err ~= 200 and err ~= 401 then -- canceled error should return nil if err == 2 then return nil; end; error(err .. body); end -- report given errors json_return = cjson.decode(body) if json_return["error"] ~= nil then error(json_return["error"].message); end; arts = json_return["records"]; for i=1, #arts do article = arts[i]; newrecord = { article.id } records_fields = article["fields"]; for k, v in pairs(records_fields) do if unique_field_names[k] ~= 1 then unique_field_names[k] = 1; unique_field_count = unique_field_count + 1; fields[unique_field_count+1] = { name = k }; end newrecord[k] = v; end; table.insert(records, newrecord); end; offset = json_return["offset"]; if offset == nil then getmore = false end; if offset == "" then getmore = false end; page = page + 1; end; return RECORDSET.new(fields, records); end |
SaveRecords(config, datasource, records)
This is an optional function. When available it is called to write back local changes to the original data source. For selective updates, on a field by field basis the updatestate of the field can be used to check if it needs updated. On successful update, this should be cleared.
1 2 3 4 5 6 7 8 9 10 11 | function SaveRecords(config, datasource, records) for i=1,records:size() do record = records:getrecord(i); for x=1, record:size() do field = record:field(x); if field:getupdatestate() & 2 == 2 then field:setupdatestate(0); end end end end |
Handling Images
When an image field is set to URL source, the option to allow the script to download the image is available.
CanResolveAssetURI(field name, uri)
The field name and the location as determined the fields location setting are passed. The function should return true or false to indicate the image should be downloaded by the script;
ResolveAssetURI(configuration, uri)
Should turn the uri as specified by the image into a fully qualified URL
GetAsset(config, location, filepath)
Preforms the download of the image. location is the result of ResolveAssetURI and file path is a full path to where the image should be stored. returns true of false to indicate success.
