Polling event
Last updated
Last updated
Version 5.4 of the Cinchy platform introduced data polling, a source option which uses the Cinchy Event Listener to continuously monitor and sync data entries from your Oracle, SQL Server, or DB2 server into your Cinchy table. This capability makes data polling a much easier, effective, and streamlined process and avoids implementing the complex orchestration logic that was previous necessary.
The Polling Event source supports real-time syncs.
The Polling Event Source supports Oracle, DB2 and SQL Server databases.
You can find the parameters in the Info tab below (Image 1).
Title
Mandatory. Input a name for your data sync
Polling Event Sync
Variables
Permissions
Data syncs are role based access systems where you can give specific groups read, write, execute, and/or all of the above with admin access. Inputting at least an Admin Group is mandatory.
The following table outlines the mandatory and optional parameters you will find on the Source tab (Image 2).
The following parameters will help to define your data sync source and how it functions.
Source
Mandatory. Select your source from the drop down menu.
Polling Event
Configure your Destination
Define your Sync Actions.
Add in your Post Sync Scripts, if required.
If more than one listener is needed for a real-time sync, configure it/them via the Listener Config table.
To run a real-time sync, enable your Listener from the Execution tab.
The messageKeyExpression parameter is an optional, but recommended, parameter that can be used to ensure that you aren't faced with a unique constraint violation during your data sync. This violation could occur if both an insert and an update statement happened at nearly the same time. If you choose not to use the messageKeyExpression parameter, you could face data loss in your sync.
Each of your Event Listener message keys a message key. By default, this key is unique for every message in the queue.
When the worker processes your Event Listener messages it does so in batches and, for efficiency and to guarantee order, messages that contain the same key won't be processed in the same batch.
The messageKeyExpression property allows you to change the default message key to something else.
Example:
"MessageKeyExpression": "id"
Optional. Review our documentation on for more information about this field.
Earliest will start reading from the beginning on the queue (when the CDC was enabled on the table). This might be a suggested configuration if your use case is recoverable or re-runnable and if you need to reprocess all events to ensure accuracy. Latest will fetch the last value after whatever was last processed. This is the typical configuration. None won't read or start reading any events. You are able to switch between Auto Offset Reset types after your initial configuration through the process outlined
Optional, but recommended to mitigate data loss. for more information on this parameter.
SELECT Id, Name
FROM [SourceTable]
WHERE Id IN (SELECT TOP (100) Id
FROM [SourceTable]
WHERE Id > 0 AND Name IS NOT NULL
ORDER BY Id)
AND Id > 0 AND Name IS NOT NULL
FROM
(
...
) AS t INNER JOIN [Table1] ts ON ts.[Id] = t.[Id]
WHERE ts.[Id] > 0
ORDER BY Id
SELECT ts.[Id],ts.[Name] FROM
(
SELECT Id,Timestamp
FROM [Table2]
WHERE Timestamp IN (SELECT TOP (2)
Timestamp
FROM [Table2]
WHERE Timestamp > '2022-11-18 11:34:09 AM'
AND Timestamp <= '2022-11-19 11:34:09 AM'
AND 1=1
ORDER BY Timestamp)
AND Timestamp > '2022-11-18 11:34:09 AM'
AND Timestamp <= '2022-11-19 11:34:09 AM'
AND 1=1
) AS t
INNER JOIN [Table1] ts ON ts.[Id] = t.[Id]
WHERE ts.[Id] > 0
ORDER BY Id