Cinchy Platform Documentation
Cinchy v5.6
Cinchy v5.6
  • Data Collaboration Overview
  • Release Notes
    • Release Notes
      • 5.0 Release Notes
      • 5.1 Release Notes
      • 5.2 Release Notes
      • 5.3 Release Notes
      • 5.4 Release Notes
      • 5.5 Release Notes
      • 5.6 Release Notes
  • Getting Help
  • Cinchy Glossary
  • Frequently Asked Questions
  • Deployment Guide
    • Deployment Installation Guides
      • Deployment Planning Overview and Checklist
        • Deployment Architecture Overview
          • Kubernetes Deployment Architecture
          • IIS Deployment Architecture
        • Deployment Prerequisites
          • Single Sign-On (SSO) Integration
            • Enabling TLS 1.2
            • Configuring ADFS
            • AD Group Integration
      • Kubernetes Deployment Installation
        • Disabling your Kubernetes Applications
        • Changing your File Storage Configuration
        • Configuring AWS IAM for Connections
        • Using Self-Signed SSL Certs (Kubernetes Deployments)
        • Deploying the CLI (Kubernetes)
      • IIS Deployment Platform Installation
    • Upgrade Guides
      • Upgrading Cinchy Versions
        • Cinchy Upgrade Utility
        • Kubernetes Upgrades
          • v5.1 (Kubernetes)
          • v5.2 (Kubernetes)
          • v5.3 (Kubernetes)
          • v5.4 (Kubernetes)
          • v5.5 (Kubernetes)
          • v5.6 (Kubernetes)
          • Upgrading AWS EKS Kubernetes Version
          • Updating the Kubernetes Image Registry
          • Upgrading AKS (Azure Kubernetes Service)
        • IIS Upgrades
          • v4.21 (IIS)
          • v4.x to v5.x (IIS)
          • v5.1 (IIS)
          • v5.2 (IIS)
          • v5.3 (IIS)
          • v5.4 (IIS)
          • v5.5 (IIS)
          • v5.6 (IIS)
      • Upgrading from v4 to v5
  • Guides for Using Cinchy
    • User Guides
      • Overview of the Data Browser
      • The Admin Panel
      • User Preferences
        • Personal Access Tokens
      • Table Features
      • Data Management
      • Queries
      • Version Management
        • Versioning Best Practices
      • Commentary
    • Builder Guides
      • Best Practices
      • Creating Tables
        • Attaching Files
        • Columns
        • Data Controls
          • Data Entitlements and Access Controls
          • Data Erasure
          • Data Compression
        • Formatting Rules
        • Indexing and Partitioning
        • Linking Data
        • Table and Column GUIDs
        • System Tables
      • Deleting Tables
        • Restoring Tables, Columns, and Rows
      • Saved Queries
      • CinchyDXD Utility
        • Building the Data Experience (CinchyDXD)
        • Packaging the Data Experience (CinchyDXD)
        • Installing the Data Experience (CinchyDXD)
        • Updating the Data Experience (CinchyDXD)
        • Repackaging the Data Experience (CinchyDXD)
        • Reinstalling the Data Experience (CinchyDXD)
      • Multi-Lingual Support
      • Integration Guides
    • Administrator Guide
    • Additional Guides
      • Monitoring and Logging on Kubernetes
        • Grafana
        • Opensearch Dashboards
          • Setting up Alerts
        • Monitoring via ArgoCD
      • Maintenance
      • System Properties
      • Enable Data At Rest Encryption
      • MDQE
      • Application Experiences
        • Network Map
          • Custom Node Results
          • Custom Results in the Network Map
        • Setting Up Experiences
  • API Guide
    • API Overview
      • API Authentication
      • API Saved Queries
      • ExecuteCQL
      • Webhook Ingestion
  • CQL
    • The Basics of CQL
      • CQL Examples
      • CQL Functions Master List
      • CQL Statements Overview
        • Cinchy DML Statements
        • Cinchy DDL Statements
      • Cinchy Supported Functions
        • Cinchy Functions
        • Cinchy System Values
        • Cinchy User Defined Functions
          • Table-Valued Functions
          • Scalar-Valued Functions
        • Conversion Functions
        • Date and Time Types and Functions
          • Return System Date and Time Values
          • Return Date and Time Parts
          • Return Date and Time Values From Their Parts
          • Return Date and Time Difference Values
          • Modify Date and Time Values
          • Validate Date and Time Values
        • Logical Functions
        • Mathematical Functions
        • String Functions
        • Geometry and Geography Data Type and Functions
          • OGC Methods on Geometry & Geography Instances
          • Extended Methods on Geometry & Geography Instances
        • Full Text Search Functions
        • Connections Functions
        • JSON Functions
  • Meta Forms
    • Introduction to Meta-Forms
    • Meta-Forms Deployment Installation Guide
      • Deploying Meta-Forms (Kubernetes)
      • Deploying Meta-Forms (IIS)
    • Forms Data Types
    • Meta-Forms Builders Guides
      • Creating a Dynamic Meta-Form (Using Tables)
      • Creating a Dynamic Meta-Form Example (Using Form Designer)
      • Adding Links to a Form
      • Rich Text Editing in Forms
  • Data Syncs
    • Getting Started with Data Syncs
    • Installation & Maintenance
      • Prerequisites
      • Installing Connections
      • Installing the Worker/Listener
      • Installing the CLI and the Maintenance CLI
    • Building Data Syncs
      • Types of Data Syncs
      • Common Design Patterns
      • Sync Behaviour
      • Columns and Mappings
        • Calculated Column Examples
      • Listener Configuration
      • Advanced Settings
        • Filters
        • Parameters
        • Auth Requests
        • Request Headers
        • Post Sync Scripts
        • Pagination
      • Batch Data Sync Example
      • Real-Time Sync Example
      • Scheduling a Data Sync
      • Connection Functions
    • CLI Commands List
    • Error Logging and Troubleshooting
    • Supported Data Sync Sources
      • Cinchy Event Broker/CDC
        • Cinchy Event Broker/CDC XML Config Example
      • Cinchy Table
        • Cinchy Table XML Config Example
      • Cinchy Query
        • Cinchy Query XML Config Example
      • Copper
      • DB2 (Query and Table)
      • Dynamics 2015
      • Dynamics
      • DynamoDB
      • File Based Sources
        • Binary File
        • Delimited File
        • Excel
        • Fixed Width File
        • Parquet
      • Kafka Topic
        • Kafka Topic Example Config
        • Apache AVRO Data Format
      • LDAP
      • MongoDB Collection
        • MongoDB Collection Source Example
      • MongoDB Collection (Cinchy Event Triggered)
      • MS SQL Server (Query and Table)
      • ODBC Query
      • Oracle (Query and Table)
      • Polling Event
        • Polling Event Example Config
      • REST API
      • REST API (Cinchy Event Triggered)
      • SAP SuccessFactors
      • Salesforce Object (Bulk API)
      • Salesforce Platform Event
      • Salesforce Push Topic
      • Snowflake
        • Snowflake Source Example Config
      • SOAP 1.2 Web Service
    • Supported Data Sync Destinations
      • Cinchy Table
      • DB2 Table
      • Dynamics
      • Kafka Topic
      • MongoDB Collection
      • MS SQL Server Table
      • Oracle Table
      • REST API
      • Salesforce Object
      • Snowflake Table
      • SOAP 1.2 Web Service
    • Supported Real-Time Sync Stream Sources
      • Cinchy Event Broker/CDC
      • Data Polling
      • Kafka Topic
      • MongoDB
      • Salesforce Push Topic
      • Salesforce Platform Event
  • Other Resources
    • Angular SDK
    • JavaScript SQK
Powered by GitBook
On this page

Was this helpful?

Export as PDF
  1. Data Syncs
  2. Building Data Syncs
  3. Advanced Settings

Pagination

PreviousPost Sync ScriptsNextBatch Data Sync Example

Last updated 2 years ago

Was this helpful?

1. Pagination

When syncing a Data Source, you may have the option to add in additional configuration sections, such as an Pagination, under the "Add a Section" drop down tab in the Connection Experience (Image 1).

There are two types of pagination available (Image 2):

  • Cursor: The cursor is considered to be a key parameter in this type of pagination. You receive a variable named Cursor along with the response. It is a pointer that points at a particular item that needs to be sent with a request. The server then uses the cursor to seek the other set of items. Cursor-based pagination is more complex and is preferred when dealing with a real-time data set.

  • Offset: Offset-based pagination is for parameters with a specific limit (the number of results) and offset (the number of records that need to be skipped). Offset-based pagination is easy to use and is preferred for static data.

Cursor Pagination

To set up cursor pagination, fill in the following parameters (Image 3):

  • Type: Select Cursor

  • Next Page URL JSON Path: This is the JSON Path within the response to the URL for the next page

  • Cursor Key: This is the key used in the query string to specify the cursor value. This is only required if the cursor returned is not a fully qualified URL.

Offset Pagination

To set up offset pagination, fill in the following parameters (Image 4):

  • Type: Select Offset

  • Limit Key: The key used in the query string to specify the limit

  • Limit: The desired page size

  • Offset By: The type of offset that the API does pagination by. This will be either Record Number or Page Number.

  • Offset Key: The key used in the query string to specify the offset

  • Initial Offset: The starting offset

Note that a pagination block is mandatory even if the API does not return results from multiple pages. You can use the following as the placeholder:

<Pagination type="OFFSET" limitField="" offsetField="" limit="0" initialOffset="0" />

<RestApiDataSource
    method="GET or POST"
    format="JSON"
    rootPath="$.result"
    endpoint="https://data.com/api/v1/records/@recordId/details?apiKey=@apiKey">
	<!-- optional -->
  <RequestHeaders>
		<Header name="" isEncrypted="true or false">value goes here</Header>
	</RequestHeaders>
    <!-- optional -->
	<Body>body goes here, only relevant for POST operations</Body>
	<!-- one of the following pagination blocks is required. If there's no pagination you still need a placeholder -->
	<Pagination type="OFFSET" 
	    limitField="name of the field in the querystring to use when passing the limit" 
	    offsetField="name of the field in the querystring to use when passing the offset"
	    limit="20"
	    initialOffset="0">
	<Pagination type="CURSOR"
	    pathInResponse="JSON path to the field in the response containing the next cursor value"
	    cursorPropertyName="name of the field in the querystring to use when passing the cursor">
  <Schema>
			<Column name="$.id" dataType="Text" />
			<Column name="$.variants.field1" dataType="Text" />
            <!-- you need to use the "item" placeholder if you're looking for fields in an array -->
			<Column name="$.colors.item[1]" dataType="Text" />
      <Column name="$.colors.item[2]" dataType="Text" />
	</Schema>
</RestApiDataSource>
Image 1: Adding a Pagination Block
Image 2: Selecting your type of pagination
Image 3: Cursor Pagination
Image 4: Offset Pagination