This page details the deployment architecture of Cinchy v5 when running on Kubernetes.
The diagram below shows a high level overview of a possible Infrastructure diagram with components on the cluster, but your specific configuration may vary (Image 1).
Tip: Click on an image to enlarge it.
When deploying Cinchy version 5 on Kubernetes, you may deploy via Amazon Web Services (AWS). The diagram below shows a high level overview of a possible AWS Infrastructure with components outside the cluster, but your specific configuration may vary (Image 2).
Tip: Click on an image to enlarge it.
When deploying Cinchy v5 on Kubernetes, you may deploy via Microsoft Azure. The diagram below shows a high level overview of possible Azure Infrastructure with components outside the cluster, but your specific configuration may vary (Image 3).
Tip: Click on an image to enlarge it.
The following highlighted area provides a high-level overview of cluster level components used when deploying Cinchy on Kubernetes and what versions they're running.
These are created once per cluster. Clients may choose to run these components outside of the cluster or replace with their own comparable components. This diagram shows them in the cluster (Image 4).
Tip: Click on an image to enlarge it.
These are created once per cluster. Clients may choose to run these components outside of the cluster or replace with their own comparable components.
Service Mesh - Istio: Istio handles and routes all inbound traffic to your Cinchy instance, keeping it secure and managed.
Monitoring/Alerting - Prometheus & Grafana: Prometheus consumes metrics from the running components in your environment, which you can visualize into user friendly graphs and dashboards by Grafana. Prometheus can also connect to third party services to provide alerting capabilities. Both Prometheus and Grafana use persistent storage.
Logging - OpenSearch and Fluent Bit: OpenSearch captures and indexes all logs in a single, accessible location. These logs can be queried, searched, and filtered, and Correlation IDs mean that they can also be traced across various components. These logging components take advantage of persistent storage.
Caching - Redis: Redis facilitates a distributed lock using RedLock, which guarantees lock synchronizations across Cinchy instances. It's also a storage location for the execution output when running batch data syncs.
Event Processing - Kafka: This acts as the middleware for messaging between components through a queuing mechanism. Kafka features persistent storage.
Before you deploy Cinchy on Kubernetes, consider the following about your cluster configuration :
How many clusters will you need?
Will you be sharing from an existing cluster?
Will you be running multiple environments on a single cluster?
Each Cinchy instance uses the following components to either provide an experience to users/applications or connect data in/out of Cinchy. You can deploy multiple Cinchy instances per cluster, so these components will repeat for each environment.
The following highlighted area provides a high-level overview of instance level components used when running Cinchy on Kubernetes (Image 5).
Tip: Click on an image to enlarge it.
Meta Experiences: Cinchy offers pre-packaged experiences that you can import into your Cinchy environment and use on your data network. This includes experiences like Meta-Forms and Meta-Reports.
Connections: Use the Cinchy Connections experience to create data syncs in/out of the platform. It features persistent storage.
Data Browser: Cinchy’s data collaboration platform features a Universal Data Browser that allows users to view, change, analyze, and otherwise interact with all data on the network. The Data Browser even enables non-technical business users to manage and update data, build models, and set controls, all through an easy and intuitive UI.
Identity Provider: An Identity Provider (IdP) creates and manages user credentials and associated identity attributes. Cinchy uses IdPs authentication services to authenticate end-users.
Event Listener: The Event Listener picks up events from connected sources during a data sync. Review the Data Sync page for further information on the Event Listener. The Event Listener uses persistent storage.
Event Stream Worker: The Event Stream Worker processes data picked up by the Event Listener during data syncs. Review the Data Sync page for further information on the Event Stream Worker. The Event Worker uses persistent storage.
Maintenance (Batch Jobs): Cinchy performs maintenance tasks through the CLI. This includes the data erasure and data compression deletions.
ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes that simplifies the application deployment and lifecycle management. ArgoCD is highly recommended for deploying Cinchy, but you can also use another tool.
Once you set up the configurations, ArgoCD automates the deployment of the desired application states into your specified target environments. Implemented as a Kubernetes controller, it continuously monitors running applications and compares the current, live state against the desired target state (as specified in your repositories).
This page details the deployment architecture of Cinchy v5 when running on a VM.
The below diagram shows a high level overview of Cinchy's Infrastructure components when deploying on IIS.
Some components and configurations are dependent on the platform usage. The table below provides a description of each component.
Tip: Click on an image to enlarge it.
| Component | Description | Technology Stack | Dependencies |
|---|---|---|---|
Cinchy Web Application
This is the primary application for Cinchy, providing both the UI for end users as well as the REST APIs that serve application integration needs. The back-end holds the engine that powers Cinchy's data / metadata management functionality.
ASP.NET MVC 5
.NET Framework 4.7.2+IIS 7.5+Windows Server 2012 or later
Cinchy IdP
This is an OpenID Connect / OAuth 2.0 based Identity Provider that comes with Cinchy for authenticating users. Cinchy supports user group management directly on the platform, but can also connect into an existing IdP available in the organization if it can issue SAML tokens. Optionally, Active Directory groups may be integrated into the platform. When using SSO, this component federates authentication to the customer's SAML enabled IdP. This centralized IdP issues tokens to all integrated applications including the Cinchy web app as well as any components accessing the REST based APIs.
.Net Core 2.1
.NET Framework 4.7.2+IIS 7.5+Windows Server 2012 or later
Cinchy Database
All data managed on Cinchy is stored in a MS SQL Server database. This is the persistence layer.
MS SQL Server Database
Windows Server 2012 or laterMS SQL Server 2012 or later
Cinchy CLI
The CLI offers utilities to get data in and out of Cinchy. It has tools to sync data from a source into a table in Cinchy. it can operate on large datasets with built-in partitioning capability and performs a reconciliation to determine differences before applying changes. Another utility is the data export, which invokes a query against the Cinchy platform and dumps the results to a file for distribution to other systems requiring batch data feeds.
.NET Core 2.0
.NET Core Runtime 2.0.7+ (on Windows or Linux)
ADO.NET Driver
For .NET applications Cinchy provides an ADO.NET driver that can be used to connect into the platform and perform CRUD operations on data.
.NET Standard 2.0
See implementation support table here
JavaScript SDK
Cinchy's JavaScript SDK for front-end developers looking to create an application that can integrate with the Cinchy platform to act as it's middle-tier and back end.
JavascriptJQuery
Angular SDK
Cinchy's Angular SDK for front-end developers looking to create an application that can integrate with the Cinchy platform to act as it's middle-tier and back end.
Angular 5
This page provides an overview for the deployment architecture of Cinchy v5.
When choosing to deploy Cinchy version 5, you must decide whether to deploy via Kubernetes or on a VM (IIS).
Kubernetes is an open-source system that manages and automates the full lifecycle of container-based applications. You now have the ability to deploy Cinchy v5 on Kubernetes, which helps to simplify your deployment and enhance your scaling. Kubernetes can maximize your container capacity and scale up/down with your current operations.
Grafana, OpenSearch, OpenSearch Dashboard: Working together, these three applications provide a visual logging dashboard for all the information coming in from your database pods. This streamlines your search for information by putting the control into your hands and compiling your logs in one easy to access place — you can now write a query against all your logs, in all your environments. You will have access to a default configuration out of the box, but you can also customize your dashboards as well.
Prometheus: With the Kubernetes addition, you now have access to Prometheus for your metrics dashboard. Prometheus records real-time metrics in a time series database used for event monitoring and alerting. You can then create custom dashboards to display your data in an easy to use visual that makes reporting on your metrics easy, and even set up push alerts based on your custom needs.
You also have the option to run Cinchy on Microsoft IIS, which was the traditional deployment method before Cinchy v5. Internet Information Services (IIS) for Windows Server is a flexible, secure and manageable Web server for hosting anything on the Web.
We recommend using Kubernetes to deploy Cinchy v5, because of the robust features that you can leverage, such as improved logging and metrics. Kubernetes helps scale your Cinchy instances and lower your costs by using PostgreSQL.
Before deploying Cinchy v5, you must select which database you want to use.
The following list outlines the supported databases for Kubernetes Deployments.
For IIS Deployments please review the architecture requirements here.
Microsoft SQL Server is a relational database management system. As a database server, it's a software product with the primary function of storing and retrieving data as requested by other software applications—which may run either on the same computer or on another computer across a network.
Microsoft Azure SQL Database is a managed cloud database provided as part of Microsoft Azure. It runs on a cloud computing platform, and access to it's provided as a service. Managed database services take care of scalability, backup, and high availability of the database.
SQL Managed Instance is a managed, cloud-based, always up-to-date SQL instance that combines broad SQL Server engine compatibility with the benefits of a fully managed PaaS.
Amazon Aurora (Aurora) is a fully managed relational database engine that's compatible with MySQL and PostgreSQL. It combines the speed and reliability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora is part of the managed database service Amazon Relational Database Service (Amazon RDS). Amazon RDS is a cloud web service that makes it easier to set up, operate, and scale a relational database.
PostgreSQL is a free and open-source relational database management system emphasizing extensibility and SQL compliance. PostgreSQL comes with features aimed to help developers build applications, administrators to protect data integrity and build fault-tolerant environments, and help you manage your data no matter how big or small the dataset.
Amazon RDS makes it easy to set up, operate, and scale cloud-based PostgreSQL deployments. With Amazon RDS, you can deploy scalable PostgreSQL deployments with cost-efficient and resizable hardware capacity. Amazon RDS manages complex and time-consuming administrative tasks such as PostgreSQL software installation and upgrades; storage management; replication for high availability and read throughput; and backups for disaster recovery. Amazon RDS for PostgreSQL gives you access to the capabilities of the familiar PostgreSQL database engine.
This is a fully managed and intelligent Azure Database for PostgreSQL. Enjoy high availability with a service-level agreement (SLA) up to 99.99 percent, AI-powered performance optimization, and advanced security. A fully managed database that automates maintenance, patching, and updates. Provision in minutes and independently scale compute or storage in seconds.
Before deploying Cinchy v5, you need to define your sizing requirements.
Cluster sizing recommendations vary and are dependant on a number of deployment factors. We've provided the following general sizing recommendations, but encourage you to explore more personalized options.
CPU: 8 Cores
Memory: 32GB Ram
Number of Servers: 3
AWS: m5.2xlarge
Azure: D8 v3
For sizing recommendations and prerequisites about an IIS deployment, please review the IIS deployment prerequisites.
If you are choosing to deploy Cinchy v5 on IIS, then you need to ensure that your VM disks have enough application storage to run your clusters.
Cinchy supports both Amazon S3 and Azure Blob Storage.
If you are using Terraform for your Kubernetes deployment, you will need to set up a new S3 compatible bucket to manually to store your state file. You will also need a bucket for Connections, to store error files created during data syncs.
You will create your two S3 compatible buckets using either Amazon or Azure. Ensure that you use the following convention when naming your buckets so that the automation script runs correctly: <org>-<component>-<cluster>. These bucket names will be referenced in your configuration files when you deploy Cinchy on Kubernetes.
Example Terraform Bucket: cinchy-terraform-state
Example Connection Bucket: cinchy-connections-cinchy-nonprod
S3 provides unlimited scalability and it charges only for what you use/how much you store on it, so there are no sizing definitions.
