There may be times when you want to temporarily disable your Kubernetes pods in order to perform maintenance or certain upgrades. You can do so through the following steps:
Access your ArgoCD
Navigate to the application directory for the namespace you wish to disable, in this case development-cinchy (Image 1). You should see your cluster component applications.
3. Click on the main application (i.e. development-cinchy) (Image 2).
4. Navigate to Summary > Sync Policy > Automated. Click on Disable Auto-Sync > OK (Image 3).
5. For each of the cluster applications that you wish to disable, click on the "..." > Delete (Image 5).
6. Your apps should all appear as "out of sync" (Image 6).
To re-enable your applications, return to the application directory for your disabled namespace (Image 7).
2. Click on the main application (i.e. development-cinchy) (Image 8).
3. Navigate to Summary > Sync Policy. Click on Enable Auto-Sync > OK (Image 9).
This page details how to change your File Storage configuration in Cinchy v5 to S3, Azure Blob, or Local.
In Cinchy v5.2, we implemented the ability to free up database space by using S3 compatible or Azure Blob Storage for file storage. This configuration is set in the deployment.json of a Kubernetes installation, or the appsettings.json of an IIS installation.
If you are using a Kubernetes deployment, you will change your file storage config in the deployment.json.
Navigate to the object storage section, where you will see either S3 or Azure Blob storage, depending on your deployment structure.
3. To utilize Blob Storage or S3, update each line with your own parameters.
4. To utilize Local storage, leave each line blank with the exception of the Connections_Storage_Type, which should be set to Local:
5. Run the deployment script by using the following command in the root directory of your devops.automations repo:
6. Commit and push your changes.
If you are using an IIS deployment, you will change your file storage config in the Cinchy Web appsettings file.
2. Locate the StorageType section of the file and set it to either "Local", "AzureBlobStorage" or "S3".
3. If you selected "AzureBlobStorage", fill out the following lines in the same file:
4. If your selected "S3", fill out the following lines in the same file:
This page details the optional steps that you can take to use self-signed SSL Certificates in a Kubernetes Deployment of Cinchy.
This process needs to be followed after running the devops.automations script during your initial deployment, as well as each additional time that you run the script (Ex: updating your Cinchy platform), since it will wipe out all of the custom configuration you set up to use a self-signed certificate.
Generate the self-signed certificate by executing the following commands in any folder:
2. Create a yaml file located at cinchy.kubernetes/platform_components/base/self-signed-ssl-root-ca.yaml.
3. Add the following to the yaml file:
4. Add the self signed root CA cert file to the cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/base folder.
5. Add the yaml code snippet to the cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/base/kustomization.yaml file, changing the below files key value as per your root ca cert file name:
6. Add the following line to the cinchy.kubernetes/platform_components/base/kustomization.yaml file
7. Add the below Deployment patchesJson6902 to each of your cinchy.kubernetes/environment_kustomizations/cinchy_nonprod/ENV_NAME/PLATFORM_COMPONENT_NAME/kustomization.yaml files, except "base".
Ensure that the rootCA.crt file name is matched with ConfigMap data, configMapGenerator files, and the patch subpath.
8. Once the changes are deployed, verify the root CA cert is available on the pod under /etc/ssl/certs with below command, inputing your own POD_NAME and NAMESPACE where noted:
Beginning in Cinchy v5.6, you are now able to run the Connections pod under a service account that uses an AWS IAM (Identity and Access Management) role, which is an IAM identity that you can create to have specific permissions and access to your AWS resources. To set up AWS IAM role authentication, please review the procedure below.
To check that you have an OpenID Connect set up with the cluster (the default for deployments made using the Cinchy automation process), run the below command within a terminal:
The output should appear like the below. Make sure to note this down for later use.
Log in to your AWS account and through the AWS UI. Ensure that it has S3 access.
Run the below command in a terminal to create a service account with the role created in step 3. If your cluster has a special character like an underscore, skip to section 1.1.
If your cluster name has a special character, like an underscore, you will need to create and apply the yaml. Follow section 1 up until step 4, and then follow the below procedure.
In an editor like Visual Code or similar, create a new file titled "my-service-account.yaml" in your working directory. It should contain the below content.
In a terminal, run the below command:
In an editor like Visual Code or similar, create a new file titled "trust-relationship.json" in your working directory. It should contain the below content.
For example,
Execute the following command to create the role, referencing the above .json file:
For example,
Execute the following command to attach the IAM policy to your role:
For example,
Execute the following command to annotate your service account with the Amazon Resource Name (ARN) of the IAM role that you want the service account to assume:
For example,
Confirm that the role and service account are configured correctly by verifying the ouput of the following commands:
To ensure that the Connections pod's role has the correct permissions, the role specified by the user in AWS must have its Trusted Relationships configured as such:
To confirm that the Connections app is using the service account:
Navigate to the cinchy.kubernetes repo > connections/kustomization.yaml file
Execute the following:
From a terminal, run the below command:
The output should look like the following:
This page details the installation instructions for deploying Cinchy v5 on Kubernetes
| Table of Contents |
|---|
This page details the instructions for deployment of Cinchy v5 on Kubernetes. We recommend, and have documented below, that this is done via Terraform and ArgoCD. This setup involves a utility to centralize and streamline your configurations.
The Terraform scripts and instructions provided enable deployment on Azure and AWS cloud environments.
The following lists are required prerequisites for installing Cinchy v5 on Kubernetes.
Note that some prerequisites will depend on whether you are deploying on Azure or on AWS.
These prerequisites apply whether you are installing on Azure or on AWS.
cinchy.terraform: This repo contains all Terraform configurations.
cinchy.argocd: This repo contains all ArgoCD configurations.
cinchy.kubernetes: This repo contains cluster and application component deployment manifests.
cinchy.devops.automations: This repo contains the single configuration file and binary utility that maintains the contents of the above three repositories.
You must have a service account with read/write permissions to the git repos created above.
The following tools should be installed on the machine where the deployment will run:
Starting in Cinchy v5.4, you will have the option between Alpine or Debian based image tags for the listener, worker, and connections. Using Debian tags will allow a Kubernetes deployment to be able to connect to a DB2 data source, and that option should be selected if you plan on leveraging a DB2 data sync.
When either installing or upgrading your platform, you can use the following Docker image tags for the listener, worker, and connections:
"5.x.x" - Alpine
"5.x.x-debian" - Debian
You will need a single domain for accessing ArgoCD, Grafana, Opensearch Dashboard, and any deployed Cinchy instances. There are two routing options for accessing these applications - path based or subdomains. See below for an example with multiple cinchy instances:
The following prerequisites are required if you are deployment Cinchy v5 on Azure.
Terraform Backend Requirements:
A resource group that will contain the Azure Blob Storage with the terraform state.
A storage account and container (Azure Blob Storage) for persisting terraform state.
The deployment template has the option of either leveraging an existing resource group or creating a new one:
If an existing resource group is preferred, the prerequisite requires the following be provisioned in advance of the deployment:
The resource group.
A VNet within the resource group.
A single subnet. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /22 to provide a range of 1024 IPs.
If a new resource group is preferred, all resources will be automatically provisioned.
The quota limit of the Total Regional vCPUs and the Standard DSv3 Family vCPUs (or equivalent) must provide sufficient availability for the required number of vCPUs (minimum of 24).
An AAD user account to connect to Azure, which has the necessary privileges to create resources in any existing resource groups and the ability to create a resource group (if required).
The following prerequisites are required if you are deployment Cinchy v5 on AWS.
Terraform Backend Requirements:
The template has the option of either leveraging an existing VPC or creating a new one:
If an existing VPC is preferred, the prerequisite requires the following be provisioned in advance of the deployment:
The VPC. It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /21 to provide a range of 2048 IPs.
3 Subnets (one per AZ). It's important that the address range be sufficient for all executing processes within the cluster, e.g. a CIDR ending with /23 to provide a range of 512 IPs.
If the subnets are private, a NAT Gateway is required to enable node group registration with the EKS cluster.
If a new VPC is preferred, all resources will be automatically provisioned.
The limit of the Running On-Demand All Standard vCPUs must provide sufficient availability for the required number of vCPUs (minimum of 24).
An IAM user account to connect to AWS which has the necessary privileges to create resources in any existing VPC and the ability to create a VPC (if required).
Tips for Success:
Ensure that your region is configured the same across your SSL Certificate, your Terraform bucket, and your deployment.json in the next step of this guide.
The following steps detail the instructions for setting up the initial configurations.
Navigate to your cinchy.devops.automations repository where you will see an aws.json and azure.json.
Depending on the cloud platform that you are deploying to, select the appropriate file and copy it into a new file named deployment.json (or <cluster name>.json) within the same directory.
This file will contain the configuration for the infrastructure resources and the Cinchy instances to deploy. Each property within the configuration file has comments in-line describing its purpose along with instructions on how to populate it.
Follow the guidance within the file to configure the properties.
5. Commit and push your changes.
Tips for Success:
You can return to this step at any point in the deployment process if you need to update your configurations. Simply rerun through the guide sequentially after making any changes.
This utility updates the configurations in the cinchy.terraform, cinchy.argocd, and cinchy.kubernetes repositories.
From a shell/terminal, navigate to the cinchy.devops.automations directory location and execute the following command:
3. The console output should terminate with a "Completed successfully".
The following steps detail how to deploy Terraform.
If deploying on AWS: Within the Terraform > AWS directory, a new folder named eks_cluster is created. Nested within that is a subdirectory with the same name as the newly created cluster.
To perform terraform operations, the cluster directory must be the working directory during execution. This applies to everything within step 4 of this guide.
If deploying on Azure: Within the Terraform > Azure directory, a new folder named aks_cluster is created. Nested within that is a subdirectory with the same name as the newly created cluster.
To perform terraform operations, the cluster directory must be the working directory during execution.
Launch a shell/terminal with the working directory set to the cluster directory within the cinchy.terraform repo.
2. If you are using AWS, run the following commands to authenticate the session:
3. If using Azure, run the following command and follow the on screen instructions to authenticate the session:
Execute the following command to create the cluster:
2. Type yes when prompted to apply the terraform changes.
The resource creation process can take approx. 15-20 minutes. At the end of the execution there will be a section with the following header
======= Output Variables =======
If deploying on AWS, this section will contain 2 values: Aurora RDS Server Host and Aurora RDS Password
If deploying on Azure, this section will contain a single value: Azure SQL Database Password
These variable values are required to update the connection string within the deployment.json file (or equivalent) in the cinchy.devops.automations repo.
The following section breaks down how to retrieve your SSH keys for both AWS and Azure deployments.
SSH keys should be saved for future reference in the event that a connection needs to be established directly to a worker node in the Kubernetes cluster.
The SSH key to connect to the Kubernetes nodes is maintained within the terraform state and can be retrieved by executing the following command:
The SSH key is output to the directory containing the cluster terraform configurations.
The following section pertains to updating the Deployment.json file.
Each object within represents an instance that will be deployed on the cluster. Each instance configuration contains a database_connection_string property. This has placeholders for the host name and password that must be updated using output variables from the previous section.
Note that in the case of an Azure deployment, the host name is not available as part of the terraform output and instead must be sourced from the Azure Portal.
The terraform script will create an S3 bucket for the cluster that must be accessible to the Cinchy application components.
To access this programmatically, an IAM user that has read/write permissions to the new S3 bucket is required. This can be an existing user.
The Access Key and Secret Access Key for the IAM user must be specified under the object_storage section of the deployment.json
Within the deployment.json, the azure_blob_storage_conn_str must be set.
The in-line comments outline the commands required to source this value from the Azure CLI.
If you have the key_vault_secrets_provider_enabled=true value in the azure.json then the below secrets files would have been created during the execution of step 3.2:
You will need to add the following secrets to your azure key vault:
worker-secret-appsettings-<cinchy_instance_name>
web-secret-appsettings-<cinchy_instance_name>
maintenance-cli-secret-appsettings-<cinchy_instance_name>
idp-secret-appsettings-<cinchy_instance_name>
forms-secret-config-<cinchy_instance_name>
event-listener-secret-appsettings-<cinchy_instance_name>
connections-secret-config-<cinchy_instance_name>
connections-secret-appsettings-<cinchy_instance_name>
To create your new secrets:
Navigate to your key vault in the Azure portal.
Open your Key Vault Settings and select Secrets.
Select Generate/Import.
On the Create a Secret screen, choose the following values:
Upload options: Manual.
Name: Choose the secret name from the above list. They will all follow the format of: <app>-secret-appsettings-<cinchy_instance_name> or <app>-secret-config-<cinchy_instance_name>
Value: The value for the secret will be the content of each app JSON located in the cinchy.kubernetes\environment_kustomizations\nonprod<cinchy_instance_name>\secrets folder, and as shown in above image.
Content type: JSON
Leave the other values to their defaults.
Select Create.
Once you receive the message that the first secret has been successfully created, you may proceed to create the other secrets. There are a total of 8 secrets to create as shown in the above list of secret names.
This utility updates the configurations in the cinchy.terraform, cinchy.argocd, and cinchy.kubernetes repositories.
From a shell/terminal, navigate to the cinchy.devops.automations directory and execute the following command:
3. The console output should terminate with a "Completed successfully".
4. The updates must be committed to Git before proceeding to the next step.
From a shell/terminal run the following command, replacing <region> and <cluster_name> with the accurate values for those placeholders:
From a shell/terminal run the following commands, replacing <subscription_id>, <deployment_resource_group>, and <cluster_name> with the accurate values for those placeholders.
These commands with the values pre-populated can also be found from the Connect panel of the AKS Cluster in the Azure Portal.
Verify that the connection has been established and the context is the correct cluster by running the following command:
In this step, we will deploy and access ArgoCD.
Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.
Execute the following command to deploy ArgoCD:
3. Monitor the pods within the argocd namespace by running the following command every 30 seconds until they all move into a healthy state:
Launch a new shell/terminal with the working directory set to the root of the cinchy.argocd repository.
Execute the following command to access ArgoCD:
This script creates a port forward using kubectl to enable ArgoCD to be accessed at http://localhost:9090
In this step, you will deploy your cluster components.
Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.
Execute the following command to deploy the cluster components using ArgoCD:
3. Navigate to ArgoCD at http://localhost:9090 and login. Wait until all components are healthy (this may take a few minutes).
Tips for Success:
If your pods are degraded or failed to sync, refresh of resync your components. You can also delete pods and ArgoCD will automatically spin them back up for you.
Check that ArgoCD is pulling from your git repo by navigating to your Settings
If your components are failing upon attempting to pull an image, refer to your deployment.json to check that each component is set to the correct version number.
Execute the following command to get the External IP used by the istio ingress gateway.
2. DNS entries must be created using the External IP for any subdomains / primary domains that will be used, including Opensearch, Grafana, and ArgoCD.
The default path to access Opensearch, unless you have configured it otherwise in your deployment.json, is <baseurl>/dashboard
The default credentials for accessing Opensearch are admin/admin. We recommend that you change these credentials the first time you log in to Opensearch.
The default path to access Grafana, unless you have configured it otherwise in your deployment.json, is <baseurl>/grafana
The default username is admin. The default password for accessing Grafana can be found by doing a search of "adminPassword" within the following path: cinchy.kubernetes/cluster_components/metrics/kube-prometheus-stack/values.yaml
We recommend that you change these credentials the first time you access Grafana. You can do so through the admin profile once logged in.
In this step, you will deploy your Cinchy components.
Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.
Execute the following command to deploy the Cinchy application components using ArgoCD:
3. Navigate to ArgoCD at http://localhost:9090 and login. Wait until all components are healthy (may take a few minutes)
You have now finished the deployment steps required for Cinchy. Navigate to your configured domain URL to verify that you can login using the default username (admin) and password (cinchy).
If ArgoCD Application Sync is stuck waiting for PreSync jobs to complete, you can run the below command to restart the application controller.
For further reference material,
The following four Git repositories must be created. Any source control platform supporting Git may be used, e.g. , ,
Download the artifacts for the four Git repositories. Check the contents of each of the directories into the respective repository.
For an introductory guide on Terraform + AWS,
For an introductory guide on Terraform + Azure,
(v1.23.0+)
( may be used on Windows)
If you are using Cinchy's docker images,
| Application | Path Based Routing | Subdomain Based Routing |
|---|
You will need an SSL certificate for the cluster. This should be a wildcard certificate if you will use subdomain based routing.
The should be installed on the machine where the deployment will be run. It must be set to the correct profile/login
that will contain the terraform state.
The should be installed on the machine where the deployment will be run. It must be set to the correct profile/login
The SSL certificate must be , or a new certificate can be requested via
If your are importing it, you will need the PEM-encoded certificate body and private key. You can find this, you can get the PEM file from your chosen domain provider (GoDaddy, Google, etc.)
The deployment.json will ask for your repo username and password, however ArgoCD may encounter errors when retrieving your credentials in certain situations (ex: if using Github). You can verify if your credentials have been picked up or not by navigating to the ArgoCD Settings once you have deployed Argo in step 7 of this guide. To avoid errors, we recommend using a
2. If the file created in has a name other than "deployment.json", the reference in the command will will need to be replaced with the correct name of the file.
Navigate to the the deployment.json () > cinchy_instance_configs section.
2. If the file has a name other than "deployment.json", the reference in the command will will need to be replaced with the correct name of the file.
The credentials for ArgoCD's portal are output at the start of the access_argocd's script execution in Base64. to get the login credentials to use for the http://localhost:9090 endpoint.
To change the default credentials for Cinchy v5.4+,
To change the default credentials and/or add new users for all other deployments, follow or navigate to Settings > Internal Roles in Opensearch.”
4. You will be able to access ArgoCD through the URL that you configured in your deployment.json, as long as you created a DNS entry for it in step
Cinchy 1 (Dev) | domain.com/dev | dev.mydomain.com |
Cinchy 2 (QA) | domain.com/qa | qa.mydomain.com |
Cinchy 3 (UAT) | domain.com/uat | uat.mydomain.com |
ArgoCD | domain.com/argocd | cluster.mydomain.com/argocd |
Grafana | domain.com/grafana | cluster.mydomain.com/grafana |
Opensearch | domain.com/dashboard | cluster.mydomain.com/dashboard |
