Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by updating the version number in your configuration files and applying the changes in ArgoCD.

Configure to the newest version

1. Navigate to your Cinchy devops.automation repository

   1. Navigate to your deployment.json (You may have renamed this during your original Kubernetes deployment)

   2. In the cinchy_instance_configs section, navigate to the image tags. Replace the version number with the instance that you wish to deploy (Ex: v5.1.0 > 5.2.0).

  // The component image tags are specified below to define which versions to deploy
  "connections_image_tag": "v5.2.0",
  "event_listener_image_tag": "v5.2.0",
  "idp_image_tag": "v5.2.0",
  "maintenance_cli_image_tag": "v5.2.0",
  "meta_forms_image_tag": "v5.2.0",
  "web_image_tag": "v5.2.0",
  "worker_image_tag": "v5.2.0"

1. Rerun the deployment script by using the following command in the root directory of your devops.automations repository:

dotnet Cinchy.DevOps.Automations.dll "deployment.json"

1. Commit and push your changes.

Apply your configurations

If your environment isn't set-up to automatically apply upon configuration, complete the following the apply the newest version:

1. Navigate to the ArgoCD portal.

2. Refresh your component(s). If that doesn't work, re-sync.

The pages under this section deal with upgrading your Cinchy platform.

Additional notes

• If you are currently running Cinchy v4+ or v5+ and wish to upgrade your components, please review the documentation here: Upgrading Cinchy Versions

• If you are currently running Cinchy v4+ and wish to upgrade to v5+, please review the documentation here: Upgrading from v4 to v5

Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by updating the version number in your configuration files and applying the changes in ArgoCD.

Configure to the newest version

1. Navigate to your Cinchy devops.automation repository

   1. Navigate to your deployment.json (You may have renamed this during your original Kubernetes deployment)

   2. In the cinchy_instance_configs section, navigate to the image tags. Replace the version number with the instance that you wish to deploy (Ex: v5.0.0 > 5.1.0).

2. Rerun the deployment script by using the following command in the root directory of your devops.automations repository:

3. Commit and push your changes.

Apply your Configurations

If your environment isn't set-up to automatically apply upon configuration, complete the following the apply the newest version:

1. Navigate to the ArgoCD portal.

2. Refresh your component(s). If that doesn't work, re-sync.

Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by following the below instructions.

Prerequisites

• Download the latest Cinchy Artifacts from the Cinchy Releases Table > Kubernetes Artifacts column (Image 1). For this upgrade, please download the Cinchy v5.5 k8s-template.zip file.

Configure to the newest version

1. Navigate to your cinchy.argocd repository. Delete all existing folder structure except for the .git folder/directory and any custom changes you may have implemented.

2. Navigate to your cinchy.kubernetes repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

1. Navigate to your cinchy.terraform repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

2. Navigate to your cinchy.devops.automation repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented and your deployment.json.

3. Open the new Cinchy v5.4 k8s-template.zip file you downloaded from the Cinchy Releases table and check the files into their respective cinchy.kubernetes, cinchy.argocd, cinchy.terraform and cinchy.devops.automation repositories.

4. Navigate to the new aws.json/azure.json files and compare them with your current deployment.json file. Any additional fields in the new aws.json/azure.json files should be added to your current deployment.json.

1. Open a shell/terminal from the cinchy.devops.automations directory and execute the following command:

1. Commit all your changes (if there were any) in each repository.

2. If there were any changes in your cinchy.argocd repository you may need to redeploy ArgoCD.

   1. Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.

   2. Execute the following command to deploy ArgoCD:

1. If there were any change to the cluster components, execute the following command from the cinchy.argocd repository:

1. If there were any change to the Cinchy instance, execute the following command from the cinchy.argocd repository:

1. Log in to your ArgoCD application console and refresh the apps to ensure that all changes were picked up.

Overview

The pages under this section can be used to help guide you through an upgrade of your Cinchy instance.

Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by updating the version number in your configuration files and applying the changes in ArgoCD.

Configure to the newest version

1. Navigate to your cinchy devops.automation repository

   1. Navigate to your deployment.json (You may have renamed this during your original Kubernetes deployment)

   2. In the cinchy_instance_configs section, navigate to the image tags. Replace the version number with the instance that you wish to deploy (Ex: v5.2.0 > 5.3.0).

1. Rerun the deployment script by using the following command in the root directory of your devops.automations repository:

3. Commit and push your changes.

Apply the configurations

If your environment isn't set-up to automatically apply upon configuration, complete the following the apply the newest version:

1. Navigate to the ArgoCD portal.

2. Refresh your component(s). If that doesn't work, re-sync.

Overview

The Cinchy Upgrade Utility was first introduced in v5.2 to ease a mandatory INT to BigInt upgrade. This tool has continued to be used in subsequent releases as an easy way to deploy necessary changes to your Cinchy platform.

Considerations

• Upgrades will also be specified on the applicable page for each release.

• Depending on your upgrade path, certain upgrades must be performed in sequential and/or specific order. This will be clearly marked in the section.

  • For example: To go from v5.1 to v5.5, you would first have to run the 5.2 upgrade utility and deploy the release. Once validated, you would then run the 5.5 upgrade and deploy that version.

• Not all new releases will have changes that require the utility to be run. Review the table in section 4 for the full list.

Prerequisites

• You will need to run this process as a user with admin/dbowner privileges to your database.

• You will need to have installed on the machine that you will run the utility on.

• Retrieve the Upgrade Utility from the Cinchy Releases table.

Upgrades

Overview and considerations

Upgrade instructions

1. Turn off your Cinchy platform. (Note: This step is only required for the 5.2 upgrade)

   2. In an IIS Deployment:

      1. Open your Windows Services Panel.

      2. Select IIS Admin Service.

      3. Stop the service.

      4. Right-click IIS Admin Service and select Properties.

      5. Change 'Start Up Type' to 'Disabled'.

2. Create a backup snapshot of your platform.
3. Retrieve the Upgrade Utility from the Cinchy Releases table if you haven't already.

4. Run the following command through a command window as an admin/dbowner, using the table below as a guide.

1. You will see the below progress bar as your upgrade completes (Image 1). Once it's done, you will see a VALIDATION PASSED check.

1. If it was turned off in step 1, turn your Cinchy platform back on.

   2. In an IIS deployment:

      1. Open your Windows Services Panel.

      2. Select IIS Admin Service.

      3. Start the service.

      4. Right-click IIS Admin Service and select Properties.

      5. Change 'Start Up Type' to 'Enabled'.

Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by following the below instructions.

Prerequisites

• Download the latest Cinchy Artifacts from the Cinchy Releases Table > Kubernetes Artifacts column (Image 1). For this upgrade, please download the Cinchy v5.4 k8s-template.zip file.

Take your platform offline

Configure to the newest version

1. Navigate to your cinchy.argocd repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

2. Navigate to your cinchy.kubernetes repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

1. Navigate to your cinchy.terraform repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

2. Navigate to your cinchy.devops.automation repository. Delete all existing folder structure except for the .git file and any custom changes you may have implemented.

3. Open the new Cinchy v5.4 k8s-template.zip file you downloaded from the Cinchy Releases table.

4. Navigate to the new aws.json/azure.json files and compare them with your current deployment.json file. Any additional fields in the new aws.json/azure.json files should be added to your current deployment.json.

1. Open a shell/terminal from the cinchy.devops.automations directory and execute the following command:

1. Commit all your changes (if there were any) in each repository.

2. If there were any changes in your cinchy.argocd repository you may need to redeploy ArgoCD.

   1. Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.

   2. Execute the following command to deploy ArgoCD:

1. If there were any change to the cluster components, execute the following command from the cinchy.argocd repository:

1. If there were any change to the Cinchy instance, execute the following command from the cinchy.argocd repository:

1. Log in to your ArgoCD application console and refresh the apps to ensure that all changes were picked up.

Turn on the platform and apply your configurations

Overview

The Kubernetes project runs a community-owned image registry called registry.k8s.io in which to host its container images. On April 3rd, 2023, the registry k8s.gcr.io was deprecated and no further images for Kubernetes and related subprojects are being pushed to this location.

**Instead, there is a new registry: **registry.k8s.io.

New Cinchy Deployments: this change will be automatically reflected in your installation.

For Current Cinchy Deployments: please follow the instructions outlined in this upgrade guide to ensure your components are pointed to the correct image repo.

You can review the full details on this change here: https://kubernetes.io/blog/2023/02/06/k8s-gcr-io-freeze-announcement/

Update Instructions

1. In a shell/terminal, run the below command to get a list of all the pods that are pointing to the old registry: k8s.gcr.io. These will need to be updated to point to the new image registry.

kubectl get pods --all-namespaces -o jsonpath="{.items[*].spec.containers[*].image}" |\
tr -s '[[:space:]]' '\n' |\
sort |\
uniq -c
grep "k8s.gcr.io"

1. Once you find which pods are using old image registry, you need to update its deployment/daemonset/stateful set with new registry.k8s.io registry. Find the right resource type for your pods with below command:

kubectl get deployment,daemonset,statefulset --all-namespaces

1. Once you have your pod name and resource type, run the below command to open the manifest file in an editor:

kubectl edit deployment cluster-autoscaler -n kube-system

1. In the editor, find any instances of k8s.gcr.io and replace it with registry.k8s.io.

2. Save and close the file.

3. Repeat steps 3-5 for the rest of your pods until they're all pointing to the correct registry.

What's new

The major changes for the 5.7 Kubernetes upgrade are the following:

• The Azure AKS and AWS EKS version supports up to 1.27

• Upgraded ArgoCD from 2.1.7 to v2.7.6

• Upgraded Istio from 1.3.1 to 1.18.0

• OpenSearch from 1.2.0 to 2.13.1

• Upgraded Logging Operator from 3.17.2 to 4.2.2

• Upgraded Kube Prometheus Stack from 17.2.2 to 47.0.0

• Upgraded Strimzi Kafka Operator from 0.1.0 to 0.34.0

• New app Kafka UI 0.7.1

• OpenSearch Index creation based on date format

Upgrading on Kubernetes

To upgrade your various components, follow the instructions below in the order presented.

Prerequisites

Upgrade from 5.1 or lower

If you are upgrading from Cinchy v5.1 or lower to Cinchy v5.7, you must run Upgrade 5.2 using the Cinchy Utility and deploy version 5.2.

Upgrade from v5.2 or higher

If you are upgrading from 5.2 or higher, follow the 5.7 upgrade instructions below, then use the Cinchy Utility and deploy the target version using the -v "X.X" argument.

Configure to the newest version

Clean existing repositories

1. Go to your cinchy.argocd repository. Delete all existing folder structure except for the .git folder/directory and any custom changes you may have implemented.

2. Go to your cinchy.kubernetes repository. Delete all existing folder structure except for the .git file.

1. Go to your cinchy.terraform repository. Delete all existing folder structure except for the .git file.

2. Go to your cinchy.devops.automation repository. Delete all existing folder structure except for the .git file and your deployment.json.

Download k8s template

1. Download and open the new Cinchy v5.7 k8s-template.zip file from the Cinchy Releases table and place the files into their respective cinchy.kubernetes, cinchy.argocd, cinchy.terraform and cinchy.devops.automation repositories.

2. Go to the new aws.json/azure.json files and compare them with your current deployment.json file. All additional fields in the new aws.json/azure.json files should be added to your current deployment.json.

3. Update the Kubernetes version in your deployment.json. To upgrade EKS to a new version, you need to follow an upgrade sequence, installing each incremental version one by one. For example, you might need to upgrade from 1.24 to 1.25, then from 1.25 to 1.26, and finally from 1.26 to 1.27.

Remove components

In the 5.7 templates, the cluster-level components will upgrade to the latest version. You need to remove kube-prometheus-stack, logging-operator app and kafka-cluster from ArgoCD. This change deletes your recent metrics from Grafana and you will only see the latest metrics after you deploy the new kube-prometheus-stack. The older CRDs created by kube-prometheus-stack and logging-operator charts aren't removed by default during upgrade and should be manually cleaned up with the below commands:

kubectl delete crd alertmanagerconfigs.monitoring.coreos.com alertmanagers.monitoring.coreos.com podmonitors.monitoring.coreos.com probes.monitoring.coreos.com prometheusagents.monitoring.coreos.com prometheuses.monitoring.coreos.com prometheusrules.monitoring.coreos.com scrapeconfigs.monitoring.coreos.com servicemonitors.monitoring.coreos.com thanosrulers.monitoring.coreos.com

kubectl delete crd clusterflows.logging.banzaicloud.io clusteroutputs.logging.banzaicloud.io flows.logging.banzaicloud.io loggings.logging.banzaicloud.io outputs.logging.banzaicloud.io eventtailers.logging-extensions.banzaicloud.io fluentbitagents.logging.banzaicloud.io hosttailers.logging-extensions.banzaicloud.io nodeagents.logging.banzaicloud.io syslogngclusterflows.logging.banzaicloud.io syslogngclusterflows.logging.banzaicloud.io syslogngclusteroutputs.logging.banzaicloud.io syslogngflows.logging.banzaicloud.io syslogngoutputs.logging.banzaicloud.io

Upgrade and redeploy components

1. Open a shell/terminal from the cinchy.devops.automations directory and execute the following command:

   Copy

   dotnet Cinchy.DevOps.Automations.dll "deployment.json"

2. Commit all of your changes (if there were any) in each repository.

3. If there were any changes in your cinchy.argocd repository you may need to redeploy ArgoCD.Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.

4. Execute the following command to deploy ArgoCD:

   Copy

   bash deploy_argocd.sh

5. Validate ArgoCD pods are running and check that ArgoCD is upgraded v2.7.6 by accessing the ArgoCD application Console.

6. Execute the following command to deploy cluster components and Cinchy components:

   Copy

   bash deploy_cluster_components.sh
   bash deploy_cinchy_components.sh

7. You might see a couple of ArgoCD apps out of sync because you deleted the logging operator. Sync them manually. Redis will take a few minutes to recover.

Upgrade AWS EKS and Azure AKS

To upgrade the AWS EKS and Azure AKS version from 1.24 up to 1.27.x, you have two methods. The method depends on the status of the subnet CIDR range. The CIDR is a blocker for Azure only. For AWS export credentials and for Azure run the az login command, if required.

CIDR range higher than 10.10.0.0/22

If you have AKS subnet CIDR range higher than 10.10.0.0/22 then you can use the 1.25.x and later AKS upgrade without much downtime and AKS resource teardown.

1. Go to your cinchy.devops.automations repository and change AKS/EKS version in deployment.json (or <cluster name>.json) within the same directory.

2. From a shell/terminal, navigate to the cinchy.devops.automations directory location and execute the following command:

   Copy

   dotnet Cinchy.DevOps.Automations.dll "deployment.json"

AWS - Cinchy.terraform repository structure

The AWS deployment updates a folder named eks_cluster in the Terraform > AWS directory. Within that directory is a subdirectory with the same name as the created cluster.

To perform terraform operations, the cluster directory must be the working directory during execution.

Azure - Cinchy.terraform repository structure

The Azure deployment updates a folder named aks_cluster Within the Terraform > Azure directory. Within that directory is a subdirectory with the same name as the created cluster.

For AWS and Azure export credentials run the az login command if required.

Run the command below to start the upgrade process. Make sure to verify before you select yes to upgrade the process. This shouldn't delete or destroy any data. It runs an in-place deployment that will update the Kubernetes version.

bash create.sh

CIDR range lower than 10.10.0.0/22

If you have 10.10.0.0/22 CIDR range or smaller, then you won't be able to upgrade AKS version to 1.25.x. The 10.10.0.0/22 CIDR range gives you 1024 IP addresses, which aren't enough to run more than 4 worker nodes. Most customers already run 3 worker nodes and the upgrade process starts another 3 nodes, which will cause a failure.

The values below gives the suggested IP address CIDR range. Cinchy recommends to make your own choice based on your needs. Update these values in your deployment.json file:

          "address_space": "10.10.0.0/19",
          "subnet_prefix": "10.10.0.0/20"
          "net_profile_service_cidr": "10.1.4.0/22",
          "net_profile_dns_service_ip": "10.1.4.2",

Delete cluster components, Cinchy components, and ArgoCD apps

Make sure you are connected to the appropriate cluster. Before you start the upgrade process of AKS, you must delete all your apps from ArgoCD which will delete Cinchy apps and custom components from ArgoCD, which includes load balancer as well kafka-cluster.

To delete Cinchy apps, cluster components and ArgoCD:

1. From terminal change directory to cinchy.argocd and run the commands below sequentially. Make sure to change your cluster(directory) name and environment name in below commands:

   Copy

   kubectl delete -k environment_kustomizations/CLUSTER_NAME/cluster_components
   kubectl delete -k environment_kustomizations/CLUSTER_NAME/ENVIRONMENT_NAME/cinchy
   kubectl delete -k argocd

2. If cluster components deletion takes longer than 10 minutes, run the command below. Check that all pods are deleted except Kubernetes default namespace(kube-system) pods.

   Copy

   kubectl patch app $(kubectl get applications -n argocd --no-headers | awk '{print $1}') -p '{"metadata":{"finalizers":null}}' --type=merge -n argocd

3. Verify the deletion of pods for all namespaces with the kubectl get pods -A command. If the namespaces and pods aren't deleted for some cluster components, then delete the namespace manually with command kubectl delete ns NAMESPACE.

Deploy

1. Change the AKS version in DevOps automation tools deployment.json against kubernetes_version and orchestrator_version key values. From a shell/terminal, go to the cinchy.devops.automations directory location and execute the following command:

   Copy

   dotnet Cinchy.DevOps.Automations.dll "deployment.json"

2. In the Terraform > Azure directory, a new folder named aks_cluster should be updated with the AKS version. In that directory is a subdirectory with the same name as the newly updated cluster.

3. Launch a shell/terminal with the working directory set to the cluster directory within the cinchy.terraform repository and run az login.

4. Execute the following command to create the cluster:

   Copy

   bash create.sh

Upgrading on Kubernetes

When it comes time to upgrade your various components, you can do so by following the below instructions.

"cinchy_instance_configs": {
          "database_connection_string": "User ID=cinchy;Password=<password>;Host=<db_hostname>;Port=5432;Database=development;Timeout=300;Keepalive=300;TrustServerCertificate=True"}

Prerequisites

• Download the latest Cinchy Artifacts from the Cinchy Releases Table > Kubernetes Artifacts column (Image 1). For this upgrade, please download the Cinchy v5.6 k8s-template.zip file.

• Review the template changes for this upgrade.

Configuring to the newest version

1. Navigate to your cinchy.argocd repository. Delete all existing folder structure except for the .git folder/directory and any custom changes you may have implemented.

2. Navigate to your cinchy.kubernetes repository. Delete all existing folder structure except for the .git file.

1. Navigate to your cinchy.terraform repository. Delete all existing folder structure except for the .git file.

2. Navigate to your cinchy.devops.automation repository. Delete all existing folder structure except for the .git file and your deployment.json.

3. Open the new Cinchy v5.6 k8s-template.zip file you downloaded from the Cinchy Releases table and check the files into their respective cinchy.kubernete, cinchy.argocd, cinchy.terraform and cinchy.devops.automation repositories.

4. Navigate to the new aws.json/azure.json files and compare them with your current deployment.json file. Any additional fields in the new aws.json/azure.json files should be added to your current deployment.json.

"cinchy_instance_configs": {
          "database_connection_string": "User ID=cinchy;Password=<password>;Host=<db_hostname>;Port=5432;Database=development;Timeout=300;Keepalive=300;TrustServerCertificate=True"},

1. Open a shell/terminal from the cinchy.devops.automations directory and execute the following command:

dotnet Cinchy.DevOps.Automations.dll "deployment.json"

1. Commit all of your changes (if there were any) in each repository.

2. If there were any changes in your cinchy.argocd repository you may need to redeploy ArgoCD.

   1. Launch a shell/terminal with the working directory set to the root of the cinchy.argocd repository.

   2. Execute the following command to deploy ArgoCD:

bash deploy_argocd.sh

1. If there were any change to the cluster components, execute the following command from the cinchy.argocd repository:

bash deploy_cluster_components.sh

1. If there were any change to the Cinchy instance, execute the following command from the cinchy.argocd repository:

bash deploy_cinchy_components.sh

1. Log in to your ArgoCD application console and refresh the apps to ensure that all changes were picked up.

Appendix A

Template changes (Kubernetes 5.6)

• The AWS EKS version has been upgraded to support up to v1.24.

• We've added support for AWS EKS EBS volume encryption. By default EKS worker nodes will have gp3 storage class.

  • For current Cinchy environments you must keep your eks_persistent_apps_storage_class to gp2 in your DevOps automation aws.json file.

  • If you want to move to gp3 storage or gp3 storage and volume encryption, you will have to delete any existing volumes/pvc's for Kafka, Redis, OpenSearch, Logging Operator and Event Listener with statefulset. This ensures that ArgoCD will take care of recreation of resources.

  • If your Kafka cluster pods aren't coming back you must restart your Kafka operators.

  • You can verify the change by running the following command: "kubectl get pvc --all-namespaces".

• The Connections app has changed from StatefulSet to Deployment. The persistence volume has changed to emptyDir.

• We've modified the replica count from 1 to 2 for istiod and istio ingress.

• We've disabled the ArgoCD namespace: istio injection.

  • If this is already enabled on your environment you may keep it as is, such as keeping the cinchy.kubernetes/cluster_components/servicemesh/istio/istio-injection/argocd-ns.yaml file as it's without commenting content in it.

• The Istio namespace injection has been removed.

  • If this is already enabled on your environment please keep it as is -- otherwise it will force you to redeploy all of your Kubernetes application components.

• We've upgraded the AWS Secret Manager CSI Driver to the latest version due to crashing pods.

• We've added support for the EKS EBS CSI driver in lieu of using in-tree EBS storage plugin.

• We've changed the EKS Metrics server port number in order to support newer versions of Kubernetes.

• We've set fixed AWS Terraform providers version for all components.

• We've installed the cluster autoscaler from local charts instead of remote charts.

• The deprecated azurerm_sql_server Terraform resource has been changed to azurerm_mssql_server

• The deprecated azurerm_sql_database resource has been changed to azurerm_mssql_database

• The deprecated azurerm_sql_failover_group has been changed to azurerm_mssql_failover_group

• The deprecated azurerm_sql_firewall_rule has been changed to azurerm_mssql_firewall_rule

Overview

AKS (Azure Kubernetes Service) is a managed K8 service used when deploying Cinchy v5 over Azure. Once you have deployed your v5 instance of Cinchy, you may want to upgrade your AKS to a newer version. You can do so with the below guide.

Prerequisites

Before proceeding for the AKS version upgrade, ensure that you have enough IP address space, as this process will need to start new nodes and pods.

You will need more than 1024 IP addresses in total when you have single Cinchy instance/environment.

If you have limited IP address space, you can upgrade the master node and then the worker node pools one by one.

Upgrading AKS

1. Navigate to your ArgoCD Dashboard.

2. Find and click on your Istio app > disable auto-sync.

3. Find and click on your Istio-ingress app > disable auto-sync.

4. Open a terminal and run the following command to get the currently available Kubernetes versions for AKS, inputting your AKS cluster name and resource group where specified:

az aks get-upgrades --resource-group <myresourcegroup> --name <myaksclustername> --output table

1. Within your cinchy.devops.automations folder/repo, navigate to the deployment-azure.json file and change the values for kubernetes_version and orchestrator_version to the currently available Kubernetes version found in step 4.

2. Within the cinchy.devops.automations folder/repo, run the following command to push your new values:

dotnet Cinchy.DevOps.Automations.dll deployment-azure.json

1. Within your cinchy.terraform/azure/aks_cluster/ directory, run bash create.sh and accept the change.

2. AKS has three node pools, one node pool per availability zone. Terraform will now start the upgrade of the master node and the zone1 node pool.

3. Verify which nodes that the istio-ingressgateway and istiod pods are running on with below command.

kubectl get pods -n istio-system -o wide

1. In the output you will see aks-zone1/2/3-xxxxx under the Nodes column.

If one or both pods are running on aks-zone1-xxxxx then you need to scale up replicas 2 by following steps 12-14. If not, skip to step 15.

1. Use the following command to start replica on zone2 or zone3 nodes; once you scale down it will terminate from zone1 upgrading nodes.

kubectl scale --replicas=2 deployment.apps/istio-ingressgateway -n istio-system
kubectl scale --replicas=2 deployment.apps/istiod -n istio-system

1. Verify that new pods are running on other node pool by running the following command:

kubectl get pods -n istio-system -o wide

1. Run the following command to scale it back. This will remove the pod from cordoned node:

kubectl scale --replicas=1 deployment.apps/istio-ingressgateway -n istio-system
kubectl scale --replicas=1 deployment.apps/istiod -n istio-system
kubectl get pods -n istio-system -o wide

1. Once Maser node and zone1 upgrade is done, you will see a message such as:

module.aks.azurerm_kubernetes_cluster.main: Modifications complete after 9m23s [id=/subscriptions/e5d6912-eeaa-461c-8f6-30e7c6d945a/resourceGroups/<vnet>/providers/Microsoft.ContainerService/managedClusters/<myaksclustername>]

1. The Terraform script will continue with the zone2 and zone3 node pool upgrade. You will see messages like the below when thee zone2 and zone3 node pool upgrade is in process:

module.aks-node-pool.azurerm_kubernetes_cluster_node_pool.node_pool["zone2"]: Modifying... [id=/subscriptions/ed16912-eeaa-461c-8f36-30e76de945a/resourceGroups/<vnet>/providers/Microsoft.ContainerService/managedClusters/<myaksclustername>/agentPools/zone2]`

`module.aks-node-pool.azurerm_kubernetes_cluster_node_pool.node_pool["zone3"]: Modifying... [id=/subscriptions/e5d1612-eeaa-461c-8f36-30e7c6e945a/resourceGroups/<vnet>/providers/Microsoft.ContainerService/managedClusters/<myaksclustername>/agentPools/zone3]

1. Repeat steps 11-13. This will make sure that istio-ingressgateway and istiod are running on zone1 nodes.

2. Return to your ArgoCD dashboard and re-enable the auto sync for Istio and Istio-ingress apps auto sync on ArgoCD dashboard.

Overview

This page will guide you through how to update your AWS EKS Kubernetes version for your Cinchy v5 platform.

Prerequisites

• Update your Cinchy platform to the latest version.

• Confirm the latest Cinchy supported version of EKS. You can find the version number in the cinchy.devops.automations\aws-deployment.json as "cluster_version": "1.xx".

Considerations

• You must upgrade your EKS sequentially. For example, if you are on EKS cluster version 1.22 and wish to upgrade to 1.24, you must upgrade from 1.22 > 1.23 > 1.24.

Instructions

1. Navigate to your cinchy.devops.automations\aws-deployment.json file.

2. Change the cluster_version key value to the EKS version you wish to upgrade to. (Example: "1.24")

3. Open a shell/terminal and navigate to the cinchy.devops.automations directory location.

4. Execute the following command:

dotnet Cinchy.DevOps.Automations.dll "deployment.json"

1. Commit changes for all the repositories (cinchy.argocd, cinchy.kubernetes, cinchy.terraform and cinchy.devops.automation).

2. Open a new shell/terminal and navigate to the cinchy.terraform\aws\eks_cluster\CLUSTER_NAME directory location.

3. Execute the following command:

bash create.sh

1. Verify the changes are as expected and then accept.

This process will first upgrade the managed master node version and then the worker nodes. During the upgrade process, existing pods get migrated to new worker nodes and all pods will get migrated to new upgraded worker nodes automatically.

The below two commands can be used to verify that all pods are being migrated to new worker nodes.

• To show both old and new nodes:

kubectl get nodes #

• To show all the pods on the new worker nodes and the old worker nodes.

kubectl get pods --all-namespaces -o wide #

Reinstall the metrics server

For EKS version 1.24, the metrics server goes into a crashed loop status. Reinstalling the metrics server will fix this, should you encounter this during your upgrade.

1. In a code editor, open the cinchy.terraform\aws\eks_cluster\CLUSTER_NAME\new-vpc.tf or existing-VPC.tf file.

2. Find the enable_metrics_server key and mark its value to false.

3. Open a new shell/terminal and navigate to the cinchy.terraform\aws\eks_cluster\CLUSTER_NAME file.

4. Run the below command to remove metrics server.

terraform apply

1. Revert the enable_metrics_server key value from step 1 to true.

2. Run the below command within the same shell/terminal as step 3 to deploy metrics server with

 terraform apply

Overview

The pages under this section can be used to help guide you through an upgrade of your Cinchy instance on IIS.

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to v5.4 on IIS.

Prerequisites

1. Take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Upgrade Process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. If you are upgrading to 5.4+ on an SQL Server Database, you will need to make a change to your connectionString in both your SSO and Cinchy appsettings.json. Adding will allow you to bypass the certificate chain during validation.

   Ex:

3. Execute the following command:

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

1. Open your Cinchy URL in your browser.

7. Ensure you can log in.

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to v5.2 on IIS.

Prerequisites

1. Take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Upgrade process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. Execute the following command:

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.

Upgrading on IIS

This process can be run when upgrading from a Cinchy version that's not v5.0+.

Prerequisites

1. to take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Upgrade process

1. Swap out the following configs with your current instance configs:

   1. Cinchy/web.config

   2. CinchySSO/appsettings.json

   3. Log4net.config

   4. Web.config

2. Execute the following command:

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

1. Start Cinchy in your browser.

Upgrading on IIS (v4 to v5+)

Prerequisites

1. to take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Update the CinchySSO appsettings.json

1. Open the C:\CinchySSO\appsettings.json file in a text editor and update the values below.

AppSettings

1. Under AppSettings section, update the values outlined in the table.

2. Wherever you see <base url> in the value, replace this with the actual protocol (HTTP or HTTPS) and the domain name (or IP address) you plan to use.

Ex:. if you're using HTTPS with the domain app.cinchy.co, then <base url> should be replaced with https://app.cinchy.co

SSO values

The values below are only required for SSO, otherwise leave them as blank.

Connection String

In order for the application to connect to the database, the "SqlServer" value needs to be set.

1. Find and update the value under the "ConnectionStrings" section:

External identity claim section

Under the "ExternalIdentityClaimSection" section you'll see the following values.

These values are used for SAML SSO. If you aren't using SSO, keep these values as blank

Serilog

1. "Serilog" has a property that allows you to configure where it logs to. In the below code, update the following:

   1. "Name" must be set to "File" so it writes to a physical file on the disk.

   2. "Path" must be set to the file path to where you want it to log.

Update the Cinchy appsettings.json

1. Navigate to C:\Cinchy

2. Delete the appsettings.Development.json

3. Navigate to the appsettings.json file and update the following properties:

1.3.1 AppSettings

Connection String

In order for the application to connect to the database, the "SqlServer" value needs to be set.

1. Find and update the value under the "ConnectionStrings" section:

1.3.3 Serilog

1. "Serilog" has a property that allows you to configure where it logs to. In the below code, update the following:

   • "Name" must be set to "File" so it writes to a physical file on the disk.

   • "Path" must be set to the file path to where you want it to log.

You can also use an alternative setting if you want to have rolling log files with retention settings by adding in the following parameters:

• Your full "Serilog" property, if you choose to use the alternative settings, would look like this, inputting your own variables as required:

Configure the IIS Manager and running your upgrade

1. Open your Internet Information Services (IIS) Manager.

2. Navigate to Connections > Sites.

3. Right click on the Cinchy site and select Manage Application > Advanced Settings.

4. Change the Cinchy folder path to that of the version you're deploying.

5. Right click on the CinchySSO site and select Manage Application > Advanced Settings

6. Ensure that both Applications Pools for Cinchy and CinchySSO have their .NET CLR Versions set to No Managed Code.

7. Change the Cinchy SSO folder path to that of the version you're deploying.

8. Execute the following command:

1. Execute the following command:

1. Open your Cinchy URL in your browser.

1. Ensure that you can log in.

Upgrading on IIS

The following process can be run when upgrading from v5.0 to v5.1 on IIS.

Prerequisites

1. Take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Upgrade process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. Execute the following command:

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to Cinchy v5.3 on IIS.

Prerequisites

1. Take a backup of your database.

2. Extract thefor the version you wish to upgrade to.

Upgrade Process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. Execute the following command:

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.

Upgrading to v5 on Kubernetes

When you upgrade to Cinchy v5 on Kubernetes you are creating a separate instance. You will need to plan the migration of your database, and then follow the deployment installation guide.

Upgrading to v5 on IIS

To upgrade from v4+ to v5+ on IIS, review the documentation here.

Compatibility issues between databases or versions

If you choose to upgrade to Cinchy v5 on PostgreSQL, please review the following features that aren't currently available in that deployment.

These may be implemented in future versions of the platform.

• No Partitioning

• No Geospatial features

• No Column Indexing

• No Full Text Indexing

• No Column Level Encryption

You may also wish to review the list of CQL functions currently unsupported in PGSQL. Please note that this is a living document and that the development team is actively working on function translations between databases. Make sure to check back for the most up to date information.

• CQL Functions

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to v5.6 on IIS.

"SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

Prerequisites

1. Take a backup of your database.

2. Extract the new buildfor the version you wish to upgrade to.

3. Download .NET 6.0

Upgrade process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. If you are upgrading to 5.6 on an SQL Server Database and didn't do so in any previous updates, you will need to make a change to your connectionString in both your SSO and Cinchy appsettings.json. Adding TrustServerCertificate=True will allow you to bypass the certificate chain during validation.

   Ex:

   Copy

   "SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

3. When upgrading to 5.6, you are required to make the following changes to various appsettings.json files:

    "AppSettings": {
      "CinchyUri": "http://localhost",
      "CertificatePath": "C:\\inetpub\\wwwroot\\cinchysso\\cinchyidentitysrv.pfx",
      "CertificatePassword": "",
      "SAMLClientEntityId": "",
      "SAMLIDPEntityId": "",
      "SAMLMetadataXmlPath": "",
      "SAMLSSOServiceURL": "",
      "SAMLEncryptedCertificatePath": "",
      "SAMLEncryptedCertificatePassword": "",
      "SAMLSignCertificatePath": "",
      "SAMLSignCertificatePassword": "",
      "HstsMaxAge": 2592000,
      "HstsIncludeSubDomains": false,
      "HstsPreload": false,
      "SAMLSignCertificateMinAlgorithm": "",
      "SAMLSignCertificateSigningBehaviour": "",
      "AcsURLModule": "",
      "StsPublicOriginUri": "",
      // Add in the below "StsPrivateOriginUri".
      //This should be the private base URL used by the .well-known discovery.
      // If left blank will match the request URL. /cinchysso
      "StsPrivateOriginUri": "",
      "MaxRequestHeadersTotalSize": 65536,
      "MaxRequestBufferSize": 65536,
      "MaxRequestBodySize": -1,
      "MachineKeyXml": "",
      "DpApiKeyRingPath": "",
      "TlsVersion": "",
      "CinchyAccessTokenLifetime": "7.00:00:00",
      "DataChangeCallbackTimeout": 7,
      "RefreshCacheTimeInMin": 10,
      "DefaultExpirationCacheTimeInMin": 360,
      "DBType": "PostgreSQL"

    "AppSettings": {
    // Add the below "StsPrivateAuthorityUri" value.
    // This should match your private Cinchy SSO URL.
      "StsPrivateAuthorityUri": "",
   // Add the below "StsPublicAuthorityUri" value.
   // This should match your public Cinchy SSO URL.
      "StsPublicAuthorityUri": "",
   // Add the below "CinchyPrivateUri" value.
   // This should match your private Cinchy URL.
      "CinchyPrivateUri": "",
   // Add the below "CinchyPublicUri" value.
   // This should match your public Cinchy URL.
      "CinchyPublicUri": "",
      "AllowLogFileDownload": false,
      "LogDirectoryPath": "C:\\CinchyLogs\\CinchyWeb",
      "SSOLogPath": "C:\\CinchyLogs\\CinchySSO\\log.json",
      "UseHttps": true,
      "HstsMaxAge": 2592000,
      "HstsIncludeSubDomains": false,
      "HstsPreload": false,
      "TlsVersion": "",
      "RouteDebuggerEnabled": false,
      "RefreshCacheTimeInMin": 10,
      "DefaultExpirationCacheTimeInMin": 360,
      "DBType": "PostgreSQL",
      "StorageType": "Local", // Local | S3 | AzureBlobStorage
      "MaxRequestBodySize": 1073741824 // 1gb
    },

{
  "CinchyClientSettings": {
    "Url": "",      // Cinchy Url
    "Username": "", // For Cinchy v4 only, remove otherwise
    "Password": ""  // For Cinchy v5, this should be the password for the user connections@cinchy.com. For v4 this will be the desired user's password.
  },

  "CinchyClientSettings": {
    "Url": "", // Cinchy Url
    "Username": "", // For Cinchy v4, remove otherwise
    "Password": "" // For Cinchy v5, this should be the password for the user eventlistener@cinchy.com. For v4 this will be the desired user's password.
  }

1. Execute the following command:

iisreset -stop

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

iisreset -start

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to v5.7 on IIS.

"SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

Prerequisites

1. Take a backup of your database.

2. Extract the new buildfor the version you wish to upgrade to.

3. Download .NET 6.0

Upgrade process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. If you are upgrading to 5.7 on an SQL Server Database and didn't do so in any previous updates, you will need to make a change to your connectionString in both your SSO and Cinchy appsettings.json. Adding TrustServerCertificate=True will allow you to bypass the certificate chain during validation.

   Ex:

   Copy

   "SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

3. When upgrading to 5.7, you are required to make the following changes to various appsettings.json files:

    "AppSettings": {
      "CinchyUri": "http://localhost",
      "CertificatePath": "C:\\inetpub\\wwwroot\\cinchysso\\cinchyidentitysrv.pfx",
      "CertificatePassword": "",
      "SAMLClientEntityId": "",
      "SAMLIDPEntityId": "",
      "SAMLMetadataXmlPath": "",
      "SAMLSSOServiceURL": "",
      "SAMLEncryptedCertificatePath": "",
      "SAMLEncryptedCertificatePassword": "",
      "SAMLSignCertificatePath": "",
      "SAMLSignCertificatePassword": "",
      "HstsMaxAge": 2592000,
      "HstsIncludeSubDomains": false,
      "HstsPreload": false,
      "SAMLSignCertificateMinAlgorithm": "",
      "SAMLSignCertificateSigningBehaviour": "",
      "AcsURLModule": "",
      "StsPublicOriginUri": "",
      // Add in the below "StsPrivateOriginUri".
      //This should be the private base URL used by the .well-known discovery.
      // If left blank will match the request URL. /cinchysso
      "StsPrivateOriginUri": "",
      "MaxRequestHeadersTotalSize": 65536,
      "MaxRequestBufferSize": 65536,
      "MaxRequestBodySize": -1,
      "MachineKeyXml": "",
      "DpApiKeyRingPath": "",
      "TlsVersion": "",
      "CinchyAccessTokenLifetime": "7.00:00:00",
      "DataChangeCallbackTimeout": 7,
      "RefreshCacheTimeInMin": 10,
      "DefaultExpirationCacheTimeInMin": 360,
      "DBType": "PostgreSQL"

    "AppSettings": {
    // Add the below "StsPrivateAuthorityUri" value.
    // This should match your private Cinchy SSO URL.
      "StsPrivateAuthorityUri": "",
   // Add the below "StsPublicAuthorityUri" value.
   // This should match your public Cinchy SSO URL.
      "StsPublicAuthorityUri": "",
   // Add the below "CinchyPrivateUri" value.
   // This should match your private Cinchy URL.
      "CinchyPrivateUri": "",
   // Add the below "CinchyPublicUri" value.
   // This should match your public Cinchy URL.
      "CinchyPublicUri": "",
      "AllowLogFileDownload": false,
      "LogDirectoryPath": "C:\\CinchyLogs\\CinchyWeb",
      "SSOLogPath": "C:\\CinchyLogs\\CinchySSO\\log.json",
      "UseHttps": true,
      "HstsMaxAge": 2592000,
      "HstsIncludeSubDomains": false,
      "HstsPreload": false,
      "TlsVersion": "",
      "RouteDebuggerEnabled": false,
      "RefreshCacheTimeInMin": 10,
      "DefaultExpirationCacheTimeInMin": 360,
      "DBType": "PostgreSQL",
      "StorageType": "Local", // Local | S3 | AzureBlobStorage
      "MaxRequestBodySize": 1073741824 // 1gb
    },

{
  "CinchyClientSettings": {
    "Url": "",      // Cinchy Url
    "Username": "", // For Cinchy v4 only, remove otherwise
    "Password": ""  // For Cinchy v5, this should be the password for the user connections@cinchy.com. For v4 this will be the desired user's password.
  },

  "CinchyClientSettings": {
    "Url": "", // Cinchy Url
    "Username": "", // For Cinchy v4, remove otherwise
    "Password": "" // For Cinchy v5, this should be the password for the user eventlistener@cinchy.com. For v4 this will be the desired user's password.
  }

1. Execute the following command:

iisreset -stop

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

iisreset -start

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.

Upgrading on IIS

The following process can be run when upgrading any v5.x instance to v5.5 on IIS.

"SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

Prerequisites

1. Take a backup of your database.

2. Extract the new build for the version you wish to upgrade to.

3. Download .NET 6.0

Upgrade process

1. Merge the following configs with your current instance configs:

   • Cinchy/web.config

   • Cinchy/appsettings.json

   • CinchySSO/appsettings.json

   • CinchySSO/web.config

2. If you are upgrading to 5.5 on an SQL Server Database and didn't do so in any previous updates, you will need to make a change to your connectionString in both your SSO and Cinchy appsettings.json. Adding TrustServerCertificate=True will allow you to bypass the certificate chain during validation.

   Ex:

   Copy

   "SqlServer" : "Server=MyServer;Database=Cinchy;User ID=cinchy;Password=password;Trusted_Connection=False;Connection Timeout=30;Min Pool Size=10;TrustServerCertificate=True"

3. Execute the following command:

iisreset -stop 

1. Replace the Cinchy and CinchySSO folders with the new build and your merged configs.

2. Execute the following command:

iisreset -start 

1. Open your Cinchy URL in your browser.

2. Ensure you can log in.