Angelo Fausti added a comment - 11/Jul/22 1:46 PM Step 0. Sync Sasquatch at TTS Step 1. Redo the data migration to Sasquatch (see the procedure in DM-35345 ) Restart Chronograf and Kapacitor. Update InfluxDB and Kapacitor configuration in Chronograf (the restored configuration points to the backed-up services!) InfluxDB connection URL: http://sasquatch-influxdb.sasquatch:8086 Kapacitor connection URL: http://sasquatch-kapacitor.sasquatch:9092 Check if everything is restored: databases, retention policies, users, permissions, dashboards, alerts, slack weebook configuration, etc Migrate Chronograf news feeds to the new repository. Step 2. Review secrets at TTS Vault Add ts-salkafka-password key to the sasquatch secret at TTS. This is used by Strimzi to create the Kafka user. The same key and password must be added to a new secret called ts-salkafka on TTS Vault secret/k8s_operator/tucson-teststand.lsst.codes/ts/software/ts-salkafka This is used by the kafka-producers . Sync sasquatch secret at TTS, make sure ts-salkafka Kafka user is created, etc Step 3. Sync kafka-producers at TTS Stop Kafka producers at TTS. Sync kafka-producers to version 0.11.0. The kafka-producers-ts-salkafka secret should be created in the kafka-producers namespace. Producers are now configured with the new URLs for the Sasquatch Kafka brokers and Schema Registry. Verify that producers are writing to the Sasquatch Kafka and InfluxDB instances This can be done at: Sasquatch Kafdrop URL: https://tucson-teststand.lsst.codes/kafdrop Sasquatch Chronograf URL: https://tucson-teststand.lsst.codes/chronograf If not, assess the problem before continuing. Step 4. Sasquatch Slack notifications Alert rules were migrated to Sasquatch Kapacitor, along with the Slack webhook configuration. Slack notifications will continue going to the com-efd-status channel. Disabled notifications from the old Chronograf instance. Step 5. Implement Chronograf redirect Users going to the old Chronograf instance will now be redirected to https://tucson-teststand.lsst.codes/chronograf The redirect is implemented so that it rewrites the URL in the browser. NOTE: After this step access to the old Chronograf instance is disabled. Note the redirect, however, requires the old Ingress to be present. Also, note that the wild card TLS certificates for https://chronograf-tucson-teststand-efd.lsst.codes expire on Aug 6. Can we disable the redirect before Aug 6? Otherwise, we’ll need to renew those certificates again. Chronograf is now authenticated via Gafaelfawr using the OIDC provider. Users that belong to the “RSP access” team on the rubin-summit GH org can access the RSP at TTS and thus Chronograf. This was tested with the chronograf-viewer user. Step 6. Implement EFD client redirect Access via the EFD client is done through the segwarides service and is transparent to the users. The efdreader user in InfluxDB is used to authenticate the EFD client. The credentials for this user need to be updated in the segwarides secret along with the new InfluxDB as Schema registry URLs for Sasquatch. Update segwarides secret. Sync Segwarides on the Roundtable cluster to use the new secret. Verify access to the Sasquatch InfluxDB instance using the EFD client at TTS. NOTE: After this step access to the old InfluxDB instance is disabled. Step 7 . Update SQR-034 with new URLs for the services deployed at TTS Step 8 . Announce roll out completion and notify users about the Chronograf redirection reminding them to start using the new URL.

Angelo Fausti added a comment - 11/Jul/22 11:52 PM Roll out of Sasquatch to TTS is complete. Some details will be addressed in other tickets before the roll out to the Summit.