Course summary of work on this ticket:
- https://github.com/lsst-sqre/terraform-gke-std was split out of lsst-sqre/deploy-publish-release to provide a basic reusable gke configuration
- lsst-sqre/deploy-eups-pkgroot was renamed to lsst-sqre/deploy-publish-release and refactored to be "pure" terraform rather than a mix of tf and k8s yaml config files – this eliminated glue scripts to support the tf -> k8s -> tf workflow AND the need to manually export secrets to jenkins, which had to be done for every jenkins job test end
- lsst-sqre/sandbox-jenkins-demo had a general tf update/cleanup and now imports s3 "remote state" from lsst-sqre/deploy-publish-release to configure secrets/credentials
- A error handling bug in lsst-sqre/sqre-codekit was discovered and fixed
- lsst-sqre/deploy-eups-redirect was renamed to lsst-sqre/terraform-pkgroot-redirect and refactored to be a tf module used by lsst-sqre/deploy-publish-release
- The hiera hierarchy in lsst-sqre/sandbox-jenkins-demo was refactored to support in-tree configuration for multiple deployment envs largely eliminating the need for a working branch per jenkins test env with configuration changes
- linting of shell/yaml/markdown/docker/terraform/make was added, as appropriate, to most repositories that were touched in the course of this ticket
- Hardcoded values and assumptions in lsst-sqre/jenkins-dm-jobs about the production environment (URLs, github repo names, etc.) were identified and either moved into yaml configuration files or pipeline logic was changed to pass more explicit information to triggered jobs.
- There were multiple "cleanup" refactors of jenkins jobs including consistent build parameter handling, de-duplication of code, logic rewrites to hopefully improve the clarity of the code, a shift towards using groovy named parameters for methods with an arity > ~2.
- A new release/official-release job was created to automate / allow testing of the "official" release workflow
- The "demo" handling in lsst-sqre/ci-scripts was refactored to try to find a git ref matching the lsstsw "BRANCH" list – this was needed as changed merged to sci-pipe code right after v16_0_rc1 was cut required breaking changes to the "demo"
- MANIFEST_ID or "manifest id" is now consistently used to refer to a manifest in a "versiondb" in place of BUILD=, BUILD_ID, bNNNN, bxxxx, etc.
An [almost] end-to-end release was demonstrated in a test env. The exception were validate_drp data was not shipped to a "squash" env nor was the experimental documenteer docs built pushed to an "lsst-the-docs" instance.
All changes have now been merged to master. I'm going to leave this ticket open in self review for a day or two in case of fallout.