We are going to be unpinning a number of packages in rubin-env soon, so this could be a good time to upgrade our Python version. After discussion in the CCB (see below), I propose to update to not update to 3.9 but instead wait for 3.10 when it is usable by all of our dependencies, possibly well into 2022.
Tim Jenness and Nate Lust will fill in the features it provides.
TSSW will not be ready to move to a new Python version until January, so that is the earliest that a 3.10 update RFC could be submitted, approved, and implemented.
Since each Python transition is a substantial amount of work for TSSW and requires a rubin-env major version change, minimizing these is important. The DM-CCB felt that 3.10 was a significant enough improvement that it is desirable to update to it, so skipping 3.9 in order to get 3.10, even at the cost of additional time at 3.8, seems reasonable.
That in turn means waiting for 3.10 to stabilize in conda-forge, which seems likely to take months but hopefully not as long as waiting for 3.11 to be released, which wouldn't be before Oct 2022.
Note that with Python's sped-up annual release process, the final non-security bugfix release for 3.9 is expected to be in May 2022.
Accordingly, I have updated this RFC to be 3.10 again. This becomes only a policy RFC, with no implementation; once the dependencies are in place, a new RFC will propose the actual transition.
The plan is to create a dev rubin-env that can be built with 3.10 to allow further testing. See https://github.com/conda-forge/rubinenv-feedstock/pull/55 for the beginnings of this.
Should we close this RFC now since there's no work attached to it any more? We have demonstrated that py3.10 builds lsst-distrib.
It looks like there will never be an "official" scl for py 3.9 or 3.10 on el7. RedHat has discontinued scl development along with el7. py 3.9 is in appstreams for el8.