Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-664

Update conda base environment

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Now that v19.0 has been released it is time I propose that we do our annual refresh of the conda base environment. This means:

      • python 3.7.6
      • astropy 4.0.x
      • numpy 1.17.4
      • scipy 1.3.1

      We will also upgrade the other packages such as pytest as defined in the scipipe_conda_env bleed definition.

      I have tested lsst_distrib with the new versions and everything works fine. In testing matplotlib 3.1 some problems were encountered on macOS and therefore I propose sticking with our current 3.0.x (see DM-22818).

      Following this update lsst_sims will need to update their copy of sncosmo to one that supports astropy4.

      I also propose we update the base conda package to the current version.

      I will also add the following new conda packages:

      • pytest-subtest (as approved previously in RFC-647)

      In addition I would like to add:

      • boto3 and moto (so that gen3 butler tests can run properly).

        Attachments

          Issue Links

          is triggered by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22125 Consider upgrading pyarrow to 0.15.1 due to memory leak in 0.13

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22793 Strange pyvo warning about votable version

          • Done
          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-22817 Update conda base environment

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23349 Build new shared stack with updated conda env

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23350 Update eups third party stubs for new conda env

          • Invalid
          relates to

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-647 Add pytest-subtests to our standard packages

          • Implemented

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-584 New Conda environment for Science Pipelines

          • Implemented
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Would  DM-22125 be relevant in this update?

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Would  DM-22125  be relevant in this update?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            In theory we could update pyarrow as part of this if we can sort out how.

            Show
            tjenness Tim Jenness added a comment - In theory we could update pyarrow as part of this if we can sort out how.
            Hide
            Permalink
            ljones Lynne Jones added a comment -

            Happy to update sncosmo (I think - will double check with the most interested parties). 

            We should likely also update healpy. 

            Is this as simple as updating the "upstream tarball" files in the repos (and checking that they work .. is there a jenkins setup I should use to test?)?

            Show
            ljones Lynne Jones added a comment - Happy to update sncosmo (I think - will double check with the most interested parties).  We should likely also update healpy.  Is this as simple as updating the "upstream tarball" files in the repos (and checking that they work .. is there a jenkins setup I should use to test?)?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            The sncosmo ticket is SIM-2621 and updating the tar ball is sufficient.

            Show
            tjenness Tim Jenness added a comment - The sncosmo ticket is SIM-2621 and updating the tar ball is sufficient.
            Hide
            Permalink
            frossie Frossie Economou added a comment -

            Definite yes from SQuaRE, on the basis of staying fresh and also Christine Banek needs astropy 4.x for DM-22793

            Show
            frossie Frossie Economou added a comment - Definite yes from SQuaRE, on the basis of staying fresh and also Christine Banek needs astropy 4.x for DM-22793
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Just remembered this but it would be very helpful if boto3 and moto could be added so that butler gen3 tests can run properly with S3.

            Show
            tjenness Tim Jenness added a comment - Just remembered this but it would be very helpful if boto3 and moto could be added so that butler gen3 tests can run properly with S3.
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            This will need final sign off by DMCCB.

            Show
            tjenness Tim Jenness added a comment - This will need final sign off by DMCCB.
            Hide
            Permalink
            swinbank John Swinbank added a comment - - edited

            DM-CCB agrees as of 2020-02-05.

            Show
            swinbank John Swinbank added a comment - - edited DM-CCB agrees as of 2020-02-05 .

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Colin Slater, Frossie Economou, Gabriele Comoretto [X] (Inactive), Hsin-Fang Chiang, John Swinbank, Kian-Tat Lim, Leanne Guy, Lynne Jones, Michelle Butler [X] (Inactive), Tim Jenness, Wil O'Mullane
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.