Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26675

Changes to validate_drp to support treecorr > 4

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: conda, validate_drp
    • Labels:
      None
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      treecorr > 4 has some small changes which break validate_drp.

       

      Specifically, varxi in ggcorrelation has been removed.

      There is varxim and varxip, described here:

      https://github.com/rmjarvis/TreeCorr/blob/master/treecorr/ggcorrelation.py#L105-L106

       

      The use is in line 187 python/lsst/validate/drp/calcsrd/tex.py.
       
      Treecorr <4 also does not have varxim or varxip, so this ticket will likely require a merge of a new environment with treecorr > 4 as well.

       

      It appears this is possibly a blocker for moving to python 3.8, due to conda-forge wanting treecorr > 4 for python 3.8 (I am not sure why - I have not checked the recipes at this point).

       

       

        Attachments

          Issue Links

            Activity

            Hide
            jcarlin Jeffrey Carlin added a comment -

            Brian Van Klaveren - in order to test this, will I need an environment with python 3.8 and treecorr > 4? I'm not exactly sure how to do that within the science pipelines conda env.

            Show
            jcarlin Jeffrey Carlin added a comment - Brian Van Klaveren - in order to test this, will I need an environment with python 3.8 and treecorr > 4? I'm not exactly sure how to do that within the science pipelines conda env.
            Hide
            bvan Brian Van Klaveren added a comment -

            There's a few ways to go about this.

            1. If you use lsstsw `rebuild` , I can give you an environment reference, then you would run `bin/deploy -r [reference]` and then `rebuild validate_drp`
            2. If you use eups workflow ( `eups distrib install ...` followed by several `setup` commands), you would re-run newinstall also with a new environment reference, and then you would install the stack on that environment
            3. If you don't mind doing some conda commands, we can clone an environment of a working stack, do a forced upgrade of only treecorr, and then you can do some development in that way.

            I can help you out with any of these.

            Show
            bvan Brian Van Klaveren added a comment - There's a few ways to go about this. 1. If you use lsstsw `rebuild` , I can give you an environment reference, then you would run `bin/deploy -r [reference] ` and then `rebuild validate_drp` 2. If you use eups workflow ( `eups distrib install ...` followed by several `setup` commands), you would re-run newinstall also with a new environment reference, and then you would install the stack on that environment 3. If you don't mind doing some conda commands, we can clone an environment of a working stack, do a forced upgrade of only treecorr, and then you can do some development in that way. I can help you out with any of these.
            Hide
            bvan Brian Van Klaveren added a comment -

            (also - we only need to test out for treecorr 4, we don't need to test for python 3.8 at this time)

            Show
            bvan Brian Van Klaveren added a comment - (also - we only need to test out for treecorr 4, we don't need to test for python 3.8 at this time)
            Hide
            jcarlin Jeffrey Carlin added a comment -

            A cloned conda env on lsst-dev would be most comfortable to me. (Option 3)

            Show
            jcarlin Jeffrey Carlin added a comment - A cloned conda env on lsst-dev would be most comfortable to me. (Option 3)
            Hide
            bvan Brian Van Klaveren added a comment -

            Hi Jeff - I created a cloned environment but after some investigation myself I think I actually just fixed the issue. Would you mind double checking?

            Show
            bvan Brian Van Klaveren added a comment - Hi Jeff - I created a cloned environment but after some investigation myself I think I actually just fixed the issue. Would you mind double checking?
            Hide
            jcarlin Jeffrey Carlin added a comment -

            Confirmed that the new version runs fine, and yields the same result as the most recent previous run of `validate_drp`.

            Show
            jcarlin Jeffrey Carlin added a comment - Confirmed that the new version runs fine, and yields the same result as the most recent previous run of `validate_drp`.

              People

              Assignee:
              bvan Brian Van Klaveren
              Reporter:
              bvan Brian Van Klaveren
              Reviewers:
              Jeffrey Carlin
              Watchers:
              Brian Van Klaveren, Jeffrey Carlin, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.