Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19470

Create jointcal PipelineTask (version 0: tests)

    XMLWordPrintable

Details

    • 12
    • AP F20-6 (November), AP S21-1 (December)
    • Alert Production
    • No

    Description

      We need to PipelineTask-ify jointcal so that we can use it in the gen3 world. jbosch suggested this would be a good pair programming project for a day or so.

      NOTE: this ticket has been modified to hold the new initial testing code and the modifications to jointcal that will likely remain, given the conversation on slack today. It turns out that the approach jbosch had initially suggested will not be viable without switching to using the VisitSummary catalogs. That's quite a bit of work on its own, so I decided to break it off into a separate ticket.

      Attachments

        Issue Links

          Activity

            Parejkoj John Parejko added a comment -

            An idea while Jim and I were pair coding on this this morning: a DataClass to hold the various things that jointcal is currently storing as class state. This will simplify jointcal's interface, remove statefullness from the class, and improve some gen3 behavior.

            Parejkoj John Parejko added a comment - An idea while Jim and I were pair coding on this this morning: a DataClass to hold the various things that jointcal is currently storing as class state. This will simplify jointcal's interface, remove statefullness from the class, and improve some gen3 behavior.

            Per our (ebellm, Parejkoj, swinbank) discussion of 2019-11-04, this is blocked by functionality in Gen 3 middleware, but it's not clear exactly what that functionality is or if there's another ticket which we can use as a blocker. Parejkoj will talk to jbosch and try to figure that out.

            swinbank John Swinbank added a comment - Per our ( ebellm , Parejkoj , swinbank ) discussion of 2019-11-04, this is blocked by functionality in Gen 3 middleware, but it's not clear exactly what that functionality is or if there's another ticket which we can use as a blocker. Parejkoj will talk to jbosch and try to figure that out.
            nlust Nate Lust added a comment -

            I think you are looking for DM-21904

            nlust Nate Lust added a comment - I think you are looking for DM-21904
            nlust Nate Lust added a comment -

            Or possibly DM-17169 depending on how you want to do the blocking.

            nlust Nate Lust added a comment - Or possibly DM-17169 depending on how you want to do the blocking.
            jbosch Jim Bosch added a comment -

            I've updated the link to DM-21904 to reflect the fact that is not a blocker; DM-17169 was enough to unblock this work.

            jbosch Jim Bosch added a comment - I've updated the link to DM-21904 to reflect the fact that is not a blocker; DM-17169 was enough to unblock this work.
            Parejkoj John Parejko added a comment -

            For the record: testing gen3 jointcal will be limited to HSC for the time being, as getting the cfht gen2 data converted is blocked on further gen3 obs_cfht work (e.g. DM-21865) and I cannot reliably convert a gen2 decam dataset yet.

            Parejkoj John Parejko added a comment - For the record: testing gen3 jointcal will be limited to HSC for the time being, as getting the cfht gen2 data converted is blocked on further gen3 obs_cfht work (e.g. DM-21865 ) and I cannot reliably convert a gen2 decam dataset yet.

            erykoff: are you still able to review these changes to jointcal and testdata_jointcal? It's about 200 lines in jointcal, and a handful of lines in the testdata (not counting the new gen3 exports yaml).

            The jointcal changes here do not implement a working gen3 pipetask, but they are the groundwork that I did along the way that I'd like to get looked while I work on the newer approach in DM-27869.

            Parejkoj John Parejko added a comment - erykoff : are you still able to review these changes to jointcal and testdata_jointcal? It's about 200 lines in jointcal, and a handful of lines in the testdata (not counting the new gen3 exports yaml). The jointcal changes here do not implement a working gen3 pipetask, but they are the groundwork that I did along the way that I'd like to get looked while I work on the newer approach in DM-27869 .
            Parejkoj John Parejko added a comment - Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33216/pipeline

            People

              Parejkoj John Parejko
              Parejkoj John Parejko
              Eli Rykoff
              Eli Rykoff, Ian Sullivan, Jim Bosch, John Parejko, John Swinbank, Nate Lust
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.