Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19470

Create jointcal PipelineTask (version 0: tests)

    XMLWordPrintable

    Details

    • Story Points:
      12
    • Sprint:
      AP F20-6 (November), AP S21-1 (December)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      We need to PipelineTask-ify jointcal so that we can use it in the gen3 world. Jim Bosch suggested this would be a good pair programming project for a day or so.

      NOTE: this ticket has been modified to hold the new initial testing code and the modifications to jointcal that will likely remain, given the conversation on slack today. It turns out that the approach Jim Bosch had initially suggested will not be viable without switching to using the VisitSummary catalogs. That's quite a bit of work on its own, so I decided to break it off into a separate ticket.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            An idea while Jim and I were pair coding on this this morning: a DataClass to hold the various things that jointcal is currently storing as class state. This will simplify jointcal's interface, remove statefullness from the class, and improve some gen3 behavior.

            Show
            Parejkoj John Parejko added a comment - An idea while Jim and I were pair coding on this this morning: a DataClass to hold the various things that jointcal is currently storing as class state. This will simplify jointcal's interface, remove statefullness from the class, and improve some gen3 behavior.
            Hide
            swinbank John Swinbank added a comment -

            Per our (Eric Bellm, John Parejko, John Swinbank) discussion of 2019-11-04, this is blocked by functionality in Gen 3 middleware, but it's not clear exactly what that functionality is or if there's another ticket which we can use as a blocker. John Parejko will talk to Jim Bosch and try to figure that out.

            Show
            swinbank John Swinbank added a comment - Per our ( Eric Bellm , John Parejko , John Swinbank ) discussion of 2019-11-04, this is blocked by functionality in Gen 3 middleware, but it's not clear exactly what that functionality is or if there's another ticket which we can use as a blocker. John Parejko will talk to Jim Bosch and try to figure that out.
            Hide
            nlust Nate Lust added a comment -

            I think you are looking for DM-21904

            Show
            nlust Nate Lust added a comment - I think you are looking for DM-21904
            Hide
            nlust Nate Lust added a comment -

            Or possibly DM-17169 depending on how you want to do the blocking.

            Show
            nlust Nate Lust added a comment - Or possibly DM-17169 depending on how you want to do the blocking.
            Hide
            jbosch Jim Bosch added a comment -

            I've updated the link to DM-21904 to reflect the fact that is not a blocker; DM-17169 was enough to unblock this work.

            Show
            jbosch Jim Bosch added a comment - I've updated the link to DM-21904 to reflect the fact that is not a blocker; DM-17169 was enough to unblock this work.
            Hide
            Parejkoj John Parejko added a comment -

            For the record: testing gen3 jointcal will be limited to HSC for the time being, as getting the cfht gen2 data converted is blocked on further gen3 obs_cfht work (e.g. DM-21865) and I cannot reliably convert a gen2 decam dataset yet.

            Show
            Parejkoj John Parejko added a comment - For the record: testing gen3 jointcal will be limited to HSC for the time being, as getting the cfht gen2 data converted is blocked on further gen3 obs_cfht work (e.g. DM-21865 ) and I cannot reliably convert a gen2 decam dataset yet.
            Hide
            Parejkoj John Parejko added a comment -

            Eli Rykoff: are you still able to review these changes to jointcal and testdata_jointcal? It's about 200 lines in jointcal, and a handful of lines in the testdata (not counting the new gen3 exports yaml).

            The jointcal changes here do not implement a working gen3 pipetask, but they are the groundwork that I did along the way that I'd like to get looked while I work on the newer approach in DM-27869.

            Show
            Parejkoj John Parejko added a comment - Eli Rykoff : are you still able to review these changes to jointcal and testdata_jointcal? It's about 200 lines in jointcal, and a handful of lines in the testdata (not counting the new gen3 exports yaml). The jointcal changes here do not implement a working gen3 pipetask, but they are the groundwork that I did along the way that I'd like to get looked while I work on the newer approach in DM-27869 .
            Show
            Parejkoj John Parejko added a comment - Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33216/pipeline

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Eli Rykoff
              Watchers:
              Eli Rykoff, Ian Sullivan, Jim Bosch, John Parejko, John Swinbank, Nate Lust
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.