Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29615

Fix connections problems in running Gen3 jointcal on HSC RC2

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jointcal
    • Labels:
    • Story Points:
      2
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      I've brought some of these up on Slack in response to Monika Adamow's problem reports, but I figured it'd be good to capture them here with my proposed fixes:

      • The fact that jointcal marks its reference catalogs as regular Inputs instead of PrerequisiteInputs currently makes it impossible to run jointcal in the same pipeline as later tasks (some of which declare the same reference catalog as a PrerequisiteInput). That's not really jointcal's fault, but I also think it's good practice to use PrerequisiteInput for reference catalogs anyway, because they're never something we'd produce in the same pipeline as something that consumes them, and when that's the case then PrerequisiteInput has some advantages, like better diagnostics when no datasets can be found.
      • The inputCamera connection definitely needs to be turned into a PrerequisiteInput (validity-range calibrations only work as prerequisites), and it also needs a special lookup function like this one in FGCM: https://github.com/lsst/fgcmcal/blob/d78879ea4eb43453c9955831f4d697073a1b463f/python/lsst/fgcmcal/fgcmMakeLut.py#L62. That works around the fact that jointcal wants a camera without saying what period in time it should be valid for. I'm kind of surprised this hasn't caused problems in CI already, and I'd guess it's because the test repo is set up with simplifer-than-real calibration content.

      I think we can work around this in the w14 processing, but we'll see how it goes.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              John Parejko
              Watchers:
              Eli Rykoff, Jim Bosch, John Parejko, Monika Adamow
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.