Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29615

Fix connections problems in running Gen3 jointcal on HSC RC2



    • Story
    • Status: Done
    • Resolution: Done
    • None
    • jointcal
    • 2
    • Data Release Production
    • No


      I've brought some of these up on Slack in response to madamow's problem reports, but I figured it'd be good to capture them here with my proposed fixes:

      • The fact that jointcal marks its reference catalogs as regular Inputs instead of PrerequisiteInputs currently makes it impossible to run jointcal in the same pipeline as later tasks (some of which declare the same reference catalog as a PrerequisiteInput). That's not really jointcal's fault, but I also think it's good practice to use PrerequisiteInput for reference catalogs anyway, because they're never something we'd produce in the same pipeline as something that consumes them, and when that's the case then PrerequisiteInput has some advantages, like better diagnostics when no datasets can be found.
      • The inputCamera connection definitely needs to be turned into a PrerequisiteInput (validity-range calibrations only work as prerequisites), and it also needs a special lookup function like this one in FGCM: https://github.com/lsst/fgcmcal/blob/d78879ea4eb43453c9955831f4d697073a1b463f/python/lsst/fgcmcal/fgcmMakeLut.py#L62. That works around the fact that jointcal wants a camera without saying what period in time it should be valid for. I'm kind of surprised this hasn't caused problems in CI already, and I'd guess it's because the test repo is set up with simplifer-than-real calibration content.

      I think we can work around this in the w14 processing, but we'll see how it goes.


        Issue Links


            No work has yet been logged on this issue.


              jbosch Jim Bosch
              jbosch Jim Bosch
              John Parejko
              Eli Rykoff, Jim Bosch, John Parejko, Monika Adamow (Inactive)
              0 Vote for this issue
              4 Start watching this issue




                  No builds found.