Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-29055

Unbitrot Gen3 SkyCorrectionTask

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      2
    • Sprint:
      DRP S21a (Dec Jan)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      SkyCorrection was converted a while, back but didn't make it into ci_hsc_gen3 (perhaps because of missing sky frames) and needs to be updated.

        Attachments

          Issue Links

            Activity

            Hide
            yusra Yusra AlSayyad added a comment -

            Christopher Waters, Jim Bosch and Nate Lust paircoded this with me today.
            I added it to the obs_subaru pipeline just to see what ci_hsc_gen3 would do with it.
            https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33730/pipeline
            I'm actually not expecting that to work, and don't have my heart set on merging the obs_subaru changes.

            But I would like to merge the pipe_drivers PR tonight. Given that 3 people were looking over my shoulder, it's pretty much reviewed.

            Show
            yusra Yusra AlSayyad added a comment - Christopher Waters , Jim Bosch and Nate Lust paircoded this with me today. I added it to the obs_subaru pipeline just to see what ci_hsc_gen3 would do with it. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/33730/pipeline I'm actually not expecting that to work, and don't have my heart set on merging the obs_subaru changes. But I would like to merge the pipe_drivers PR tonight. Given that 3 people were looking over my shoulder, it's pretty much reviewed.
            Hide
            yusra Yusra AlSayyad added a comment -

            I'm also looking into why the gen3 skyCorr's seem to be missing spatial structure.

            Show
            yusra Yusra AlSayyad added a comment - I'm also looking into why the gen3 skyCorr's seem to be missing spatial structure.
            Hide
            jbosch Jim Bosch added a comment -

            What's there now looks good; just some doc fixes and a theory on what's broken. Happy to take another look if you have more changes.

            Show
            jbosch Jim Bosch added a comment - What's there now looks good; just some doc fixes and a theory on what's broken. Happy to take another look if you have more changes.
            Hide
            yusra Yusra AlSayyad added a comment -

            Thanks Jim. Fixed those doc issues, reordered the per-detector inputRefs and outputRefs to match the inputRef.calexpArray detector order.

            Kicked off a another Jenkins with that update and with obs_subaru set back to master in case ci_hsc_gen3 is unhappy with my initial Jenkins run.

            Show
            yusra Yusra AlSayyad added a comment - Thanks Jim. Fixed those doc issues, reordered the per-detector inputRefs and outputRefs to match the inputRef.calexpArray detector order. Kicked off a another Jenkins with that update and with obs_subaru set back to master in case ci_hsc_gen3 is unhappy with my initial Jenkins run.
            Hide
            yusra Yusra AlSayyad added a comment -

            Tonight went better than I was expecting:

            • ci_hsc_gen3 is happy. The logs look OK. <1% more sources detected on the coadds. While I'm not 100% sure that skyCorr can make a better background with a partial focal plane, remember that the purpose of ci_hsc_gen3 is to keep our pipeline functional, not to demonstrate performance. We have validation_data_hsc for that. So with that in mind, I'm merging the obs_subaru PR too.
            • It's giving the same results on some selected visit as gen2. Yippy.
            Show
            yusra Yusra AlSayyad added a comment - Tonight went better than I was expecting: ci_hsc_gen3 is happy. The logs look OK. <1% more sources detected on the coadds. While I'm not 100% sure that skyCorr can make a better background with a partial focal plane, remember that the purpose of ci_hsc_gen3 is to keep our pipeline functional, not to demonstrate performance. We have validation_data_hsc for that. So with that in mind, I'm merging the obs_subaru PR too. It's giving the same results on some selected visit as gen2. Yippy.

              People

              Assignee:
              yusra Yusra AlSayyad
              Reporter:
              yusra Yusra AlSayyad
              Reviewers:
              Jim Bosch
              Watchers:
              Jim Bosch, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.