Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30351

Make sure brighter-fatter correction is working on DC2 data for DP0.2

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
    • Story Points:
      6
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      I'm not sure if we lack kernels, or code to generate kernels, or just the right config options.  But we need to fix enough of that (ingesting Gen2 kernels if necessary/possible) to make sure we turn on brighter-fatter correction in DP0.2.

        Attachments

          Issue Links

            Activity

            Hide
            czw Christopher Waters added a comment - - edited

            "Old" brighter-fatter kernels exist in  /datasets/DC2/repoRun2.2i/CALIB/bfkernels/​.  Unfortunately, these are in an intermediate format that as far as I can tell, were only ever used in this one place.  Reading the kernels is no longer possible with the current stack, as the pickle storage for the kernel attempts to map them to the current style.  Hacking with ip_isr and cp_pipe at w.2020.15 (a date I'm reasonably certain had the old code) allows me to access the kernel, and can demonstrate that two randomly selected kernels have identical values.  This indicates that there is only one set of kernel values, so constructing new kernels should not be difficult.  I've attached the kernel value to this ticket.

            Show
            czw Christopher Waters added a comment - - edited "Old" brighter-fatter kernels exist in  /datasets/DC2/repoRun2.2i/CALIB/bfkernels/ ​.  Unfortunately, these are in an intermediate format that as far as I can tell, were only ever used in this one place.  Reading the kernels is no longer possible with the current stack, as the pickle storage for the kernel attempts to map them to the current style.  Hacking with ip_isr  and cp_pipe  at w.2020.15  (a date I'm reasonably certain had the old code) allows me to access the kernel, and can demonstrate that two randomly selected kernels have identical values.  This indicates that there is only one set of kernel values, so constructing new kernels should not be difficult.  I've attached the kernel value to this ticket.
            Hide
            lauren Lauren MacArthur added a comment -

            Plot revealing the lack of correction from the 2.2i/runs/test-med-1/w_2021_16/DM-29899: gen3 run:

            Show
            lauren Lauren MacArthur added a comment - Plot revealing the lack of correction from the  2.2i/runs/test-med-1/w_2021_16/ DM-29899 : gen3 run:
            Hide
            czw Christopher Waters added a comment -

            Based on the comments on DM-26092, this should be working now.

            Show
            czw Christopher Waters added a comment - Based on the comments on DM-26092 , this should be working now.
            Hide
            czw Christopher Waters added a comment -

            I've split this review into two parts, with `ip_isr` for Andrés Alejandro Plazas Malagón, and the remaining PRs for Dan Taranu.  

            Show
            czw Christopher Waters added a comment - I've split this review into two parts, with `ip_isr` for Andrés Alejandro Plazas Malagón , and the remaining PRs for Dan Taranu .  
            Hide
            dtaranu Dan Taranu added a comment -

            Looks good. I tested locally with ci_imsim and had one jenkins build pass (the macos timeout and failures in other runs are not related to this ticket).

            Show
            dtaranu Dan Taranu added a comment - Looks good. I tested locally with ci_imsim and had one jenkins build pass (the macos timeout and failures in other runs are not related to this ticket).
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Looks good. My main question was if the changes here were only related to the logic for AMP and DETECTOR levels and how that is related to fixing the BFKs for DC2. Chris confirmed that, in fact, these are only AMP/DETECTOR differences to match what gen3 is expecting (a single kernel per detector).

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Looks good. My main question was if the changes here were only related to the logic for AMP and DETECTOR levels and how that is related to fixing the BFKs for DC2. Chris confirmed that, in fact, these are only AMP/DETECTOR differences to match what gen3 is expecting (a single kernel per detector).

              People

              Assignee:
              czw Christopher Waters
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Andrés Alejandro Plazas Malagón, Dan Taranu
              Watchers:
              Andrés Alejandro Plazas Malagón, Christopher Waters, Dan Taranu, Jim Bosch, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.