Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26545

Add spline linearizer.

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: cp_pipe, ip_isr
    • Labels:
      None
    • Story Points:
      8
    • Epic Link:
    • Team:
      Data Release Production

      Description

      Craig Lage has recently been obtaining results that indicate that we might also need to implement a spline fit option for the linearity.  This ticket will add this both to the measurement and application code.

        Attachments

          Issue Links

            Activity

            czw Christopher Waters created issue -
            czw Christopher Waters made changes -
            Field Original Value New Value
            Epic Link DM-25321 [ 435711 ]
            czw Christopher Waters made changes -
            Link This issue is blocked by DM-24703 [ DM-24703 ]
            Hide
            price Paul Price added a comment -

            We have spline code in PFS that you're welcome to port if that would be useful.

            Show
            price Paul Price added a comment - We have spline code in PFS that you're welcome to port if that would be useful.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Craig Lage reports that it's important to force the spline to match the linear curve at a low flux( less than 10000-20000 ADU), he says he has a way to do this if needed.

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Craig Lage reports that it's important to force the spline to match the linear curve at a low flux( less than 10000-20000 ADU), he says he has a way to do this if needed.
            czw Christopher Waters made changes -
            Link This issue is duplicated by DM-26582 [ DM-26582 ]
            czw Christopher Waters made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            czw Christopher Waters made changes -
            Assignee Christopher Waters [ cwaters ]
            yusra Yusra AlSayyad made changes -
            Epic Link DM-25321 [ 435711 ] DM-26795 [ 439745 ]
            jbosch Jim Bosch made changes -
            Link This issue blocks DM-23985 [ DM-23985 ]
            cslage Craig Lage made changes -
            Hide
            cslage Craig Lage added a comment -

            After correcting a sign error, I ran this on 9-raft data from last year.  It appears to correct most of the "bump" in the variance/flux vs flux curve.  See the attached plot. There is still a small residual bump around 70,000 ADU that I don't like.  I'll continue investigating this.

            Show
            cslage Craig Lage added a comment - After correcting a sign error, I ran this on 9-raft data from last year.  It appears to correct most of the "bump" in the variance/flux vs flux curve.  See the attached plot. There is still a small residual bump around 70,000 ADU that I don't like.  I'll continue investigating this.
            Show
            czw Christopher Waters added a comment - Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/32813/pipeline
            czw Christopher Waters made changes -
            Reviewers Merlin Fisher-Levine [ mfisherlevine ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            czw Christopher Waters made changes -
            Remote Link This issue links to "Page (Confluence)" [ 26134 ]
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Comments on PRs.

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Comments on PRs.
            mfisherlevine Merlin Fisher-Levine made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            czw Christopher Waters made changes -
            Story Points 6
            czw Christopher Waters made changes -
            Remote Link This issue links to "Page (Confluence)" [ 26199 ]
            czw Christopher Waters made changes -
            Attachment splineResid_f3.png [ 45873 ]
            Attachment splineFit_f2.png [ 45874 ]
            Attachment inputData_f1.png [ 45875 ]
            Hide
            czw Christopher Waters added a comment -

            From running a test with the debug code enabled:

             

            F1: plot of the input data (expTime vs Flux), with filtered points marked in red.  The bottom panel shows the difference between the inputOrdinate and the linearOrdinate.

            F2: The top panel shows the bincenters and values for the correction in the top panel.  Bottom panel is useless due to choice of overloading one debug method.

            F3: The top panel shows the relation between the linear flux (x-axis) and the input flux (y-axis).  This is a less useful plot.  The bottom panel shows the residual between the the input flux and the model correction. 

             

            Show
            czw Christopher Waters added a comment - From running a test with the debug code enabled:   F1: plot of the input data (expTime vs Flux), with filtered points marked in red.  The bottom panel shows the difference between the inputOrdinate and the linearOrdinate. F2: The top panel shows the bincenters and values for the correction in the top panel.  Bottom panel is useless due to choice of overloading one debug method. F3: The top panel shows the relation between the linear flux (x-axis) and the input flux (y-axis).  This is a less useful plot.  The bottom panel shows the residual between the the input flux and the model correction.   
            czw Christopher Waters made changes -
            Story Points 6 8
            czw Christopher Waters made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              czw Christopher Waters
              Reporter:
              czw Christopher Waters
              Reviewers:
              Merlin Fisher-Levine
              Watchers:
              Andrés Alejandro Plazas Malagón, Christopher Waters, Craig Lage, Merlin Fisher-Levine, Paul Price
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: