Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31193

Add new PSF chi2/npixels fields to apdb

    XMLWordPrintable

    Details

    • Story Points:
      4
    • Epic Link:
    • Sprint:
      AP F21-5 (October)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Once the chi2 and npixels fields are being output by PsfFluxAlgorithm, we can get them into the apdb output. This will require modifying the apdb schema and likely some changes to ap_association, among other things.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32197 Use psFlux chi2 values to investigate image subtraction quality

          • Done
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-29346 Compute PSFlux chi squared metric

          • Done
          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32265 ap_verify_hits2015 detectors 2 and 61 broken in gen3

          • Won't Fix
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-37911 Write dipoleNdata, dipoleLnl, dipoleCov fields to APDB

          • To Do

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Adding psChi2 and psNdata to the DiaSource.yaml file resulted in an ap_verify_ci_hits2015 run having those fields filled out in the assocation.db.

            Here's a jenkins run with ci_hsc to see if anything else breaks. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35140/pipeline

            Show
            Parejkoj John Parejko added a comment - Adding psChi2 and psNdata to the DiaSource.yaml file resulted in an ap_verify_ci_hits2015 run having those fields filled out in the assocation.db . Here's a jenkins run with ci_hsc to see if anything else breaks. https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35140/pipeline
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I've uploaded a histogram of the chi2/ndata values for my ap_verify_hits2015 run. These seem reasonable, though there are also 15 chi2 values < 0 (this worries me a bit!) and 26954 NaNs (no infinities).

            Show
            Parejkoj John Parejko added a comment - I've uploaded a histogram of the chi2/ndata values for my ap_verify_hits2015 run. These seem reasonable, though there are also 15 chi2 values < 0 (this worries me a bit!) and 26954 NaNs (no infinities).
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Chris Morrison [X]: Please review my changes to the ap_association yaml file on the following PR. Do I need to change anything else to call this complete? It looks like that was all I needed to get this value into the association.db file when running ap_verify.

            Krzysztof Findeisen: Please review these three PRs. I made some changes to the ap_verify docs to reflect the hits2015 processing trouble. I also added a note about using `-j`, since I wouldn't have thought to do so until I was told about it on slack.

            I also tweaked the API of one of the functions I added to analysis_ap: I'd forgotten to remove the `return`, which we don't want because it modifies the table in place.

            Show
            Parejkoj John Parejko added a comment - Chris Morrison [X] : Please review my changes to the ap_association yaml file on the following PR. Do I need to change anything else to call this complete? It looks like that was all I needed to get this value into the association.db file when running ap_verify. https://github.com/lsst/ap_association/pull/134 Krzysztof Findeisen : Please review these three PRs. I made some changes to the ap_verify docs to reflect the hits2015 processing trouble. I also added a note about using `-j`, since I wouldn't have thought to do so until I was told about it on slack. https://github.com/lsst/ap_verify_hits2015/pull/29 https://github.com/lsst/ap_verify/pull/141 I also tweaked the API of one of the functions I added to analysis_ap: I'd forgotten to remove the `return`, which we don't want because it modifies the table in place. https://github.com/lsst/analysis_ap/pull/3
            Hide
            Permalink
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            It looks good. If you see that values are being stored in this column in the Apdb after running some data through it, that's basically enough to confirm it's working.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - It looks good. If you see that values are being stored in this column in the Apdb after running some data through it, that's basically enough to confirm it's working.
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Sorry for the delay, looks good to me.

            Show
            krzys Krzysztof Findeisen added a comment - Sorry for the delay, looks good to me.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Krzysztof Findeisen
              Watchers:
              Chris Morrison [X] (Inactive), Eric Bellm, Ian Sullivan, John Parejko, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.