Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23104

Augment ObjectTable to be useable for QA

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      2
    • Epic Link:
    • Sprint:
      DRP S20-2 (Jan), DRP S20-3 (Feb)
    • Team:
      Data Release Production

      Description

      I went through the exercise of trying to use the ObjectTable for some QA last week. I had to add a lot of columns we use regularly like per-band CalibFlux, and per-band PixelFlags.

      In addition to these, I'm going to add the columns that pipe_analysis uses as well. ObjectTable is going to bitrot if its too skinny to be used in practice.

      ideally, it'd be something that Tim Morton [X] could base the example qa-drilldown site on.

        Attachments

          Activity

          Show
          yusra Yusra AlSayyad added a comment - In the market for a reviewer. 2 PRs: https://github.com/lsst/obs_subaru/pull/255 https://github.com/lsst/pipe_tasks/pull/351 Jenkins pending here: https://ci.lsst.codes/job/stack-os-matrix/31232/
          Hide
          cmorrison Chris Morrison [X] (Inactive) added a comment -

          Take a look at the github comments. Mostly just need an explanation of why a functor default changed.

          Show
          cmorrison Chris Morrison [X] (Inactive) added a comment - Take a look at the github comments. Mostly just need an explanation of why a functor default changed.
          Hide
          yusra Yusra AlSayyad added a comment -

          Thanks for the review Chris Morrison [X]!

          Hsin-Fang Chiang beat me by merging DM-23224 first, which enforces all new columns in the Object.yaml have a matching column name in cat/yml/hsc.yaml. There's now a PR for cat. github is NOT handling the diff well. It's really simpler than it seems.

          Hsin-Fang Chiang, will you take a look at the new cat PR, https://github.com/lsst/cat/pull/25/files , when you get a chance? I would really like to use this change for the PDR2 run of the transform tasks. If you review this tonight, I'll try to get it in tonight. If not, will you use the w_2020_08 weekly for the transform tasks?

          Show
          yusra Yusra AlSayyad added a comment - Thanks for the review Chris Morrison [X] ! Hsin-Fang Chiang beat me by merging DM-23224 first, which enforces all new columns in the Object.yaml have a matching column name in cat/yml/hsc.yaml . There's now a PR for cat . github is NOT handling the diff well. It's really simpler than it seems. Hsin-Fang Chiang , will you take a look at the new cat PR, https://github.com/lsst/cat/pull/25/files , when you get a chance? I would really like to use this change for the PDR2 run of the transform tasks. If you review this tonight, I'll try to get it in tonight. If not, will you use the w_2020_08 weekly for the transform tasks?
          Hide
          hchiang2 Hsin-Fang Chiang added a comment -

          You can merge if Jenkins ci_hsc_gen2 is green and if coord_ra/coord_dec can continue being the coordinates of an object. The per-band coordinates would be treated like any other columns and the smart things Qserv does with spatial queries will be based on coord_ra/coord_dec only. Does that sound alright to you? I'd like to understand better how the per-band coordinates may be used and bring to the DAX team for discussions.

          For the current PDR2 run, the transform tasks are definitely >1 week away (or maybe 2...)

          Show
          hchiang2 Hsin-Fang Chiang added a comment - You can merge if Jenkins ci_hsc_gen2 is green and if coord_ra/coord_dec can continue being the coordinates of an object. The per-band coordinates would be treated like any other columns and the smart things Qserv does with spatial queries will be based on coord_ra/coord_dec only. Does that sound alright to you? I'd like to understand better how the per-band coordinates may be used and bring to the DAX team for discussions. For the current PDR2 run, the transform tasks are definitely >1 week away (or maybe 2...)
          Hide
          yusra Yusra AlSayyad added a comment -

          That's right: coord_ra and coord_dec are still the "reference coordinates" taken from deepCoadd_ref

          You can safely assume that no one will ever need to run spatial queries on the per-band ones.

          Show
          yusra Yusra AlSayyad added a comment - That's right: coord_ra and coord_dec are still the "reference coordinates" taken from deepCoadd_ref You can safely assume that no one will ever need to run spatial queries on the per-band ones.
          Show
          yusra Yusra AlSayyad added a comment - We'll see what https://ci.lsst.codes/job/stack-os-matrix/31252/display/redirect says
          Hide
          yusra Yusra AlSayyad added a comment -

          Jenkins didn't finish until 55 minutes after the weekly is tagged (and I was asleep anyway), so this did not make it into w07.

          Show
          yusra Yusra AlSayyad added a comment - Jenkins didn't finish until 55 minutes after the weekly is tagged (and I was asleep anyway), so this did not make it into w07.

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Chris Morrison [X] (Inactive)
            Watchers:
            Chris Morrison [X] (Inactive), Hsin-Fang Chiang, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.