Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22277

Convert TransformObjectTableTask and friends to Gen 3

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Story Points:
      2
    • Sprint:
      DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct), DRP S21a (Dec Jan)
    • Team:
      Data Release Production
    • Urgent?:
      No

      Attachments

        Issue Links

          Activity

          No builds found.
          yusra Yusra AlSayyad created issue -
          yusra Yusra AlSayyad made changes -
          Field Original Value New Value
          Status To Do [ 10001 ] In Progress [ 3 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov) [ 972 ] DRP F19-6 (Nov), DRP S20-2 (Jan) [ 972, 998 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan) [ 972, 998 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb) [ 972, 998, 1000 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb) [ 972, 998, 1000 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr) [ 972, 998, 1000, 1014 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr) [ 972, 998, 1000, 1014 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May) [ 972, 998, 1000, 1014, 1021 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May) [ 972, 998, 1000, 1014, 1021 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June) [ 972, 998, 1000, 1014, 1021, 1030 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June) [ 972, 998, 1000, 1014, 1021, 1030 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July) [ 972, 998, 1000, 1014, 1021, 1030, 1036 ]
          yusra Yusra AlSayyad made changes -
          Epic Link PREOPS-93 [ 438318 ]
          yusra Yusra AlSayyad made changes -
          Urgent? off
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July) [ 972, 998, 1000, 1014, 1021, 1030, 1036 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug) [ 972, 998, 1000, 1014, 1021, 1030, 1036, 1042 ]
          yusra Yusra AlSayyad made changes -
          Remote Link This issue links to "Page (Confluence)" [ 25988 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug) [ 972, 998, 1000, 1014, 1021, 1030, 1036, 1042 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct) [ 972, 998, 1000, 1014, 1021, 1030, 1036, 1042, 1054 ]
          yusra Yusra AlSayyad made changes -
          Epic Link PREOPS-93 [ 438318 ] PREOPS-281 [ 442892 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct) [ 972, 998, 1000, 1014, 1021, 1030, 1036, 1042, 1054 ] DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct), DRP S21 (Dec Jan) [ 972, 998, 1000, 1014, 1021, 1030, 1036, 1042, 1054, 1071 ]
          yusra Yusra AlSayyad made changes -
          Remote Link This issue links to "Page (Confluence)" [ 27759 ]
          yusra Yusra AlSayyad made changes -
          Labels drp-parity w_2020_14
          yusra Yusra AlSayyad made changes -
          Remote Link This issue links to "Page (Confluence)" [ 27759 ]
          yusra Yusra AlSayyad made changes -
          Reviewers Eli Rykoff [ erykoff ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          yusra Yusra AlSayyad made changes -
          Status In Review [ 10004 ] In Progress [ 3 ]
          yusra Yusra AlSayyad made changes -
          Comment [ Since this is causing you pain, want to review? :) (Not today)
          Jenkins running: https://ci.lsst.codes/job/stack-os-matrix/33744/display/redirect ]
          yusra Yusra AlSayyad made changes -
          Labels drp-parity w_2020_14 drp-parity w_2021_14
          yusra Yusra AlSayyad made changes -
          Epic Link PREOPS-281 [ 442892 ] PREOPS-450 [ 463950 ]
          Hide
          yusra Yusra AlSayyad added a comment - - edited

          Ready for review.
          These tasks are run in ci_hsc. Jenkins working here: https://ci.lsst.codes/job/stack-os-matrix/33916/display/redirect

          Will compare the Gen2 and Gen3 file contents from ci_hsc tomorrow. We don't necessarily expect these to work on older converted repos. Bands are different in the older repos (These tasks run on older converted repos, but contents are NaN because of the bands) and patch numbers are strings->ints.

          Gen3 objectTables will have one more column: 'skymap'.

          Show
          yusra Yusra AlSayyad added a comment - - edited Ready for review. These tasks are run in ci_hsc . Jenkins working here: https://ci.lsst.codes/job/stack-os-matrix/33916/display/redirect Will compare the Gen2 and Gen3 file contents from ci_hsc tomorrow. We don't necessarily expect these to work on older converted repos. Bands are different in the older repos (These tasks run on older converted repos, but contents are NaN because of the bands) and patch numbers are strings->ints. Gen3 objectTables will have one more column: 'skymap'.
          yusra Yusra AlSayyad made changes -
          Status In Progress [ 3 ] In Review [ 10004 ]
          erykoff Eli Rykoff made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          Hide
          yusra Yusra AlSayyad added a comment -

          OK, made the 2 fixes you requested and checked the outputs.

          I'm glad I decided to try to get this ticket in before w14, because we haven't had colors in our object tables since the filter to band migration (in either Gen2 or Gen3). I opened a new PR for obs_subaru specifying the filters as 'r' rather than 'HSC-R'. This works in both Gen2 and Gen3. Also fixed the NaN reference columns ci_hsc where we only have 'r' and 'i'.

          New Jenkins working here: https://ci.lsst.codes/job/stack-os-matrix/33933/display/redirect

          Show
          yusra Yusra AlSayyad added a comment - OK, made the 2 fixes you requested and checked the outputs. I'm glad I decided to try to get this ticket in before w14, because we haven't had colors in our object tables since the filter to band migration (in either Gen2 or Gen3). I opened a new PR for obs_subaru specifying the filters as 'r' rather than 'HSC-R'. This works in both Gen2 and Gen3. Also fixed the NaN reference columns ci_hsc where we only have 'r' and 'i'. New Jenkins working here: https://ci.lsst.codes/job/stack-os-matrix/33933/display/redirect
          yusra Yusra AlSayyad made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Eli Rykoff
            Watchers:
            Eli Rykoff, Hsin-Fang Chiang, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.