Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31825

Parquet Table clean up before DP0.2 Preliminary run.

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      6
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Attachments

        Issue Links

        duplicates

        Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31813 Add diaObjectId's coords to DiaSource Parquet Table before ingest

        • Won't Fix
        is triggered by

        RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-807 Modify column names in Source/Object DRP Parquet Tables

        • Implemented
        is triggering

        Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-32022 add index to column list

        • Done
        relates to

        Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31339 Add a round of columns needed for QA to object table

        • Done

        Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31912 Add additional GAaP circular apertures for DP0.2

        • Done
        mentioned in

        Page Loading...

          Activity

          Ascending order - Click to sort in descending order
          No builds found.
          yusra Yusra AlSayyad created issue -
          yusra Yusra AlSayyad made changes -
          Field Original Value New Value
          Link This issue is triggered by RFC-807 [ RFC-807 ]
          yusra Yusra AlSayyad made changes -
          Labels drp-gen3
          yusra Yusra AlSayyad made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          kannawad Arun Kannawadi made changes -
          Link This issue relates to DM-31912 [ DM-31912 ]
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          Alright, I'm going to leave sdm_schemas until tomorrow, so ci_imsim and ci_hsc won't pass, but will still give useful logs. ci_imsim completes with no errors locally (expect for the schema check). running ci_hsc_gen3 locally now. Basic Jenkins running now.

          Sophie Reed
          https://github.com/lsst/obs_subaru/pull/391
          https://github.com/lsst/obs_lsst/pull/353
          https://github.com/lsst/pipe_tasks/pull/579
          https://github.com/lsst/meas_base/pull/193

          Eli Rykoff If you make PRs for fgcmcal and jointcal, I'll review those

          Chris Morrison [X]
          https://github.com/lsst/ap_association/pull/131
          There are commits on the pipe_tasks PR that you'll care about too. The DM-31813 implementation and a change to your simpleAssociation unit test.
          I owe you a ap_verify run too.

          Show
          yusra Yusra AlSayyad added a comment - Alright, I'm going to leave sdm_schemas until tomorrow, so ci_imsim and ci_hsc won't pass, but will still give useful logs. ci_imsim completes with no errors locally (expect for the schema check). running ci_hsc_gen3 locally now. Basic Jenkins running now. Sophie Reed https://github.com/lsst/obs_subaru/pull/391 https://github.com/lsst/obs_lsst/pull/353 https://github.com/lsst/pipe_tasks/pull/579 https://github.com/lsst/meas_base/pull/193 Eli Rykoff If you make PRs for fgcmcal and jointcal, I'll review those Chris Morrison [X] https://github.com/lsst/ap_association/pull/131 There are commits on the pipe_tasks PR that you'll care about too. The DM-31813 implementation and a change to your simpleAssociation unit test. I owe you a ap_verify run too.
          yusra Yusra AlSayyad made changes -
          Reviewers Chris Morrison, Sophie Reed [ cmorrison, sophiereed ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          ap_verify is happy, ci_hsc is happy, ci_imsim is happy.
          Final Jenkins run going now.

          Final columns: https://github.com/yalsayyad/dm_notebooks/blob/master/object-table/ci_imsim_check_for_DRP_Table_and_Column.ipynb

          Show
          yusra Yusra AlSayyad added a comment - ap_verify is happy, ci_hsc is happy, ci_imsim is happy. Final Jenkins run going now. Final columns: https://github.com/yalsayyad/dm_notebooks/blob/master/object-table/ci_imsim_check_for_DRP_Table_and_Column.ipynb
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          Just waiting for the green light from Chris Morrison [X] and jenkins.

          Show
          yusra Yusra AlSayyad added a comment - Just waiting for the green light from Chris Morrison [X] and jenkins.
          cmorrison Chris Morrison [X] (Inactive) made changes -
          Reviewers Chris Morrison, Sophie Reed [ cmorrison, sophiereed ] Sophie Reed [ sophiereed ]
          yusra Yusra AlSayyad made changes -
          Link This issue duplicates DM-31813 [ DM-31813 ]
          sophiereed Sophie Reed made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          yusra Yusra AlSayyad made changes -
          Link This issue relates to DM-31339 [ DM-31339 ]
          hchiang2 Hsin-Fang Chiang made changes -
          Link This issue is triggering DM-32022 [ DM-32022 ]
          yusra Yusra AlSayyad made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]
          lguy Leanne Guy made changes -
          Remote Link This issue links to "Page (Confluence)" [ 31242 ]
          igoodenow Iain Goodenow made changes -
          Summary Parquet Table clean up before DP0.2 Preliminary run. Parquet Table clean up before DP0.2 Preliminary run.

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Sophie Reed
            Watchers:
            Chris Morrison [X] (Inactive), Sophie Reed, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.