Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33959

Calibrate Source Tables with Global Calibrations (jointcal/FGCM)

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      DRP S22B, DRP Plotting Sprint Mar 2022
    • Team:
      Data Release Production
    • Urgent?:
      No

      Attachments

        Issue Links

          Activity

          No builds found.
          yusra Yusra AlSayyad created issue -
          yusra Yusra AlSayyad made changes -
          Field Original Value New Value
          Team Data Release Production [ 10301 ]
          yusra Yusra AlSayyad made changes -
          Assignee Yusra AlSayyad [ yusra ]
          yusra Yusra AlSayyad made changes -
          Status To Do [ 10001 ] In Progress [ 3 ]
          lguy Leanne Guy made changes -
          Link This issue is duplicated by DM-31855 [ DM-31855 ]
          Hide
          kbechtol Keith Bechtol added a comment -

          Would this work also include applying astrometric corrections, or is that separate?

          Show
          kbechtol Keith Bechtol added a comment - Would this work also include applying astrometric corrections, or is that separate?
          Hide
          yusra Yusra AlSayyad added a comment -

          We use jointcal for astrometry. This ticket covers both.

          Show
          yusra Yusra AlSayyad added a comment - We use jointcal for astrometry. This ticket covers both.
          lguy Leanne Guy made changes -
          Remote Link This issue links to "Page (Confluence)" [ 32593 ]
          yusra Yusra AlSayyad made changes -
          Sprint DRP Plotting Sprint Mar 2022 [ 1158 ] DRP S22B, DRP Plotting Sprint Mar 2022 [ 1138, 1158 ]
          kbechtol Keith Bechtol made changes -
          Remote Link This issue links to "Page (Confluence)" [ 32834 ]
          yusra Yusra AlSayyad made changes -
          Labels hsc-pdr4
          Hide
          yusra Yusra AlSayyad added a comment - - edited

          This is ready for review. Jenkins going here: https://ci.lsst.codes/job/stack-os-matrix/36645/display/redirect

          Note that this ticket addresses the production of PreSources (preSourceTable_visit), Sources (sourceTable_visit), and fgcm/jointcal using preSources as input in all the HSC DRP pipelines and the DECam pipeline, and rc2_subset pipeline.

          It does not address that faro in the future will want to use Sources as input during RC2 and production runs. Jeffrey Carlin and I discussed this pair coding last week. Because that requires a reordering of tasks in the subsets, I'm going to address that on DM-34853.

          Show
          yusra Yusra AlSayyad added a comment - - edited This is ready for review. Jenkins going here: https://ci.lsst.codes/job/stack-os-matrix/36645/display/redirect Note that this ticket addresses the production of PreSources (preSourceTable_visit), Sources (sourceTable_visit), and fgcm/jointcal using preSources as input in all the HSC DRP pipelines and the DECam pipeline, and rc2_subset pipeline. It does not address that faro in the future will want to use Sources as input during RC2 and production runs. Jeffrey Carlin and I discussed this pair coding last week. Because that requires a reordering of tasks in the subsets, I'm going to address that on DM-34853 .
          Hide
          yusra Yusra AlSayyad added a comment -

          Thanks for volunteering:

          Another note is that in on piece of naming unfortunateness is that the new `writeSourceTable` has skymap added to its dimensions (because jointcal is per tract and it needs to know the tract geometry to choose the best one), which means that I can't reuse the old name `writeSourceTable`. I renamed it to `writeRecalibratedSourceTable` and kept `transformSourceTable` and `consolidateSourceTable` the same, but am open to renaming those too.

          Show
          yusra Yusra AlSayyad added a comment - Thanks for volunteering: https://github.com/lsst/ci_hsc_gen3/pull/76 https://github.com/lsst/pipe_tasks/pull/685 https://github.com/lsst/drp_pipe/pull/20 https://github.com/lsst-dm/rc2_subset/pull/16 Another note is that in on piece of naming unfortunateness is that the new `writeSourceTable` has skymap added to its dimensions (because jointcal is per tract and it needs to know the tract geometry to choose the best one), which means that I can't reuse the old name `writeSourceTable`. I renamed it to `writeRecalibratedSourceTable` and kept `transformSourceTable` and `consolidateSourceTable` the same, but am open to renaming those too.
          yusra Yusra AlSayyad made changes -
          Reviewers Clare Saunders [ csaunder ]
          Status In Progress [ 3 ] In Review [ 10004 ]
          Hide
          yusra Yusra AlSayyad added a comment -

          Oh and if you're curious what happened with that Jenkins run, between the macos and centos Jenkins runs starting, Dan merged something to pipe_tasks,

          Show
          yusra Yusra AlSayyad added a comment - Oh and if you're curious what happened with that Jenkins run, between the macos and centos Jenkins runs starting, Dan merged something to pipe_tasks,
          Hide
          csaunder Clare Saunders added a comment -

          This mostly looks fine to me. I just have a few questions on the drp_pipe PR, which may turn out to be off base.

          Show
          csaunder Clare Saunders added a comment - This mostly looks fine to me. I just have a few questions on the drp_pipe PR, which may turn out to be off base.
          csaunder Clare Saunders made changes -
          Status In Review [ 10004 ] Reviewed [ 10101 ]
          yusra Yusra AlSayyad made changes -
          Resolution Done [ 10000 ]
          Status Reviewed [ 10101 ] Done [ 10002 ]
          yusra Yusra AlSayyad made changes -
          Epic Link DM-30551 [ 511217 ]
          Story Points 4
          yusra Yusra AlSayyad made changes -
          Epic Link DM-30551 [ 511217 ] DM-30555 [ 511221 ]

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Clare Saunders
            Watchers:
            Clare Saunders, Keith Bechtol, Peter Ferguson, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.