Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33959

Calibrate Source Tables with Global Calibrations (jointcal/FGCM)

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      DRP S22B, DRP Plotting Sprint Mar 2022
    • Team:
      Data Release Production
    • Urgent?:
      No

      Attachments

        Issue Links

        is duplicated by

        Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-31855 Apply external calibs and insert calibrated fluxes/positions into the source tables.

        • Won't Fix
        mentioned in

        Page Loading...

        Page Loading...

          Activity

          Ascending order - Click to sort in descending order
          Hide
          Permalink
          kbechtol Keith Bechtol added a comment -

          Would this work also include applying astrometric corrections, or is that separate?

          Show
          kbechtol Keith Bechtol added a comment - Would this work also include applying astrometric corrections, or is that separate?
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          We use jointcal for astrometry. This ticket covers both.

          Show
          yusra Yusra AlSayyad added a comment - We use jointcal for astrometry. This ticket covers both.
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment - - edited

          This is ready for review. Jenkins going here: https://ci.lsst.codes/job/stack-os-matrix/36645/display/redirect

          Note that this ticket addresses the production of PreSources (preSourceTable_visit), Sources (sourceTable_visit), and fgcm/jointcal using preSources as input in all the HSC DRP pipelines and the DECam pipeline, and rc2_subset pipeline.

          It does not address that faro in the future will want to use Sources as input during RC2 and production runs. Jeffrey Carlin and I discussed this pair coding last week. Because that requires a reordering of tasks in the subsets, I'm going to address that on DM-34853.

          Show
          yusra Yusra AlSayyad added a comment - - edited This is ready for review. Jenkins going here: https://ci.lsst.codes/job/stack-os-matrix/36645/display/redirect Note that this ticket addresses the production of PreSources (preSourceTable_visit), Sources (sourceTable_visit), and fgcm/jointcal using preSources as input in all the HSC DRP pipelines and the DECam pipeline, and rc2_subset pipeline. It does not address that faro in the future will want to use Sources as input during RC2 and production runs. Jeffrey Carlin and I discussed this pair coding last week. Because that requires a reordering of tasks in the subsets, I'm going to address that on DM-34853 .
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          Thanks for volunteering:

          Another note is that in on piece of naming unfortunateness is that the new `writeSourceTable` has skymap added to its dimensions (because jointcal is per tract and it needs to know the tract geometry to choose the best one), which means that I can't reuse the old name `writeSourceTable`. I renamed it to `writeRecalibratedSourceTable` and kept `transformSourceTable` and `consolidateSourceTable` the same, but am open to renaming those too.

          Show
          yusra Yusra AlSayyad added a comment - Thanks for volunteering: https://github.com/lsst/ci_hsc_gen3/pull/76 https://github.com/lsst/pipe_tasks/pull/685 https://github.com/lsst/drp_pipe/pull/20 https://github.com/lsst-dm/rc2_subset/pull/16 Another note is that in on piece of naming unfortunateness is that the new `writeSourceTable` has skymap added to its dimensions (because jointcal is per tract and it needs to know the tract geometry to choose the best one), which means that I can't reuse the old name `writeSourceTable`. I renamed it to `writeRecalibratedSourceTable` and kept `transformSourceTable` and `consolidateSourceTable` the same, but am open to renaming those too.
          Hide
          Permalink
          yusra Yusra AlSayyad added a comment -

          Oh and if you're curious what happened with that Jenkins run, between the macos and centos Jenkins runs starting, Dan merged something to pipe_tasks,

          Show
          yusra Yusra AlSayyad added a comment - Oh and if you're curious what happened with that Jenkins run, between the macos and centos Jenkins runs starting, Dan merged something to pipe_tasks,
          Hide
          Permalink
          csaunder Clare Saunders added a comment -

          This mostly looks fine to me. I just have a few questions on the drp_pipe PR, which may turn out to be off base.

          Show
          csaunder Clare Saunders added a comment - This mostly looks fine to me. I just have a few questions on the drp_pipe PR, which may turn out to be off base.

            People

            Assignee:
            yusra Yusra AlSayyad
            Reporter:
            yusra Yusra AlSayyad
            Reviewers:
            Clare Saunders
            Watchers:
            Clare Saunders, Keith Bechtol, Peter Ferguson, Yusra AlSayyad
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.