Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-34809

Add isolatedStarAssociation/finalizeCharacterization to DECam step2

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: drp_pipe, pipe_tasks
    • Labels:
      None

      Description

      The recent ticket DM-34391 added in two new tasks into HSC Step 2 processing: isolatedStarAssociation and finalizeCharacterization. The imSim branch of this ticket adds both of these tasks into both Step 2 and Step 3.

      Step 3 now relies on these two tasks having been run, therefore breaking DECam Step 3 processing. This ticket adds these tasks into the standard DECam data processing pipeline in Step 2, and appends standard DECam bands to the end of the band order config list for the isolated star association task.

      Update: during testing, a small typo was discovered in pipe_tasks/python/lsst/pipe/tasks/finalizeCharacterization.py#L206. This typo causes all GAaP measurements to fail (producing a warning, rather than a failure). We also fix this typo on this ticket.

      Update 2: it appears that both isolatedStarAssociation and finalizeCharacterization were added into the step 2 subset and the step 3 subset in $DRP_PIPE_DIR/ingredients/LSSTCam-imSim/DRP.yaml. These only need to be in step 2, so we remove the extra superfluous tasks from the step 3 subset on this ticket also.

        Attachments

          Issue Links

            Activity

            Hide
            lskelvin Lee Kelvin added a comment - - edited

            The necessary tasks have been added into DECam step 2 processing, as have the config over-rides adding the two currently-used DECam Merian narrow-bands. The typo in finalizeCharacterization.py and the extra superfluous tasks, both discussed above, have also been fixed on this ticket.

            A test step 2 DECam run is currently running on lsst-devl using this command:

            DATERANGE="exposure.day_obs > 20210901 AND exposure.day_obs < 20211215"
            SELECTION="exposure.observation_type='science' AND detector NOT IN (31,61)"
             
            pipetask --long-log run --register-dataset-types -j 12 \
            --instrument lsst.obs.decam.DarkEnergyCamera \
            -b /project/lskelvin/repo \
            -i DECam/runs/merian/w_2022_20 \
            -o u/lskelvin/scratch/merian/w_2022_20b \
            -p $DRP_PIPE_DIR/pipelines/DECam/DRP-Merian.yaml#step2 \
            -d "instrument='DECam' AND $DATERANGE AND $SELECTION"
            

            A Jenkins run has been kicked off for this ticket using the following build products: lsst_distrib lsst_ci ci_hsc ci_imsim. Would you be okay reviewing this ticket, Eli Rykoff?

            Show
            lskelvin Lee Kelvin added a comment - - edited The necessary tasks have been added into DECam step 2 processing, as have the config over-rides adding the two currently-used DECam Merian narrow-bands. The typo in finalizeCharacterization.py and the extra superfluous tasks, both discussed above, have also been fixed on this ticket. A test step 2 DECam run is currently running on lsst-devl using this command: DATERANGE="exposure.day_obs > 20210901 AND exposure.day_obs < 20211215" SELECTION="exposure.observation_type='science' AND detector NOT IN (31,61)"   pipetask --long-log run --register-dataset-types -j 12 \ --instrument lsst.obs.decam.DarkEnergyCamera \ -b /project/lskelvin/repo \ -i DECam/runs/merian/w_2022_20 \ -o u/lskelvin/scratch/merian/w_2022_20b \ -p $DRP_PIPE_DIR/pipelines/DECam/DRP-Merian.yaml#step2 \ -d "instrument='DECam' AND $DATERANGE AND $SELECTION" A Jenkins run has been kicked off for this ticket using the following build products: lsst_distrib lsst_ci ci_hsc ci_imsim . Would you be okay reviewing this ticket, Eli Rykoff ?
            Hide
            erykoff Eli Rykoff added a comment -

            Thanks for finding my typos on this ticket!

            Show
            erykoff Eli Rykoff added a comment - Thanks for finding my typos on this ticket!
            Hide
            lskelvin Lee Kelvin added a comment -

            Both Jenkins and my DECam test run completed successfully. Thanks for the quick review Eli, much appreciated. Branches merged and deleted.

            Show
            lskelvin Lee Kelvin added a comment - Both Jenkins and my DECam test run completed successfully. Thanks for the quick review Eli, much appreciated. Branches merged and deleted.

              People

              Assignee:
              lskelvin Lee Kelvin
              Reporter:
              lskelvin Lee Kelvin
              Reviewers:
              Eli Rykoff
              Watchers:
              Eli Rykoff, Lee Kelvin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.