Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36356

Reprocess HSC-RC2 at USDF with w_2022_40

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Team:
      Campaign Management
    • Urgent?:
      No

      Description

      Use the as-is PanDA setup at USDF.

      Base pipelineYaml: $DRP_PIPE_DIR/pipelines/HSC/DRP-RC2.yaml

      Use S3DF for submission and execution if possible. Otherwise SDF.

      Use the butler repo /sdf/group/rubin/repo/main_20210215 if possible.
      Otherwise use the temporary HSC repo /sdf/group/rubin/repo/main_20220411

      When everything is at S3DF, use /sdf/data/rubin/repo/main

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            1600 json metrics have been generated and dispatched.

            The dispatch Job ID ranges from 118721 to 120322.

            Angelo Fausti may you please help us synchronize the SQuaSH DB with InfluxDB? Thank you.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - 1600 json metrics have been generated and dispatched. The dispatch Job ID ranges from 118721 to 120322. Angelo Fausti may you please help us synchronize the SQuaSH DB with InfluxDB? Thank you.
            Hide
            afausti Angelo Fausti added a comment -

            Hsin-Fang Chiang that's done and I confirm I see the w_2022_40 tag in the Chronograf dashboards.

            Show
            afausti Angelo Fausti added a comment - Hsin-Fang Chiang  that's done and I confirm I see the w_2022_40 tag in the Chronograf dashboards.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Just getting a rough estimate of clock time between the collection timestamps in butler and the corresponding merge job timestamps in PanDA (like previously )

            • step1 took 70 hours
            • step2 took 27 hours
            • step3 took 136 hours
            • step4 took 27 hours
            • step5 took 15 hours
            • step6 took 3 hours
            • step7 took 1 hours
            • faro took 15 hours
            • both analysis 3 hours
            Show
            hchiang2 Hsin-Fang Chiang added a comment - Just getting a rough estimate of clock time between the collection timestamps in butler and the corresponding merge job timestamps in PanDA ( like previously ) step1 took 70 hours step2 took 27 hours step3 took 136 hours step4 took 27 hours step5 took 15 hours step6 took 3 hours step7 took 1 hours faro took 15 hours both analysis 3 hours
            Hide
            yusra Yusra AlSayyad added a comment -

            Warning note for posterity: the object tables from this rerun are not useable for coadd science quality metrics/plots. They're missing the 47 forcedPhotCoadd results.

            Matched-visit metrics are fine.

            Show
            yusra Yusra AlSayyad added a comment - Warning note for posterity: the object tables from this rerun are not useable for coadd science quality metrics/plots. They're missing the 47 forcedPhotCoadd results. Matched-visit metrics are fine.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Post mortem on the execution issues... After whatever happened with the PanDA DAG bug and other issues until about Oct 14,  we should have run another rescue of step3, to recover those forcedPhotCoadd, before continuing with consolidateObjectTable. 

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Post mortem on the execution issues... After whatever happened with the PanDA DAG bug and other issues until about Oct 14,  we should have run another rescue of step3, to recover those forcedPhotCoadd, before continuing with consolidateObjectTable. 

              People

              Assignee:
              eiger Orion Eiger
              Reporter:
              hchiang2 Hsin-Fang Chiang
              Watchers:
              Angelo Fausti, Hsin-Fang Chiang, Orion Eiger, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.