Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-31699

Gen3 DC2 reprocessing with bps and w_2021_36

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Attachments

        Issue Links

          Activity

          Hide
          brendal4 Brock Brendal [X] (Inactive) added a comment - - edited

          steps 1 and 2 ran to completion with no errors. step3 had some DB connection errors in forcedPhotCcd which were resubmitted and completed successfully with the dag rescue file. There were also 3 errors in imageDifference which were: "RuntimeError: Registry inconsistency while checking for existing outputs" and looked like:

          183772/28233_imageDifference_.4506709.err:RuntimeError: Registry inconsistency while checking for existing outputs: collection=2.2i/runs/test-med-1/w_2021_36/DM-31699/20210913T015853Z existingRefs=[DatasetRef(DatasetType('imageDifference_log', {band, instrument, skymap, detector, physical_filter, visit_system, visit}, ButlerLogRecords), {instrument: 'LSSTCam-imSim', skymap: 'DC2', detector: 58, visit: 183772, ...}, id=3e95c0f0-0eb2-496e-a8c2-b51fc50393bd, run='2.2i/runs/test-med-1/w_2021_36/DM-31699/20210913T015853Z')] missingRefs=[DatasetRef(DatasetType('goodSeeingDiff_differenceExp', {band, instrument, detector, physical_filter, visit_system, visit}, ExposureF), {instrument: 'LSSTCam-imSim', detector: 58, visit: 183772, ...}), DatasetRef(DatasetType('goodSeeingDiff_diaSrc', {band, instrument, detector, physical_filter, visit_system, visit}, SourceCatalog), {instrument: 'LSSTCam-imSim', detector: 58, visit: 183772, ...}), DatasetRef(DatasetType('imageDifference_metadata', {band, instrument, skymap, detector, physical_filter, visit_system, visit}, PropertySet), {instrument: 'LSSTCam-imSim', skymap: 'DC2', detector: 58, visit: 183772, ...})]
          

          Because there were only 3 errors to 11458 successes, I moved on to steps 4, 5, and 6, all of which completed with no errors (except for perhaps a few more DB connection issues)!

          Show
          brendal4 Brock Brendal [X] (Inactive) added a comment - - edited steps 1 and 2 ran to completion with no errors. step3 had some DB connection errors in forcedPhotCcd which were resubmitted and completed successfully with the dag rescue file. There were also 3 errors in imageDifference which were: "RuntimeError: Registry inconsistency while checking for existing outputs" and looked like: 183772 /28233_imageDifference_. 4506709 .err:RuntimeError: Registry inconsistency while checking for existing outputs: collection= 2 .2i/runs/test-med- 1 /w_2021_36/DM- 31699 /20210913T015853Z existingRefs=[DatasetRef(DatasetType( 'imageDifference_log' , {band, instrument, skymap, detector, physical_filter, visit_system, visit}, ButlerLogRecords), {instrument: 'LSSTCam-imSim' , skymap: 'DC2' , detector: 58 , visit: 183772 , ...}, id=3e95c0f0-0eb2-496e-a8c2-b51fc50393bd, run= '2.2i/runs/test-med-1/w_2021_36/DM-31699/20210913T015853Z' )] missingRefs=[DatasetRef(DatasetType( 'goodSeeingDiff_differenceExp' , {band, instrument, detector, physical_filter, visit_system, visit}, ExposureF), {instrument: 'LSSTCam-imSim' , detector: 58 , visit: 183772 , ...}), DatasetRef(DatasetType( 'goodSeeingDiff_diaSrc' , {band, instrument, detector, physical_filter, visit_system, visit}, SourceCatalog), {instrument: 'LSSTCam-imSim' , detector: 58 , visit: 183772 , ...}), DatasetRef(DatasetType( 'imageDifference_metadata' , {band, instrument, skymap, detector, physical_filter, visit_system, visit}, PropertySet), {instrument: 'LSSTCam-imSim' , skymap: 'DC2' , detector: 58 , visit: 183772 , ...})] Because there were only 3 errors to 11458 successes, I moved on to steps 4, 5, and 6, all of which completed with no errors (except for perhaps a few more DB connection issues)!
          Hide
          brendal4 Brock Brendal [X] (Inactive) added a comment - - edited

          All subsequent steps and faro completed successfully.

          Show
          brendal4 Brock Brendal [X] (Inactive) added a comment - - edited All subsequent steps and faro completed successfully.

            People

            Assignee:
            brendal4 Brock Brendal [X] (Inactive)
            Reporter:
            brendal4 Brock Brendal [X] (Inactive)
            Watchers:
            Brock Brendal [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.