Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-27779

Fix validate_drp_gen3 breakage due to lack of HSC/masks collection

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      DM-27147 broke the validate_drp_gen3 job, presumably because the code that creates the HSC/defaults umbrella collection at the end of ConvertRepoTask assumed that there was a HSC/masks. I suspect that's because the configuration (from obs_subaru) told it that there would be, but I'm not sure if the best fix is to add a config override in this conversion, or make the code in obs_base ignore that configuration if the collection isn't actually present.

        Attachments

          Activity

          Hide
          jbosch Jim Bosch added a comment -

          The easy fix for this is to set config.extraUmbrellaChildren=[] when running ConvertRepoTask. Simon Krughoff, Jeffrey Carlin, Keith Bechtol - can one of you point me at where that happens for validate_drp_gen3? I can't find the relevant git repos.

          Show
          jbosch Jim Bosch added a comment - The easy fix for this is to set config.extraUmbrellaChildren=[] when running ConvertRepoTask. Simon Krughoff , Jeffrey Carlin , Keith Bechtol - can one of you point me at where that happens for validate_drp_gen3? I can't find the relevant git repos.
          Hide
          krughoff Simon Krughoff added a comment -

          This is the script that does the conversion: https://github.com/lsst-dmsst/metric-pipeline-tasks/blob/master/bin/hsc_gen2_to_gen3.sh

          Let me know if that's not the information you need.

          Show
          krughoff Simon Krughoff added a comment - This is the script that does the conversion: https://github.com/lsst-dmsst/metric-pipeline-tasks/blob/master/bin/hsc_gen2_to_gen3.sh Let me know if that's not the information you need.
          Hide
          jbosch Jim Bosch added a comment -

          Thanks, that's exactly what I was looking for. I'll have a fix out for review shortly.

          Show
          jbosch Jim Bosch added a comment - Thanks, that's exactly what I was looking for. I'll have a fix out for review shortly.
          Hide
          jbosch Jim Bosch added a comment -

          Probable fix is ready for review on https://github.com/lsst-dmsst/metric-pipeline-tasks/pull/41.

          Is there a way to trigger a Jenkins run to test this on the branch, or some similarly easy way to do that? Or should we just merge to master and wait for the next scheduled run?

          Show
          jbosch Jim Bosch added a comment - Probable fix is ready for review on https://github.com/lsst-dmsst/metric-pipeline-tasks/pull/41 . Is there a way to trigger a Jenkins run to test this on the branch, or some similarly easy way to do that? Or should we just merge to master and wait for the next scheduled run?
          Hide
          krughoff Simon Krughoff added a comment -

          I'll get to this ASAP, but to answer the question, there is no easy way to test for this. I think we just need to merge and run it. I still haven't gotten this in the weekly, so maybe just running the validate_drp_gen3 job by hand is the best thing to do.

          Show
          krughoff Simon Krughoff added a comment - I'll get to this ASAP, but to answer the question, there is no easy way to test for this. I think we just need to merge and run it. I still haven't gotten this in the weekly, so maybe just running the validate_drp_gen3 job by hand is the best thing to do.
          Hide
          krughoff Simon Krughoff added a comment -

          Our normal process is that you would merge this PR since you made it, but since it's an odd repository, I'm happy to merge for you and kick a Jenkins job off.

          Show
          krughoff Simon Krughoff added a comment - Our normal process is that you would merge this PR since you made it, but since it's an odd repository, I'm happy to merge for you and kick a Jenkins job off.
          Hide
          jbosch Jim Bosch added a comment -

          I'm happy to merge for you and kick a Jenkins job off.

          In that case, please go ahead, and ping me if there's anything more for me to do on this ticket. Thanks!

          Show
          jbosch Jim Bosch added a comment - I'm happy to merge for you and kick a Jenkins job off. In that case, please go ahead, and ping me if there's anything more for me to do on this ticket. Thanks!
          Hide
          krughoff Simon Krughoff added a comment -

          Here's the build.

          Show
          krughoff Simon Krughoff added a comment - Here's the build .

            People

            Assignee:
            jbosch Jim Bosch
            Reporter:
            jbosch Jim Bosch
            Reviewers:
            Simon Krughoff
            Watchers:
            Jim Bosch, Simon Krughoff
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins Builds

                No builds found.