Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26987

Update filtering of matched catalog to operate before creating GroupView

    XMLWordPrintable

    Details

      Description

      The way filterMatches.py is working it throws out the entire group of matched sources when any of them is flagged. We need to change the filtering to happen on the individual source measurements, rather than the entire group.

        Attachments

          Issue Links

            Activity

            Hide
            jcarlin Jeffrey Carlin added a comment - - edited

            This turned out to require updates to drp_pipe as well. The Jenkins build with changes to both packages was successful.

            Show
            jcarlin Jeffrey Carlin added a comment - - edited This turned out to require updates to drp_pipe  as well. The Jenkins build with changes to both packages was successful.
            Hide
            jcarlin Jeffrey Carlin added a comment - - edited

            A note for the future: an error in building/running ci_hsc_gen3 was difficult to track down. The issue was that we needed to add the new matched catalog datasets to pipelines/HSC/DRP-RC2.yaml from drp_pipe that is imported by ci_hsc_gen3. Even though the pipeline in ci_hsc_gen3 excludes almost all of the faro steps, the fact that they were not added explicitly to the faro_step3 and faro_step5 subsets meant that the new dataset types were not getting excluded. The solution was to add them to those subsets in the "default" pipelines, so they'll be part of the list that is excluded from consideration.

            Show
            jcarlin Jeffrey Carlin added a comment - - edited A note for the future: an error in building/running ci_hsc_gen3 was difficult to track down. The issue was that we needed to add the new matched catalog datasets to pipelines/HSC/DRP-RC2.yaml  from drp_pipe  that is imported by ci_hsc_gen3 . Even though the pipeline in ci_hsc_gen3  excludes almost all of the faro  steps, the fact that they were not added explicitly to the faro_step3  and faro_step5  subsets meant that the new dataset types were not getting excluded. The solution was to add them to those subsets in the "default" pipelines, so they'll be part of the list that is excluded from consideration.
            Hide
            ctslater Colin Slater added a comment -

            Good find on that convoluted ci_hsc issue.

            Show
            ctslater Colin Slater added a comment - Good find on that convoluted ci_hsc issue.

              People

              Assignee:
              jcarlin Jeffrey Carlin
              Reporter:
              jcarlin Jeffrey Carlin
              Reviewers:
              Colin Slater
              Watchers:
              Colin Slater, Jeffrey Carlin, Leanne Guy
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.