Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30630

Filter look-up for in isrTask is broken in a few places

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Labels:
      None
    • Story Points:
      3
    • Epic Link:
    • Sprint:
      DRP S21b
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      In updating the isr code to use filterLabel, a bug was introduced in the filter look up, whereby the check is being made on:

      if filter in filterLabel
      

      which will always evaluate to False. Rather, this should be:

      if filter in filterLabel.physicalLabel
      

      This affects the setting of a rough zeropoint when config overrides are provided for the filters defined config.isr.fluxMag0T1, illumination correction for any filters defined in config.isr.illumFilters and a validation check for fringe correction for any filters defined in fringe.config.filters (but the fringe correction does indeed run as the check within the subtask is correct).

      For example, when running on HSC data (with a few extra debugging prints):

      $ pipetask run -b /repo/main -i HSC/defaults -o u/lauren/test -p /home/lauren/tickets/DM-28936/singleFrame.yaml -d "instrument='HSC' AND physical_filter='HSC-I' AND visit=19662 AND detector=1"
       
      filterLabel =  FilterLabel(band="y", physical="HSC-Y")
      self.fringe.config.filters =  ['HSC-Y', 'NB0921', 'NB0926', 'NB0973', 'NB1010']
      filterLabel in self.fringe.config.filters =  False
      filterLabel.physicalLabel in self.fringe.config.filters =  True
       
      roughZeroPoint: filterLabel =  FilterLabel(band="y", physical="HSC-Y")
      roughZeroPoint: self.config.fluxMag0T1 =  {'HSC-G': 398107170553.49854, 'HSC-R': 398107170553.49854, 'HSC-R2': 398107170553.49854, 'HSC-I': 275422870333.81744, 'HSC-I2': 275422870333.81744, 'HSC-Z': 120226443461.74132, 'HSC-Y': 91201083935.59116, 'NB0515': 20892961308.54041, 'NB0816': 15848931924.611174, 'NB0921': 19054607179.632523}
      roughZeroPoint: filterLabel in self.config.fluxMag0T1 =  False
      roughZeroPoint: filterLabel.physicalLabel in self.config.fluxMag0T1 =  True
       
      isr WARN: No rough magnitude zero point set for filter FilterLabel(band="y", physical="HSC-Y").
      

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            Jenkins + ci_hsc + ci_cpp_gen3 is happy.

            And I can confirm that the proper fluxMag0T1 values are being picked up in HSC processing.

            Show
            lauren Lauren MacArthur added a comment - Jenkins + ci_hsc + ci_cpp_gen3 is happy . And I can confirm that the proper fluxMag0T1 values are being picked up in HSC processing.
            Hide
            czw Christopher Waters added a comment -

            A few comments on the PR, mostly about using isrFunctions.checkFilter which handles the case of a config file having bands instead of physical filters (and does the right thing while complaining).

            Show
            czw Christopher Waters added a comment - A few comments on the PR, mostly about using isrFunctions.checkFilter which handles the case of a config file having bands instead of physical filters (and does the right thing while complaining).
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks Chris.  I rebased and reran Jenkins + ci_hsc + ci_cpp_gen3 (still happy). Out of an abundance of paranoia (and as an excuse to try out running this), I also successfully ran:

            ap_verify.py --gen2 --dataset ap_verify_ci_cosmos_pdr2 --output /home/lauren/tickets/DM-30630
            ap_verify.py --gen3 --dataset ap_verify_ci_cosmos_pdr2 --output /home/lauren/tickets/DM-30630/gen3
            

            so I think we're good. Merged and done!

            Show
            lauren Lauren MacArthur added a comment - Thanks Chris.  I rebased and reran Jenkins + ci_hsc + ci_cpp_gen3 (still happy ). Out of an abundance of paranoia (and as an excuse to try out running this), I also successfully ran: ap_verify.py - - gen2 - - dataset ap_verify_ci_cosmos_pdr2 - - output / home / lauren / tickets / DM - 30630 ap_verify.py - - gen3 - - dataset ap_verify_ci_cosmos_pdr2 - - output / home / lauren / tickets / DM - 30630 / gen3 so I think we're good. Merged and done!

              People

              Assignee:
              lauren Lauren MacArthur
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Christopher Waters
              Watchers:
              Christopher Waters, Eli Rykoff, Jim Bosch, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.