Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15119

Run pipe_analysis and validate_drp with w_2018_28 HSC RC2 outputs

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • None
    • None

    Description

      Run pipe_analysis scripts

      1. visitAnalysis
      2. coaddAnalysis
      3. colorAnalysis
      4. compareVisitAnalysis
      5. compareCoaddAnalysis

      and validate_drp

      1. matchedVisitMetrics.py
      2. validateDrp.py
      3. reportPerformance.py
      4. dispatch_verify.py

      with the w_2018_28 RC2 outputs from DM-14988

      Attachments

        Issue Links

          Activity

            The version of pipe_analysis was master at the time of the w_2018_28 release, at commit ebd48cf.
            The comparison scripts were run against RC/w_2018_26/DM-14689

            hchiang2 Hsin-Fang Chiang added a comment - The version of pipe_analysis was master at the time of the w_2018_28 release, at commit ebd48cf . The comparison scripts were run against RC/w_2018_26/ DM-14689
            hchiang2 Hsin-Fang Chiang added a comment - - edited

            For validateDrp.py, ignored filter=NB0921 of COSMOS because running validateDrp.py /datasets/hsc/repo/rerun/RC/w_2018_28/DM-14988/validateDrp/matchedVisitMetrics/9813/NB0921/matchedVisit_NB0921.json --noplot gave the following error like before (e.g. DM-14341)

            Traceback (most recent call last):
              File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/bin/validateDrp.py", line 109, in <module>
                validate.run(args.repo, **kwargs)
              File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/python/lsst/validate/drp/validate.py", line 157, in run
                print_metrics(job)
              File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/python/lsst/validate/drp/validate.py", line 482, in print_metrics
                metric = job.metrics[metric_set[0]] # Pick the first one for the description
              File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/verify/16.0+1/python/lsst/verify/metricset.py", line 207, in __getitem__
                return self._metrics[key]
            KeyError: Name('release', 'PF1')
            

            DM-15127  has been filed.

            hchiang2 Hsin-Fang Chiang added a comment - - edited For validateDrp.py , ignored filter=NB0921  of COSMOS because running validateDrp.py /datasets/hsc/repo/rerun/RC/w_2018_28/ DM-14988 /validateDrp/matchedVisitMetrics/9813/NB0921/matchedVisit_NB0921.json --noplot gave the following error like before (e.g. DM-14341 ) Traceback (most recent call last): File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/bin/validateDrp.py" , line 109 , in <module> validate.run(args.repo, **kwargs) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/python/lsst/validate/drp/validate.py" , line 157 , in run print_metrics(job) File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/validate_drp/16.0-4-g0b0e9de+6/python/lsst/validate/drp/validate.py" , line 482 , in print_metrics metric = job.metrics[metric_set[ 0 ]] # Pick the first one for the description File "/software/lsstsw/stack3_20171023/stack/miniconda3-4.3.21-10a4fa6/Linux64/verify/16.0+1/python/lsst/verify/metricset.py" , line 207 , in __getitem__ return self._metrics[key] KeyError: Name( 'release' , 'PF1' ) DM-15127   has been filed.

            compareVisitAnalysis.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2018_28/DM-14988 --rerun2 RC/w_2018_26/DM-14689 --tract=9697 --id visit=36234 failed with TaskError: No matches found.

            I guess this is related to DM-15117 the fact that w_2018_28 failed in one more CCD at visit=36234 compared to w_2018_26. So I'll just ignore this visit this time. lauren please let me know if that's not okay.

            hchiang2 Hsin-Fang Chiang added a comment - compareVisitAnalysis.py /datasets/hsc/repo --calib /datasets/hsc/repo/CALIB --rerun RC/w_2018_28/ DM-14988 --rerun2 RC/w_2018_26/ DM-14689 --tract=9697 --id visit=36234 failed with TaskError: No matches found . I guess this is related to DM-15117 the fact that w_2018_28 failed in one more CCD at visit=36234 compared to w_2018_26. So I'll just ignore this visit this time. lauren please let me know if that's not okay.

            Sounds good to me.  I will pursue this a bit further (but not in the very near future) as that particular CCD/visit does not actually overlap with tract 9697 (so doesn't get included in the plots...see CCD overlays on sky plots), so I'm not sure if it being missing in one of the repos should bring compareVisitAnalysis to a halt.  I suppose its presence could affect the meas_mosaic fitting, so perhaps I should let this situation bring it down, but perhaps try to add a more informative error message.

            lauren Lauren MacArthur added a comment - Sounds good to me.  I will pursue this a bit further (but not in the very near future) as that particular CCD/visit does not actually overlap with tract 9697 (so doesn't get included in the plots...see CCD overlays on sky plots), so I'm not sure if it being missing in one of the repos should bring compareVisitAnalysis to a halt.  I suppose its presence could affect the meas_mosaic fitting, so perhaps I should let this situation bring it down, but perhaps try to add a more informative error message.

            Indeed, it's weird that a non-overlapping CCD could affect the matches. Maybe other CCDs have changed enough as well? From the logs, the "number of sources in catalogs" are quite smaller than before.

            coaddAnalysis.py failed in the COSMOS tract=9813 filters HSC-G, HSC-R, HSC-Z, and HSC-Y with matches problem; DM-15133 has been filed.

            hchiang2 Hsin-Fang Chiang added a comment - Indeed, it's weird that a non-overlapping CCD could affect the matches. Maybe other CCDs have changed enough as well? From the logs, the "number of sources in catalogs" are quite smaller than before. coaddAnalysis.py failed in the COSMOS tract=9813 filters HSC-G, HSC-R, HSC-Z, and HSC-Y with matches problem; DM-15133 has been filed.
            lauren Lauren MacArthur added a comment - - edited

            The number of sources is indeed smaller as a result of DM-14874 (some duplication of sources was occurring because I was not selecting on "isPrimary").

             It's really puzzling that HSC-I and NB0921 worked but the other four filters didn't!

             Thanks for tracking this all...hopefully I'll get to the bottom of it soon-ish!

            lauren Lauren MacArthur added a comment - - edited The number of sources is indeed smaller as a result of DM-14874 (some duplication of sources was occurring because I was not selecting on "isPrimary").  It's really puzzling that HSC-I and NB0921 worked but the other four filters didn't!  Thanks for tracking this all...hopefully I'll get to the bottom of it soon-ish!

            pipe_analysis logs are at
            /datasets/hsc/repo/rerun/RC/w_2018_28/DM-14988/qaLogs/
            including the failed ones:

            • compareVisitAnalysis/compareVisitAnalysis-9697-VVDS_I-141058-09.out
            • coaddAnalysis/coaddAnalysis-9813-HSC-G-141104.log
            • coaddAnalysis/coaddAnalysis-9813-HSC-R-141105.log
            • coaddAnalysis/coaddAnalysis-9813-HSC-Y-141108.log
            • coaddAnalysis/coaddAnalysis-9813-HSC-Z-141107.log

            validate_drp results and logs are at
            /datasets/hsc/repo/rerun/RC/w_2018_28/DM-14988/validateDrp/

            They can also be accessed via
            https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_28/

            validate_drp results of all 6 filters in the COSMOS tract are uploaded to SQuaSH (dataset = HSC RC2) at https://squash.lsst.codes/

            hchiang2 Hsin-Fang Chiang added a comment - pipe_analysis logs are at /datasets/hsc/repo/rerun/RC/w_2018_28/ DM-14988 /qaLogs/ including the failed ones: compareVisitAnalysis/compareVisitAnalysis-9697-VVDS_I-141058-09.out coaddAnalysis/coaddAnalysis-9813-HSC-G-141104.log coaddAnalysis/coaddAnalysis-9813-HSC-R-141105.log coaddAnalysis/coaddAnalysis-9813-HSC-Y-141108.log coaddAnalysis/coaddAnalysis-9813-HSC-Z-141107.log validate_drp results and logs are at /datasets/hsc/repo/rerun/RC/w_2018_28/ DM-14988 /validateDrp/ They can also be accessed via https://lsst-web.ncsa.illinois.edu/~hchiang2/RC2_w_2018_28/ validate_drp results of all 6 filters in the COSMOS tract are uploaded to SQuaSH (dataset = HSC RC2) at  https://squash.lsst.codes/

            People

              hchiang2 Hsin-Fang Chiang
              hchiang2 Hsin-Fang Chiang
              Hsin-Fang Chiang, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.