Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17552

Check SQuaSH metric regressions using Match PessimisticB

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_astrom
    • Labels:
      None

      Description

      After the new matcher (Pessimistic B) became default on DM-14857, the astrometric metrics recorded on SQuaSH regressed (for both HSC and CFHT). Reproduce this locally and understand what happened. Fix if this is a matcher problem; if it's a validate_drp problem, ticket that for future work.

        Attachments

          Issue Links

            Activity

            No builds found.
            swinbank John Swinbank created issue -
            swinbank John Swinbank made changes -
            Field Original Value New Value
            Epic Link DM-16722 [ 235355 ]
            cmorrison Chris Morrison [X] (Inactive) made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Pessimistic matcher is performing as expected with no failures. The regression in squash values comes from differences in the matched pattern resulting in slightly different shift/rotations and therefore calculated distances. When these distances are clipped and feed to the WCS fitter the matches are then slightly different resulting in a bit more slop in the fit WCS that is apparent in the drp metrics.

            The solution found is to add a single iteration to the "final verify" step, using the first round of verified matches to refit the shift/rotation matrix resulting in a more stable set of matches regardless of the initial matched pattern.

            Below I'll post results from the 3 different validation datasets.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Pessimistic matcher is performing as expected with no failures. The regression in squash values comes from differences in the matched pattern resulting in slightly different shift/rotations and therefore calculated distances. When these distances are clipped and feed to the WCS fitter the matches are then slightly different resulting in a bit more slop in the fit WCS that is apparent in the drp metrics. The solution found is to add a single iteration to the "final verify" step, using the first round of verified matches to refit the shift/rotation matrix resulting in a more stable set of matches regardless of the initial matched pattern. Below I'll post results from the 3 different validation datasets.
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            validation_data_cfht:

            Median value of the astrometric scatter - all magnitudes: 56.896 marcsec
            Astrometric scatter (median) - snr > 100.0 : 12.9 marcsec
            No stars found that are 199.0 arcmin--201.0 arcmin apart.
            =================================================================
            r band metric measurements
            =================================================================
            validate_drp.PF1_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum fraction of magnitudes deviating by more than
            PA2_minimum_gri from mean.
                    Failed minimum      40.74 % <= 20 %
                    Failed design       40.74 % <= 10 %
                    Failed stretch      56.3 % <= 5 %
            validate_drp.PA1 - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum rms of the unresolved source magnitude
            distribution around the mean value.
                    Failed design       16.99 mmag <= 5 mmag
                    Failed minimum      16.99 mmag <= 8 mmag
                    Failed stretch      16.99 mmag <= 3 mmag
            validate_drp.PA2_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            At most PF1_minimum_gri % of magnitudes may deviate by more
            than this magnitude from the mean.
                    Failed minimum      24.44 mmag <= 15 mmag
                    Failed design       35.4 mmag <= 15 mmag
                    Failed stretch      50.32 mmag <= 10 mmag
            validate_drp.AD1_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF1 of the astrometric distances will deviate by
            more than this from the median (see AM1, AF1).
                    Passed design       14.61 marcsec <= 20 marcsec
                    Passed minimum      8.062 marcsec <= 40 marcsec
                    Failed stretch      20.7 marcsec <= 10 marcsec
            validate_drp.AD2_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF2 of the astrometric distances will deviate by
            more than this from the median (see AM2, AF2).
                    Passed design       16.16 marcsec <= 20 marcsec
                    Passed minimum      9.359 marcsec <= 40 marcsec
                    Failed stretch      22.36 marcsec <= 10 marcsec
            validate_drp.AD3_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF3 of the astrometric distances will deviate by
            more than this from the median (see AM3, AF3).
                    Skipped AD3.design   no measurement
                    Skipped AD3.minimum  no measurement
                    Skipped AD3.stretch  no measurement
            validate_drp.AF1_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD1 milliarcsec (see AM1).
                    Passed design       5.315 % <= 10 %
                    Passed minimum      0.991 % <= 20 %
                    Failed stretch      15.81 % <= 5 %
            validate_drp.AF2_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD2 milliarcsec (see AM2).
                    Passed design       6.658 % <= 10 %
                    Passed minimum      1.198 % <= 20 %
                    Failed stretch      18.84 % <= 5 %
            validate_drp.AF3_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD3 milliarcsec (see AM3).
                    Skipped AF3.design   no measurement
                    Skipped AF3.minimum  no measurement
                    Skipped AF3.stretch  no measurement
            validate_drp.AM1 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=5 arcmin (repeatability).
                    Passed design       7.914 marcsec <= 10 marcsec
                    Passed minimum      7.914 marcsec <= 20 marcsec
                    Failed stretch      7.914 marcsec <= 5 marcsec
            validate_drp.AM2 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=20 arcmin (repeatability).
                    Passed design       8.937 marcsec <= 10 marcsec
                    Passed minimum      8.937 marcsec <= 20 marcsec
                    Failed stretch      8.937 marcsec <= 5 marcsec
            validate_drp.AM3 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=200 arcmin (repeatability).
                    Skipped AM3.design   no measurement
                    Skipped AM3.minimum  no measurement
                    Skipped AM3.stretch  no measurement
            validate_drp.TE1 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations < 1 arcmin.
                    Failed design       0.0001075 <= 2e-05
                    Failed minimum      0.0001075 <= 3e-05
                    Failed stretch      0.0001075 <= 1e-05
            validate_drp.TE2 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations > 5 arcmin.
                    Passed design       9.242e-06 <= 5e-05
                    Passed minimum      9.242e-06 <= 3e-05
                    Passed stretch      9.242e-06 <= 1e-05
            Wrote plot: Cfht_output_r_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png
            Wrote plot: Cfht_output_r_validate_drp.AM2_D_20_arcmin_17.0_21.5_mag.png
            Skipping validate_drp.AM3 -- no measurement
            =================================================================
            r band summary
            =================================================================
            Failed minimum      8 of 23 failed
            Failed design       12 of 23 failed
            Failed stretch      18 of 23 failed
            ==================================================================================================================================
            design level summary
            =================================================================
            FAILED (12/23 measurements)
            =================================================================
            

            validation_data_decam:

            Median value of the astrometric scatter - all magnitudes: 53.643 marcsec
            Astrometric scatter (median) - snr > 100.0 : 11.9 marcsec
            No stars found that are 199.0 arcmin--201.0 arcmin apart.
            =================================================================
            z band metric measurements
            =================================================================
            validate_drp.PF1_minimum_uzy - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum fraction of magnitudes deviating by more than
            PA2_minimum_uzy from mean.
                    Failed minimum      69.55 % <= 20 %
                    Failed design       69.55 % <= 10 %
                    Failed stretch      81.68 % <= 5 %
            validate_drp.PA1 - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum rms of the unresolved source magnitude 
            distribution around the mean value.
                    Failed design       52.44 mmag <= 7.5 mmag
                    Failed minimum      52.44 mmag <= 12 mmag
                    Failed stretch      52.44 mmag <= 4.5 mmag
            validate_drp.PA2_minimum_uzy - LPM-17, p. 21, http://ls.st/lpm-17
            At most PF1_minimum_uzy % of magnitudes may deviate by more
            than this magnitude from the mean.
                    Failed minimum      49.74 mmag <= 22.5 mmag
                    Failed design       57.75 mmag <= 22.5 mmag
                    Failed stretch      73.76 mmag <= 15 mmag
            validate_drp.AD1_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF1 of the astrometric distances will deviate by
            more than this from the median (see AM1, AF1).
                    Passed design       16.62 marcsec <= 20 marcsec
                    Passed minimum      9.969 marcsec <= 40 marcsec
                    Failed stretch      22.28 marcsec <= 10 marcsec
            validate_drp.AD2_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF2 of the astrometric distances will deviate by
            more than this from the median (see AM2, AF2).
                    Passed design       17.09 marcsec <= 20 marcsec
                    Passed minimum      10.41 marcsec <= 40 marcsec
                    Failed stretch      24.44 marcsec <= 10 marcsec
            validate_drp.AD3_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF3 of the astrometric distances will deviate by
            more than this from the median (see AM3, AF3).
                    Skipped AD3.design   no measurement
                    Skipped AD3.minimum  no measurement
                    Skipped AD3.stretch  no measurement
            validate_drp.AF1_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD1 milliarcsec (see AM1).
                    Passed design       6.439 % <= 10 %
                    Passed minimum      0.5036 % <= 20 %
                    Failed stretch      19.86 % <= 5 %
            validate_drp.AF2_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD2 milliarcsec (see AM2).
                    Passed design       7.503 % <= 10 %
                    Passed minimum      0.8336 % <= 20 %
                    Failed stretch      21.04 % <= 5 %
            validate_drp.AF3_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD3 milliarcsec (see AM3).
                    Skipped AF3.design   no measurement
                    Skipped AF3.minimum  no measurement
                    Skipped AF3.stretch  no measurement
            validate_drp.AM1 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=5 arcmin (repeatability).
                    Passed design       9.493 marcsec <= 10 marcsec
                    Passed minimum      9.493 marcsec <= 20 marcsec
                    Failed stretch      9.493 marcsec <= 5 marcsec
            validate_drp.AM2 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=20 arcmin (repeatability).
                    Failed design       10.28 marcsec <= 10 marcsec
                    Passed minimum      10.28 marcsec <= 20 marcsec
                    Failed stretch      10.28 marcsec <= 5 marcsec
            validate_drp.AM3 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=200 arcmin (repeatability).
                    Skipped AM3.design   no measurement
                    Skipped AM3.minimum  no measurement
                    Skipped AM3.stretch  no measurement
            validate_drp.TE1 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations < 1 arcmin.
                    Failed design       3.507e-05 <= 2e-05
                    Failed minimum      3.507e-05 <= 3e-05
                    Failed stretch      3.507e-05 <= 1e-05
            validate_drp.TE2 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations > 5 arcmin.
                    Passed design       4.878e-06 <= 5e-05
                    Passed minimum      4.878e-06 <= 3e-05
                    Passed stretch      4.878e-06 <= 1e-05
            Wrote plot: Decam_output_z_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png
            Wrote plot: Decam_output_z_validate_drp.AM2_D_20_arcmin_17.0_21.5_mag.png
            Skipping validate_drp.AM3 -- no measurement
            =================================================================
            z band summary
            =================================================================
            Failed minimum      9 of 23 failed
            Failed design       13 of 23 failed
            Failed stretch      19 of 23 failed
            ==================================================================================================================================
            design level summary
            =================================================================
            FAILED (13/23 measurements)
            =================================================================

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - validation_data_cfht: Median value of the astrometric scatter - all magnitudes: 56.896 marcsec Astrometric scatter (median) - snr > 100.0 : 12.9 marcsec No stars found that are 199.0 arcmin-- 201.0 arcmin apart. ================================================================= r band metric measurements ================================================================= validate_drp.PF1_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum fraction of magnitudes deviating by more than PA2_minimum_gri from mean. Failed minimum 40.74 % <= 20 % Failed design 40.74 % <= 10 % Failed stretch 56.3 % <= 5 % validate_drp.PA1 - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum rms of the unresolved source magnitude distribution around the mean value. Failed design 16.99 mmag <= 5 mmag Failed minimum 16.99 mmag <= 8 mmag Failed stretch 16.99 mmag <= 3 mmag validate_drp.PA2_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 At most PF1_minimum_gri % of magnitudes may deviate by more than this magnitude from the mean. Failed minimum 24.44 mmag <= 15 mmag Failed design 35.4 mmag <= 15 mmag Failed stretch 50.32 mmag <= 10 mmag validate_drp.AD1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF1 of the astrometric distances will deviate by more than this from the median (see AM1, AF1). Passed design 14.61 marcsec <= 20 marcsec Passed minimum 8.062 marcsec <= 40 marcsec Failed stretch 20.7 marcsec <= 10 marcsec validate_drp.AD2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF2 of the astrometric distances will deviate by more than this from the median (see AM2, AF2). Passed design 16.16 marcsec <= 20 marcsec Passed minimum 9.359 marcsec <= 40 marcsec Failed stretch 22.36 marcsec <= 10 marcsec validate_drp.AD3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF3 of the astrometric distances will deviate by more than this from the median (see AM3, AF3). Skipped AD3.design no measurement Skipped AD3.minimum no measurement Skipped AD3.stretch no measurement validate_drp.AF1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD1 milliarcsec (see AM1). Passed design 5.315 % <= 10 % Passed minimum 0.991 % <= 20 % Failed stretch 15.81 % <= 5 % validate_drp.AF2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD2 milliarcsec (see AM2). Passed design 6.658 % <= 10 % Passed minimum 1.198 % <= 20 % Failed stretch 18.84 % <= 5 % validate_drp.AF3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD3 milliarcsec (see AM3). Skipped AF3.design no measurement Skipped AF3.minimum no measurement Skipped AF3.stretch no measurement validate_drp.AM1 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 5 arcmin (repeatability). Passed design 7.914 marcsec <= 10 marcsec Passed minimum 7.914 marcsec <= 20 marcsec Failed stretch 7.914 marcsec <= 5 marcsec validate_drp.AM2 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 20 arcmin (repeatability). Passed design 8.937 marcsec <= 10 marcsec Passed minimum 8.937 marcsec <= 20 marcsec Failed stretch 8.937 marcsec <= 5 marcsec validate_drp.AM3 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 200 arcmin (repeatability). Skipped AM3.design no measurement Skipped AM3.minimum no measurement Skipped AM3.stretch no measurement validate_drp.TE1 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations < 1 arcmin. Failed design 0.0001075 <= 2e- 05 Failed minimum 0.0001075 <= 3e- 05 Failed stretch 0.0001075 <= 1e- 05 validate_drp.TE2 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations > 5 arcmin. Passed design 9 .242e- 06 <= 5e- 05 Passed minimum 9 .242e- 06 <= 3e- 05 Passed stretch 9 .242e- 06 <= 1e- 05 Wrote plot: Cfht_output_r_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png Wrote plot: Cfht_output_r_validate_drp.AM2_D_20_arcmin_17.0_21.5_mag.png Skipping validate_drp.AM3 -- no measurement ================================================================= r band summary ================================================================= Failed minimum 8 of 23 failed Failed design 12 of 23 failed Failed stretch 18 of 23 failed ================================================================================================================================== design level summary ================================================================= FAILED ( 12 / 23 measurements) ================================================================= validation_data_decam: Median value of the astrometric scatter - all magnitudes: 53.643 marcsec Astrometric scatter (median) - snr > 100.0 : 11.9 marcsec No stars found that are 199.0 arcmin-- 201.0 arcmin apart. ================================================================= z band metric measurements ================================================================= validate_drp.PF1_minimum_uzy - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum fraction of magnitudes deviating by more than PA2_minimum_uzy from mean. Failed minimum 69.55 % <= 20 % Failed design 69.55 % <= 10 % Failed stretch 81.68 % <= 5 % validate_drp.PA1 - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum rms of the unresolved source magnitude distribution around the mean value. Failed design 52.44 mmag <= 7.5 mmag Failed minimum 52.44 mmag <= 12 mmag Failed stretch 52.44 mmag <= 4.5 mmag validate_drp.PA2_minimum_uzy - LPM- 17 , p. 21 , http: //ls.st/lpm-17 At most PF1_minimum_uzy % of magnitudes may deviate by more than this magnitude from the mean. Failed minimum 49.74 mmag <= 22.5 mmag Failed design 57.75 mmag <= 22.5 mmag Failed stretch 73.76 mmag <= 15 mmag validate_drp.AD1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF1 of the astrometric distances will deviate by more than this from the median (see AM1, AF1). Passed design 16.62 marcsec <= 20 marcsec Passed minimum 9.969 marcsec <= 40 marcsec Failed stretch 22.28 marcsec <= 10 marcsec validate_drp.AD2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF2 of the astrometric distances will deviate by more than this from the median (see AM2, AF2). Passed design 17.09 marcsec <= 20 marcsec Passed minimum 10.41 marcsec <= 40 marcsec Failed stretch 24.44 marcsec <= 10 marcsec validate_drp.AD3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF3 of the astrometric distances will deviate by more than this from the median (see AM3, AF3). Skipped AD3.design no measurement Skipped AD3.minimum no measurement Skipped AD3.stretch no measurement validate_drp.AF1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD1 milliarcsec (see AM1). Passed design 6.439 % <= 10 % Passed minimum 0.5036 % <= 20 % Failed stretch 19.86 % <= 5 % validate_drp.AF2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD2 milliarcsec (see AM2). Passed design 7.503 % <= 10 % Passed minimum 0.8336 % <= 20 % Failed stretch 21.04 % <= 5 % validate_drp.AF3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD3 milliarcsec (see AM3). Skipped AF3.design no measurement Skipped AF3.minimum no measurement Skipped AF3.stretch no measurement validate_drp.AM1 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 5 arcmin (repeatability). Passed design 9.493 marcsec <= 10 marcsec Passed minimum 9.493 marcsec <= 20 marcsec Failed stretch 9.493 marcsec <= 5 marcsec validate_drp.AM2 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 20 arcmin (repeatability). Failed design 10.28 marcsec <= 10 marcsec Passed minimum 10.28 marcsec <= 20 marcsec Failed stretch 10.28 marcsec <= 5 marcsec validate_drp.AM3 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 200 arcmin (repeatability). Skipped AM3.design no measurement Skipped AM3.minimum no measurement Skipped AM3.stretch no measurement validate_drp.TE1 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations < 1 arcmin. Failed design 3 .507e- 05 <= 2e- 05 Failed minimum 3 .507e- 05 <= 3e- 05 Failed stretch 3 .507e- 05 <= 1e- 05 validate_drp.TE2 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations > 5 arcmin. Passed design 4 .878e- 06 <= 5e- 05 Passed minimum 4 .878e- 06 <= 3e- 05 Passed stretch 4 .878e- 06 <= 1e- 05 Wrote plot: Decam_output_z_validate_drp.AM1_D_5_arcmin_17.0_21.5_mag.png Wrote plot: Decam_output_z_validate_drp.AM2_D_20_arcmin_17.0_21.5_mag.png Skipping validate_drp.AM3 -- no measurement ================================================================= z band summary ================================================================= Failed minimum 9 of 23 failed Failed design 13 of 23 failed Failed stretch 19 of 23 failed ================================================================================================================================== design level summary ================================================================= FAILED ( 13 / 23 measurements) =================================================================
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment - - edited

            validation_data_hsc:

            =================================================================
            HSC-R band metric measurements
            =================================================================
            validate_drp.PF1_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum fraction of magnitudes deviating by more than
            PA2_minimum_gri from mean.
                    Failed minimum      30.9 % <= 20 %
                    Failed design       30.9 % <= 10 %
                    Failed stretch      48.38 % <= 5 %
            validate_drp.PA1 - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum rms of the unresolved source magnitude
            distribution around the mean value.
                    Failed design       14.29 mmag <= 5 mmag
                    Failed minimum      14.29 mmag <= 8 mmag
                    Failed stretch      14.29 mmag <= 3 mmag
            validate_drp.PA2_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            At most PF1_minimum_gri % of magnitudes may deviate by more
            than this magnitude from the mean.
                    Failed minimum      19.6 mmag <= 15 mmag
                    Failed design       27.21 mmag <= 15 mmag
                    Failed stretch      34.46 mmag <= 10 mmag
            validate_drp.AD1_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF1 of the astrometric distances will deviate by
            more than this from the median (see AM1, AF1).
                    Passed design       5.769 marcsec <= 20 marcsec
                    Passed minimum      3.499 marcsec <= 40 marcsec
                    Passed stretch      7.856 marcsec <= 10 marcsec
            validate_drp.AD2_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF2 of the astrometric distances will deviate by
            more than this from the median (see AM2, AF2).
                    Passed design       5.941 marcsec <= 20 marcsec
                    Passed minimum      3.591 marcsec <= 40 marcsec
                    Passed stretch      8.128 marcsec <= 10 marcsec
            validate_drp.AD3_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF3 of the astrometric distances will deviate by
            more than this from the median (see AM3, AF3).
                    Skipped AD3.design   no measurement
                    Skipped AD3.minimum  no measurement
                    Skipped AD3.stretch  no measurement
            validate_drp.AF1_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD1 milliarcsec (see AM1).
                    Passed design       0.1762 % <= 10 %
                    Passed minimum      0.1112 % <= 20 %
                    Passed stretch      2.443 % <= 5 %
            validate_drp.AF2_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD2 milliarcsec (see AM2).
                    Passed design       0.1611 % <= 10 %
                    Passed minimum      0.09087 % <= 20 %
                    Passed stretch      2.672 % <= 5 %
            validate_drp.AF3_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD3 milliarcsec (see AM3).
                    Skipped AF3.design   no measurement
                    Skipped AF3.minimum  no measurement
                    Skipped AF3.stretch  no measurement
            validate_drp.AM1 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=5 arcmin (repeatability).
                    Passed design       3.646 marcsec <= 10 marcsec
                    Passed minimum      3.646 marcsec <= 20 marcsec
                    Passed stretch      3.646 marcsec <= 5 marcsec
            validate_drp.AM2 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=20 arcmin (repeatability).
                    Passed design       3.744 marcsec <= 10 marcsec
                    Passed minimum      3.744 marcsec <= 20 marcsec
                    Passed stretch      3.744 marcsec <= 5 marcsec
            validate_drp.AM3 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=200 arcmin (repeatability).
                    Skipped AM3.design   no measurement
                    Skipped AM3.minimum  no measurement
                    Skipped AM3.stretch  no measurement
            validate_drp.TE1 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations < 1 arcmin.
                    Passed design       1.261e-05 <= 2e-05
                    Passed minimum      1.261e-05 <= 3e-05
                    Failed stretch      1.261e-05 <= 1e-05
            validate_drp.TE2 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations > 5 arcmin.
                    Passed design       3.482e-07 <= 5e-05
                    Passed minimum      3.482e-07 <= 3e-05
                    Passed stretch      3.482e-07 <= 1e-05
            =================================================================
            HSC-I band metric measurements
            =================================================================
            validate_drp.PF1_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum fraction of magnitudes deviating by more than
            PA2_minimum_gri from mean.
                    Failed minimum      27.07 % <= 20 %
                    Failed design       27.07 % <= 10 %
                    Failed stretch      42.77 % <= 5 %
            validate_drp.PA1 - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum rms of the unresolved source magnitude
            distribution around the mean value.
                    Failed design       12.18 mmag <= 5 mmag
                    Failed minimum      12.18 mmag <= 8 mmag
                    Failed stretch      12.18 mmag <= 3 mmag
            validate_drp.PA2_minimum_gri - LPM-17, p. 21, http://ls.st/lpm-17
            At most PF1_minimum_gri % of magnitudes may deviate by more
            than this magnitude from the mean.
                    Failed minimum      18.25 mmag <= 15 mmag
                    Failed design       25.56 mmag <= 15 mmag
                    Failed stretch      33.33 mmag <= 10 mmag
            validate_drp.AD1_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF1 of the astrometric distances will deviate by
            more than this from the median (see AM1, AF1).
                    Passed design       8.35 marcsec <= 20 marcsec
                    Passed minimum      5.101 marcsec <= 40 marcsec
                    Failed stretch      11.36 marcsec <= 10 marcsec
            validate_drp.AD2_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF2 of the astrometric distances will deviate by
            more than this from the median (see AM2, AF2).
                    Passed design       8.169 marcsec <= 20 marcsec
                    Passed minimum      4.976 marcsec <= 40 marcsec
                    Failed stretch      11.15 marcsec <= 10 marcsec
            validate_drp.AD3_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF3 of the astrometric distances will deviate by
            more than this from the median (see AM3, AF3).
                    Skipped AD3.design   no measurement
                    Skipped AD3.minimum  no measurement
                    Skipped AD3.stretch  no measurement
            validate_drp.AF1_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD1 milliarcsec (see AM1).
                    Passed design       0.5963 % <= 10 %
                    Passed minimum      0.01295 % <= 20 %
                    Failed stretch      6.894 % <= 5 %
            validate_drp.AF2_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD2 milliarcsec (see AM2).
                    Passed design       0.6144 % <= 10 %
                    Passed minimum      0.01471 % <= 20 %
                    Failed stretch      6.57 % <= 5 %
            validate_drp.AF3_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD3 milliarcsec (see AM3).
                    Skipped AF3.design   no measurement
                    Skipped AF3.minimum  no measurement
                    Skipped AF3.stretch  no measurement
            validate_drp.AM1 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=5 arcmin (repeatability).
                    Passed design       7.096 marcsec <= 10 marcsec
                    Passed minimum      7.096 marcsec <= 20 marcsec
                    Failed stretch      7.096 marcsec <= 5 marcsec
            validate_drp.AM2 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=20 arcmin (repeatability).
                    Passed design       7.366 marcsec <= 10 marcsec
                    Passed minimum      7.366 marcsec <= 20 marcsec
                    Failed stretch      7.366 marcsec <= 5 marcsec
            validate_drp.AM3 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=200 arcmin (repeatability).
                    Skipped AM3.design   no measurement
                    Skipped AM3.minimum  no measurement
                    Skipped AM3.stretch  no measurement
            validate_drp.TE1 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations < 1 arcmin.
                    Passed design       1.787e-06 <= 2e-05
                    Passed minimum      1.787e-06 <= 3e-05
                    Passed stretch      1.787e-06 <= 1e-05
            validate_drp.TE2 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations > 5 arcmin.
                    Passed design       5.421e-07 <= 5e-05
                    Passed minimum      5.421e-07 <= 3e-05
                    Passed stretch      5.421e-07 <= 1e-05
            =================================================================
            HSC-Y band metric measurements
            =================================================================
            validate_drp.PF1_design_uzy - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum fraction of magnitudes deviating by more than
            PA2_design_uzy from mean.
                    Failed design       38.08 % <= 10 %
                    Failed minimum      38.08 % <= 20 %
                    Failed stretch      55.54 % <= 5 %
            validate_drp.PA1 - LPM-17, p. 21, http://ls.st/lpm-17
            The maximum rms of the unresolved source magnitude 
            distribution around the mean value.
                    Failed design       25.35 mmag <= 7.5 mmag
                    Failed minimum      25.35 mmag <= 12 mmag
                    Failed stretch      25.35 mmag <= 4.5 mmag
            validate_drp.PA2_design_uzy - LPM-17, p. 21, http://ls.st/lpm-17
            At most PF1_design_uzy % of magnitudes may deviate by more
            than this magnitude from the mean.
                    Failed design       45.11 mmag <= 22.5 mmag
                    Failed minimum      33.91 mmag <= 22.5 mmag
                    Failed stretch      54.62 mmag <= 15 mmag
            validate_drp.AD1_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF1 of the astrometric distances will deviate by
            more than this from the median (see AM1, AF1).
                    Passed design       8.938 marcsec <= 20 marcsec
                    Passed minimum      5.556 marcsec <= 40 marcsec
                    Failed stretch      12.12 marcsec <= 10 marcsec
            validate_drp.AD2_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF2 of the astrometric distances will deviate by
            more than this from the median (see AM2, AF2).
                    Passed design       9.81 marcsec <= 20 marcsec
                    Passed minimum      6.051 marcsec <= 40 marcsec
                    Failed stretch      13.33 marcsec <= 10 marcsec
            validate_drp.AD3_design - LPM-17, p. 23, http://ls.st/lpm-17
            No more than AF3 of the astrometric distances will deviate by
            more than this from the median (see AM3, AF3).
                    Skipped AD3.design   no measurement
                    Skipped AD3.minimum  no measurement
                    Skipped AD3.stretch  no measurement
            validate_drp.AF1_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD1 milliarcsec (see AM1).
                    Passed design       0.8148 % <= 10 %
                    Passed minimum      0.1267 % <= 20 %
                    Failed stretch      7.993 % <= 5 %
            validate_drp.AF2_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD2 milliarcsec (see AM2).
                    Passed design       1.175 % <= 10 %
                    Passed minimum      0.1828 % <= 20 %
                    Failed stretch      9.672 % <= 5 %
            validate_drp.AF3_design - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum fraction of astrometric distances which deviate
            by more than AD3 milliarcsec (see AM3).
                    Skipped AF3.design   no measurement
                    Skipped AF3.minimum  no measurement
                    Skipped AF3.stretch  no measurement
            validate_drp.AM1 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=5 arcmin (repeatability).
                    Passed design       5.823 marcsec <= 10 marcsec
                    Passed minimum      5.823 marcsec <= 20 marcsec
                    Failed stretch      5.823 marcsec <= 5 marcsec
            validate_drp.AM2 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=20 arcmin (repeatability).
                    Passed design       6.481 marcsec <= 10 marcsec
                    Passed minimum      6.481 marcsec <= 20 marcsec
                    Failed stretch      6.481 marcsec <= 5 marcsec
            validate_drp.AM3 - LPM-17, p. 23, http://ls.st/lpm-17
            The maximum rms of the astrometric distance distribution for
            stellar pairs with separations of D=200 arcmin (repeatability).
                    Skipped AM3.design   no measurement
                    Skipped AM3.minimum  no measurement
                    Skipped AM3.stretch  no measurement
            validate_drp.TE1 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations < 1 arcmin.
                    Failed design       0.0002384 <= 2e-05
                    Failed minimum      0.0002384 <= 3e-05
                    Failed stretch      0.0002384 <= 1e-05
            validate_drp.TE2 - LPM-17, p. 31, http://ls.st/lpm-17
            The averaged E1, E2, and Ex residual PSF ellipticity
            correlations must have median absolute value less than this for
            separations > 5 arcmin.
                    Passed design       2.357e-06 <= 5e-05
                    Passed minimum      2.357e-06 <= 3e-05
                    Passed stretch      2.357e-06 <= 1e-05
            =================================================================
            HSC-R band summary
            =================================================================
            Failed minimum      7 of 23 failed
            Failed design       7 of 23 failed
            Failed stretch      8 of 23 failed
            =================================================================
            =================================================================
            HSC-I band summary
            =================================================================
            Failed minimum      7 of 23 failed
            Failed design       7 of 23 failed
            Failed stretch      13 of 23 failed
            =================================================================
            =================================================================
            HSC-Y band summary
            =================================================================
            Failed minimum      8 of 23 failed
            Failed design       8 of 23 failed
            Failed stretch      14 of 23 failed
            ==================================================================================================================================
            design level summary
            =================================================================
            FAILED (22/69 measurements)
            =================================================================

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - - edited validation_data_hsc: ================================================================= HSC-R band metric measurements ================================================================= validate_drp.PF1_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum fraction of magnitudes deviating by more than PA2_minimum_gri from mean. Failed minimum 30.9 % <= 20 % Failed design 30.9 % <= 10 % Failed stretch 48.38 % <= 5 % validate_drp.PA1 - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum rms of the unresolved source magnitude distribution around the mean value. Failed design 14.29 mmag <= 5 mmag Failed minimum 14.29 mmag <= 8 mmag Failed stretch 14.29 mmag <= 3 mmag validate_drp.PA2_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 At most PF1_minimum_gri % of magnitudes may deviate by more than this magnitude from the mean. Failed minimum 19.6 mmag <= 15 mmag Failed design 27.21 mmag <= 15 mmag Failed stretch 34.46 mmag <= 10 mmag validate_drp.AD1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF1 of the astrometric distances will deviate by more than this from the median (see AM1, AF1). Passed design 5.769 marcsec <= 20 marcsec Passed minimum 3.499 marcsec <= 40 marcsec Passed stretch 7.856 marcsec <= 10 marcsec validate_drp.AD2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF2 of the astrometric distances will deviate by more than this from the median (see AM2, AF2). Passed design 5.941 marcsec <= 20 marcsec Passed minimum 3.591 marcsec <= 40 marcsec Passed stretch 8.128 marcsec <= 10 marcsec validate_drp.AD3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF3 of the astrometric distances will deviate by more than this from the median (see AM3, AF3). Skipped AD3.design no measurement Skipped AD3.minimum no measurement Skipped AD3.stretch no measurement validate_drp.AF1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD1 milliarcsec (see AM1). Passed design 0.1762 % <= 10 % Passed minimum 0.1112 % <= 20 % Passed stretch 2.443 % <= 5 % validate_drp.AF2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD2 milliarcsec (see AM2). Passed design 0.1611 % <= 10 % Passed minimum 0.09087 % <= 20 % Passed stretch 2.672 % <= 5 % validate_drp.AF3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD3 milliarcsec (see AM3). Skipped AF3.design no measurement Skipped AF3.minimum no measurement Skipped AF3.stretch no measurement validate_drp.AM1 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 5 arcmin (repeatability). Passed design 3.646 marcsec <= 10 marcsec Passed minimum 3.646 marcsec <= 20 marcsec Passed stretch 3.646 marcsec <= 5 marcsec validate_drp.AM2 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 20 arcmin (repeatability). Passed design 3.744 marcsec <= 10 marcsec Passed minimum 3.744 marcsec <= 20 marcsec Passed stretch 3.744 marcsec <= 5 marcsec validate_drp.AM3 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 200 arcmin (repeatability). Skipped AM3.design no measurement Skipped AM3.minimum no measurement Skipped AM3.stretch no measurement validate_drp.TE1 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations < 1 arcmin. Passed design 1 .261e- 05 <= 2e- 05 Passed minimum 1 .261e- 05 <= 3e- 05 Failed stretch 1 .261e- 05 <= 1e- 05 validate_drp.TE2 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations > 5 arcmin. Passed design 3 .482e- 07 <= 5e- 05 Passed minimum 3 .482e- 07 <= 3e- 05 Passed stretch 3 .482e- 07 <= 1e- 05 ================================================================= HSC-I band metric measurements ================================================================= validate_drp.PF1_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum fraction of magnitudes deviating by more than PA2_minimum_gri from mean. Failed minimum 27.07 % <= 20 % Failed design 27.07 % <= 10 % Failed stretch 42.77 % <= 5 % validate_drp.PA1 - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum rms of the unresolved source magnitude distribution around the mean value. Failed design 12.18 mmag <= 5 mmag Failed minimum 12.18 mmag <= 8 mmag Failed stretch 12.18 mmag <= 3 mmag validate_drp.PA2_minimum_gri - LPM- 17 , p. 21 , http: //ls.st/lpm-17 At most PF1_minimum_gri % of magnitudes may deviate by more than this magnitude from the mean. Failed minimum 18.25 mmag <= 15 mmag Failed design 25.56 mmag <= 15 mmag Failed stretch 33.33 mmag <= 10 mmag validate_drp.AD1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF1 of the astrometric distances will deviate by more than this from the median (see AM1, AF1). Passed design 8.35 marcsec <= 20 marcsec Passed minimum 5.101 marcsec <= 40 marcsec Failed stretch 11.36 marcsec <= 10 marcsec validate_drp.AD2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF2 of the astrometric distances will deviate by more than this from the median (see AM2, AF2). Passed design 8.169 marcsec <= 20 marcsec Passed minimum 4.976 marcsec <= 40 marcsec Failed stretch 11.15 marcsec <= 10 marcsec validate_drp.AD3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF3 of the astrometric distances will deviate by more than this from the median (see AM3, AF3). Skipped AD3.design no measurement Skipped AD3.minimum no measurement Skipped AD3.stretch no measurement validate_drp.AF1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD1 milliarcsec (see AM1). Passed design 0.5963 % <= 10 % Passed minimum 0.01295 % <= 20 % Failed stretch 6.894 % <= 5 % validate_drp.AF2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD2 milliarcsec (see AM2). Passed design 0.6144 % <= 10 % Passed minimum 0.01471 % <= 20 % Failed stretch 6.57 % <= 5 % validate_drp.AF3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD3 milliarcsec (see AM3). Skipped AF3.design no measurement Skipped AF3.minimum no measurement Skipped AF3.stretch no measurement validate_drp.AM1 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 5 arcmin (repeatability). Passed design 7.096 marcsec <= 10 marcsec Passed minimum 7.096 marcsec <= 20 marcsec Failed stretch 7.096 marcsec <= 5 marcsec validate_drp.AM2 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 20 arcmin (repeatability). Passed design 7.366 marcsec <= 10 marcsec Passed minimum 7.366 marcsec <= 20 marcsec Failed stretch 7.366 marcsec <= 5 marcsec validate_drp.AM3 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 200 arcmin (repeatability). Skipped AM3.design no measurement Skipped AM3.minimum no measurement Skipped AM3.stretch no measurement validate_drp.TE1 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations < 1 arcmin. Passed design 1 .787e- 06 <= 2e- 05 Passed minimum 1 .787e- 06 <= 3e- 05 Passed stretch 1 .787e- 06 <= 1e- 05 validate_drp.TE2 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations > 5 arcmin. Passed design 5 .421e- 07 <= 5e- 05 Passed minimum 5 .421e- 07 <= 3e- 05 Passed stretch 5 .421e- 07 <= 1e- 05 ================================================================= HSC-Y band metric measurements ================================================================= validate_drp.PF1_design_uzy - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum fraction of magnitudes deviating by more than PA2_design_uzy from mean. Failed design 38.08 % <= 10 % Failed minimum 38.08 % <= 20 % Failed stretch 55.54 % <= 5 % validate_drp.PA1 - LPM- 17 , p. 21 , http: //ls.st/lpm-17 The maximum rms of the unresolved source magnitude distribution around the mean value. Failed design 25.35 mmag <= 7.5 mmag Failed minimum 25.35 mmag <= 12 mmag Failed stretch 25.35 mmag <= 4.5 mmag validate_drp.PA2_design_uzy - LPM- 17 , p. 21 , http: //ls.st/lpm-17 At most PF1_design_uzy % of magnitudes may deviate by more than this magnitude from the mean. Failed design 45.11 mmag <= 22.5 mmag Failed minimum 33.91 mmag <= 22.5 mmag Failed stretch 54.62 mmag <= 15 mmag validate_drp.AD1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF1 of the astrometric distances will deviate by more than this from the median (see AM1, AF1). Passed design 8.938 marcsec <= 20 marcsec Passed minimum 5.556 marcsec <= 40 marcsec Failed stretch 12.12 marcsec <= 10 marcsec validate_drp.AD2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF2 of the astrometric distances will deviate by more than this from the median (see AM2, AF2). Passed design 9.81 marcsec <= 20 marcsec Passed minimum 6.051 marcsec <= 40 marcsec Failed stretch 13.33 marcsec <= 10 marcsec validate_drp.AD3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 No more than AF3 of the astrometric distances will deviate by more than this from the median (see AM3, AF3). Skipped AD3.design no measurement Skipped AD3.minimum no measurement Skipped AD3.stretch no measurement validate_drp.AF1_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD1 milliarcsec (see AM1). Passed design 0.8148 % <= 10 % Passed minimum 0.1267 % <= 20 % Failed stretch 7.993 % <= 5 % validate_drp.AF2_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD2 milliarcsec (see AM2). Passed design 1.175 % <= 10 % Passed minimum 0.1828 % <= 20 % Failed stretch 9.672 % <= 5 % validate_drp.AF3_design - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum fraction of astrometric distances which deviate by more than AD3 milliarcsec (see AM3). Skipped AF3.design no measurement Skipped AF3.minimum no measurement Skipped AF3.stretch no measurement validate_drp.AM1 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 5 arcmin (repeatability). Passed design 5.823 marcsec <= 10 marcsec Passed minimum 5.823 marcsec <= 20 marcsec Failed stretch 5.823 marcsec <= 5 marcsec validate_drp.AM2 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 20 arcmin (repeatability). Passed design 6.481 marcsec <= 10 marcsec Passed minimum 6.481 marcsec <= 20 marcsec Failed stretch 6.481 marcsec <= 5 marcsec validate_drp.AM3 - LPM- 17 , p. 23 , http: //ls.st/lpm-17 The maximum rms of the astrometric distance distribution for stellar pairs with separations of D= 200 arcmin (repeatability). Skipped AM3.design no measurement Skipped AM3.minimum no measurement Skipped AM3.stretch no measurement validate_drp.TE1 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations < 1 arcmin. Failed design 0.0002384 <= 2e- 05 Failed minimum 0.0002384 <= 3e- 05 Failed stretch 0.0002384 <= 1e- 05 validate_drp.TE2 - LPM- 17 , p. 31 , http: //ls.st/lpm-17 The averaged E1, E2, and Ex residual PSF ellipticity correlations must have median absolute value less than this for separations > 5 arcmin. Passed design 2 .357e- 06 <= 5e- 05 Passed minimum 2 .357e- 06 <= 3e- 05 Passed stretch 2 .357e- 06 <= 1e- 05 ================================================================= HSC-R band summary ================================================================= Failed minimum 7 of 23 failed Failed design 7 of 23 failed Failed stretch 8 of 23 failed ================================================================= ================================================================= HSC-I band summary ================================================================= Failed minimum 7 of 23 failed Failed design 7 of 23 failed Failed stretch 13 of 23 failed ================================================================= ================================================================= HSC-Y band summary ================================================================= Failed minimum 8 of 23 failed Failed design 8 of 23 failed Failed stretch 14 of 23 failed ================================================================================================================================== design level summary ================================================================= FAILED ( 22 / 69 measurements) =================================================================
            cmorrison Chris Morrison [X] (Inactive) made changes -
            Reviewers Yusra AlSayyad [ yusra ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment - - edited

            Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29389/pipeline

             

            As part of this ticket I've also fixed the config in validate_drp to use the HTM reference catalogs. An initial ticket branch on obs_subaru turned out to not be necessary and has been removed.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - - edited Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29389/pipeline   As part of this ticket I've also fixed the config in validate_drp to use the HTM reference catalogs. An initial ticket branch on obs_subaru turned out to not be necessary and has been removed.
            Hide
            yusra Yusra AlSayyad added a comment -

            Looks OK, but I didn't exactly follow what was going on here: https://github.com/lsst/meas_astrom/pull/111/commits/990c7b45593fd80732d0595633d4ed9ce42753f1#diff-97929b2006e789cef1b1beb342333037R327

            Why does fitting one last time fix things? Because the distances on the matches reflect the final iteration now? 

            Also will you double check your commit messages on the meas_astrom PR?  My eyeballs pattern-matched "interative" with interactive rather than "iterative." PessB already has a final verify step, so I was a little confused about the description that you were adding the final verify step in this commit. 

            Show
            yusra Yusra AlSayyad added a comment - Looks OK, but I didn't exactly follow what was going on here:  https://github.com/lsst/meas_astrom/pull/111/commits/990c7b45593fd80732d0595633d4ed9ce42753f1#diff-97929b2006e789cef1b1beb342333037R327 Why does fitting one last time fix things? Because the distances on the matches reflect the final iteration now?  Also will you double check your commit messages on the meas_astrom PR?  My eyeballs pattern-matched "interative" with interactive rather than "iterative." PessB already has a final verify step, so I was a little confused about the description that you were adding the final verify step in this commit. 
            yusra Yusra AlSayyad made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Basically it's to make the final matches more consistent across the chip. Rather than using just 6 points to fit the shift/rotation matrix it uses the full set of matches to fit a more consistent shift/rotation matrix. This basically prevents matches that should be used and fed to the astrometry from being clipped out because the match is far from where the shift/ration was fit.

            I'm happy to change the wording if needed if you have a suggestion.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Basically it's to make the final matches more consistent across the chip. Rather than using just 6 points to fit the shift/rotation matrix it uses the full set of matches to fit a more consistent shift/rotation matrix. This basically prevents matches that should be used and fed to the astrometry from being clipped out because the match is far from where the shift/ration was fit. I'm happy to change the wording if needed if you have a suggestion.
            Hide
            swinbank John Swinbank added a comment -

            Chris Morrison [X], Yusra AlSayyad — I reckon this is a blocker for pipelines release v17. Do you concur?

            Show
            swinbank John Swinbank added a comment - Chris Morrison [X] , Yusra AlSayyad — I reckon this is a blocker for pipelines release v17. Do you concur?
            swinbank John Swinbank made changes -
            Link This issue blocks DM-16397 [ DM-16397 ]
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            I've updated the comment in the ticket. I'm waiting on either John Swinbank or yourself, Yusra AlSayyad to give it a thumbs up. I'm ready to merge once that happens.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - I've updated the comment in the ticket. I'm waiting on either John Swinbank or yourself, Yusra AlSayyad to give it a thumbs up. I'm ready to merge once that happens.
            Hide
            swinbank John Swinbank added a comment - - edited

            Chris Morrison [X] — a couple of things:

            • Could you take another pass through the commit messages? I still see an “interative” there, as well as a couple of “explaination”s.
            • I actually think the comment you've written here on this ticket is more informative than the one you added to the source. Do you think you could adapt it? Something like:

            The shift/rotation matrix returned by ``_construct_pattern_and_shift_rot_matrix``, above, was based on only six points. Here, we refine that result by using all of the good matches from the “final verification” step, above. This will produce a more consistent result.

            perhaps?

            Show
            swinbank John Swinbank added a comment - - edited Chris Morrison [X] — a couple of things: Could you take another pass through the commit messages? I still see an “interative” there, as well as a couple of “explaination”s. I actually think the comment you've written here on this ticket is more informative than the one you added to the source. Do you think you could adapt it? Something like: The shift/rotation matrix returned by ``_construct_pattern_and_shift_rot_matrix``, above, was based on only six points. Here, we refine that result by using all of the good matches from the “final verification” step, above. This will produce a more consistent result. perhaps?
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Done, sorry I forgot about the typo in the commit. I double checked and the Cfht values are the same as I reported above. Let me know when I'm clear to merge.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Done, sorry I forgot about the typo in the commit. I double checked and the Cfht values are the same as I reported above. Let me know when I'm clear to merge.
            Hide
            swinbank John Swinbank added a comment -

            . Go for it.

            Show
            swinbank John Swinbank added a comment - . Go for it.
            Hide
            cmorrison Chris Morrison [X] (Inactive) added a comment -

            Fingers crosses that the squash metrics improve as I have seen. Talked to Simon Krughoff this morning and he said that squash will run with this version tomorrow morning.

            Show
            cmorrison Chris Morrison [X] (Inactive) added a comment - Fingers crosses that the squash metrics improve as I have seen. Talked to Simon Krughoff this morning and he said that squash will run with this version tomorrow morning.
            cmorrison Chris Morrison [X] (Inactive) made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            swinbank John Swinbank made changes -
            Epic Link DM-16722 [ 235355 ] DM-17995 [ 240551 ]

              People

              Assignee:
              cmorrison Chris Morrison [X] (Inactive)
              Reporter:
              swinbank John Swinbank
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Chris Morrison [X] (Inactive), John Swinbank, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.