[DM-17843] A new set of processCcd failure in HSC-RC2 reprocessing - Jira

XML

Word

Printable

Details

Type: Story
Status: Done
Resolution: Done
Fix Version/s: None
Component/s: None
Labels:
None

Story Points:
4
Epic Link:
s19b-ap-matcher
Sprint:
AP S19-3
Team:
Alert Production

Description

Comparing HSC-RC2 reprocessing by stack w_2019_02 and w_2019_06, we've got a new set of processCcd errors. They fail at calibrate.astrometry.matcher with

  File "/software/lsstsw/stack_20181012/stack/miniconda3-4.5.4-fcd27eb/Linux64/meas_astrom/16.0-24-gfa57b64+1/python/lsst/meas/astrom/matchPessimisticB.py", line 278, in matchObjectsToSources

    raise RuntimeError("Unable to match sources")

The 60 data ids of the new failure set is as follows

--id visit=34338 ccd=75

--id visit=34362 ccd=38

--id visit=34382 ccd=43

--id visit=34384 ccd=88

--id visit=34402 ccd=61

--id visit=34422 ccd=42

--id visit=34424 ccd=2

--id visit=34448 ccd=79

--id visit=34450 ccd=25

--id visit=34478 ccd=21

--id visit=34482 ccd=50

--id visit=34644 ccd=72

--id visit=34690 ccd=39

--id visit=36140 ccd=30

--id visit=36170 ccd=18

--id visit=36182 ccd=80

--id visit=36212 ccd=34

--id visit=36234 ccd=82

--id visit=36240 ccd=0

--id visit=36258 ccd=73

--id visit=36428 ccd=61

--id visit=36432 ccd=23

--id visit=36492 ccd=50

--id visit=34942 ccd=44

--id visit=36762 ccd=18

--id visit=36774 ccd=24

--id visit=36792 ccd=34

--id visit=36808 ccd=26

--id visit=36828 ccd=16

--id visit=36828 ccd=73

--id visit=26046 ccd=32

--id visit=26050 ccd=74

--id visit=26058 ccd=34

--id visit=26060 ccd=70

-id visit=26072 ccd=12

--id visit=26080 ccd=66

--id visit=26084 ccd=64

--id visit=23864 ccd=12

--id visit=1308 ccd=40

--id visit=23224 ccd=59

--id visit=23232 ccd=32

--id visit=27106 ccd=33

--id visit=27106 ccd=41

--id visit=27128 ccd=26

--id visit=27134 ccd=28

--id visit=440 ccd=92

--id visit=452 ccd=35

--id visit=452 ccd=42

--id visit=472 ccd=30

--id visit=1246 ccd=19

--id visit=19696 ccd=20

--id visit=30500 ccd=20

--id visit=1180 ccd=89

--id visit=1184 ccd=26

--id visit=1194 ccd=26

--id visit=360 ccd=26

--id visit=22628 ccd=18

--id visit=22664 ccd=26

--id visit=23046 ccd=26

--id visit=23050 ccd=84

Attachments

Issue Links

blocks

DM-16397 Do release 17.0 of science pipelines

Done

relates to

DM-17918 Fix PessimisticB automatic tolerance minimum

Invalid

DM-17400 Reprocess RC2 with w_2019_06

Done

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Hsin-Fang Chiang added a comment - 11/Feb/19 1:42 PM

The change looks expected by ~~DM-14857~~ (Switch the default matcher to PessimisticB) which merged before w_2019_04. I'm not sure if further action is necessary; Yusra AlSayyad please close this if you think it's fine to close.

Show

Hsin-Fang Chiang added a comment - 11/Feb/19 1:42 PM The change looks expected by DM-14857 (Switch the default matcher to PessimisticB) which merged before w_2019_04. I'm not sure if further action is necessary; Yusra AlSayyad please close this if you think it's fine to close.

Hide

Permalink

John Swinbank added a comment - 11/Feb/19 1:52 PM

Regressions in the SQuaSH metrics are expected, and should be addressed by ~~DM-17552~~ — we should, of course, confirm they really are fixed when that closes.

It's not written here, but I understand from Slack that we have jumped from 9 CCDs failing to 60 failing with the new matcher. That tells me that this issue is not fine to close without further investigation.

Chris Morrison [X], any thoughts about what's happening here?

Show

John Swinbank added a comment - 11/Feb/19 1:52 PM Regressions in the SQuaSH metrics are expected, and should be addressed by DM-17552 — we should, of course, confirm they really are fixed when that closes. It's not written here, but I understand from Slack that we have jumped from 9 CCDs failing to 60 failing with the new matcher. That tells me that this issue is not fine to close without further investigation. Chris Morrison [X] , any thoughts about what's happening here?

Hide

Permalink

Chris Morrison [X] (Inactive) added a comment - 11/Feb/19 2:02 PM

The regression is already fixed in ~~DM-17552~~.

As for what is happening with the new set of failures, I wouldn't know without looking. My guess is some of the default obs_subaru/HSC settings that are hard coded and were designed for the old matcher don't apply anymore and should be tweaked. What reference catalog is being used?

Show

Chris Morrison [X] (Inactive) added a comment - 11/Feb/19 2:02 PM The regression is already fixed in DM-17552 . As for what is happening with the new set of failures, I wouldn't know without looking. My guess is some of the default obs_subaru/HSC settings that are hard coded and were designed for the old matcher don't apply anymore and should be tweaked. What reference catalog is being used?

Hide

Permalink

Hsin-Fang Chiang added a comment - 11/Feb/19 2:07 PM

From the config file

config.processCcd.charImage.refObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110'

config.processCcd.calibrate.astromRefObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110'

config.processCcd.calibrate.photoRefObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110'

config.processCcd.calibrate.photoCal.photoCatName='ps1_pv3_3pi_20170110'

Show

Hsin-Fang Chiang added a comment - 11/Feb/19 2:07 PM From the config file config.processCcd.charImage.refObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110' config.processCcd.calibrate.astromRefObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110' config.processCcd.calibrate.photoRefObjLoader.ref_dataset_name='ps1_pv3_3pi_20170110' config.processCcd.calibrate.photoCal.photoCatName='ps1_pv3_3pi_20170110'

Hide

Permalink

Chris Morrison [X] (Inactive) added a comment - 11/Feb/19 2:22 PM

Okay, my other wonder is if these failures are happening more in one given band than another. For instance, are all the failures in Y or Z say where fewer source detentions are available?

Show

Chris Morrison [X] (Inactive) added a comment - 11/Feb/19 2:22 PM Okay, my other wonder is if these failures are happening more in one given band than another. For instance, are all the failures in Y or Z say where fewer source detentions are available?

Hide

Permalink

Hsin-Fang Chiang added a comment - 11/Feb/19 2:50 PM

It happens in non-Y/Z bands too. Their full data IDs are:

dataId={'visit': 1246, 'ccd': 19, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-I', 'taiObs': '2014-03-28', 'expTime': 270.0}

dataId={'visit': 19696, 'ccd': 20, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-21', 'pointing': 1116, 'filter': 'HSC-I', 'taiObs': '2015-01-21', 'expTime': 300.0}

dataId={'visit': 30500, 'ccd': 20, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-05-21', 'pointing': 1236, 'filter': 'HSC-I', 'taiObs': '2015-05-21', 'expTime': 300.0}

dataId={'visit': 23046, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'NB0921', 'taiObs': '2015-03-16', 'expTime': 900.0}

dataId={'visit': 23050, 'ccd': 84, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'NB0921', 'taiObs': '2015-03-16', 'expTime': 900.0}

dataId={'visit': 360, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 240.0}

dataId={'visit': 22628, 'ccd': 18, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-27', 'pointing': 1122, 'filter': 'HSC-Y', 'taiObs': '2015-01-27', 'expTime': 300.0}

dataId={'visit': 22664, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-27', 'pointing': 1122, 'filter': 'HSC-Y', 'taiObs': '2015-01-27', 'expTime': 300.0}

dataId={'visit': 1180, 'ccd': 89, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0}

dataId={'visit': 1184, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0}

dataId={'visit': 1194, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0}

dataId={'visit': 26046, 'ccd': 32, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26050, 'ccd': 74, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26058, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26060, 'ccd': 70, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26072, 'ccd': 12, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26080, 'ccd': 66, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 26084, 'ccd': 64, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0}

dataId={'visit': 1308, 'ccd': 40, 'field': 'SSP_WIDE', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-I', 'taiObs': '2014-03-28', 'expTime': 200.0}

dataId={'visit': 23864, 'ccd': 12, 'field': 'SSP_WIDE', 'dateObs': '2015-03-18', 'pointing': 1172, 'filter': 'HSC-R', 'taiObs': '2015-03-18', 'expTime': 150.0}

dataId={'visit': 440, 'ccd': 92, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0}

dataId={'visit': 452, 'ccd': 35, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0}

dataId={'visit': 452, 'ccd': 42, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0}

dataId={'visit': 472, 'ccd': 30, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0}

dataId={'visit': 23224, 'ccd': 59, 'field': 'SSP_WIDE', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'HSC-Z', 'taiObs': '2015-03-16', 'expTime': 200.0}

dataId={'visit': 23232, 'ccd': 32, 'field': 'SSP_WIDE', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'HSC-Z', 'taiObs': '2015-03-16', 'expTime': 200.0}

dataId={'visit': 27106, 'ccd': 33, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0}

dataId={'visit': 27106, 'ccd': 41, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0}

dataId={'visit': 27128, 'ccd': 26, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0}

dataId={'visit': 27134, 'ccd': 28, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0}

dataId={'visit': 34338, 'ccd': 75, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34362, 'ccd': 38, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34382, 'ccd': 43, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34384, 'ccd': 88, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34402, 'ccd': 61, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34422, 'ccd': 42, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34424, 'ccd': 2, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34448, 'ccd': 79, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34450, 'ccd': 25, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34478, 'ccd': 21, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 34482, 'ccd': 50, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0}

dataId={'visit': 36140, 'ccd': 30, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36170, 'ccd': 18, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36182, 'ccd': 80, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36212, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36234, 'ccd': 82, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36240, 'ccd': 0, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 36258, 'ccd': 73, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0}

dataId={'visit': 34644, 'ccd': 72, 'field': 'SSP_WIDE', 'dateObs': '2015-07-15', 'pointing': 1291, 'filter': 'HSC-R', 'taiObs': '2015-07-15', 'expTime': 150.0}

dataId={'visit': 34690, 'ccd': 39, 'field': 'SSP_WIDE', 'dateObs': '2015-07-15', 'pointing': 1291, 'filter': 'HSC-R', 'taiObs': '2015-07-15', 'expTime': 150.0}

dataId={'visit': 34942, 'ccd': 44, 'field': 'SSP_WIDE', 'dateObs': '2015-07-16', 'pointing': 1292, 'filter': 'HSC-Y', 'taiObs': '2015-07-16', 'expTime': 200.0}

dataId={'visit': 36762, 'ccd': 18, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36774, 'ccd': 24, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36792, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36808, 'ccd': 26, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36828, 'ccd': 16, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36828, 'ccd': 73, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0}

dataId={'visit': 36428, 'ccd': 61, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0}

dataId={'visit': 36432, 'ccd': 23, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0}

dataId={'visit': 36492, 'ccd': 50, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0}

Show

Hsin-Fang Chiang added a comment - 11/Feb/19 2:50 PM It happens in non-Y/Z bands too. Their full data IDs are: dataId={'visit': 1246, 'ccd': 19, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-I', 'taiObs': '2014-03-28', 'expTime': 270.0} dataId={'visit': 19696, 'ccd': 20, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-21', 'pointing': 1116, 'filter': 'HSC-I', 'taiObs': '2015-01-21', 'expTime': 300.0} dataId={'visit': 30500, 'ccd': 20, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-05-21', 'pointing': 1236, 'filter': 'HSC-I', 'taiObs': '2015-05-21', 'expTime': 300.0} dataId={'visit': 23046, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'NB0921', 'taiObs': '2015-03-16', 'expTime': 900.0} dataId={'visit': 23050, 'ccd': 84, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'NB0921', 'taiObs': '2015-03-16', 'expTime': 900.0} dataId={'visit': 360, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 240.0} dataId={'visit': 22628, 'ccd': 18, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-27', 'pointing': 1122, 'filter': 'HSC-Y', 'taiObs': '2015-01-27', 'expTime': 300.0} dataId={'visit': 22664, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2015-01-27', 'pointing': 1122, 'filter': 'HSC-Y', 'taiObs': '2015-01-27', 'expTime': 300.0} dataId={'visit': 1180, 'ccd': 89, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0} dataId={'visit': 1184, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0} dataId={'visit': 1194, 'ccd': 26, 'field': 'SSP_UDEEP_COSMOS', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-Z', 'taiObs': '2014-03-28', 'expTime': 270.0} dataId={'visit': 26046, 'ccd': 32, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26050, 'ccd': 74, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26058, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26060, 'ccd': 70, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26072, 'ccd': 12, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26080, 'ccd': 66, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 26084, 'ccd': 64, 'field': 'SSP_WIDE', 'dateObs': '2015-03-25', 'pointing': 1179, 'filter': 'HSC-G', 'taiObs': '2015-03-25', 'expTime': 150.0} dataId={'visit': 1308, 'ccd': 40, 'field': 'SSP_WIDE', 'dateObs': '2014-03-28', 'pointing': 817, 'filter': 'HSC-I', 'taiObs': '2014-03-28', 'expTime': 200.0} dataId={'visit': 23864, 'ccd': 12, 'field': 'SSP_WIDE', 'dateObs': '2015-03-18', 'pointing': 1172, 'filter': 'HSC-R', 'taiObs': '2015-03-18', 'expTime': 150.0} dataId={'visit': 440, 'ccd': 92, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0} dataId={'visit': 452, 'ccd': 35, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0} dataId={'visit': 452, 'ccd': 42, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0} dataId={'visit': 472, 'ccd': 30, 'field': 'SSP_WIDE', 'dateObs': '2014-03-25', 'pointing': 814, 'filter': 'HSC-Y', 'taiObs': '2014-03-25', 'expTime': 200.0} dataId={'visit': 23224, 'ccd': 59, 'field': 'SSP_WIDE', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'HSC-Z', 'taiObs': '2015-03-16', 'expTime': 200.0} dataId={'visit': 23232, 'ccd': 32, 'field': 'SSP_WIDE', 'dateObs': '2015-03-16', 'pointing': 1170, 'filter': 'HSC-Z', 'taiObs': '2015-03-16', 'expTime': 200.0} dataId={'visit': 27106, 'ccd': 33, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0} dataId={'visit': 27106, 'ccd': 41, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0} dataId={'visit': 27128, 'ccd': 26, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0} dataId={'visit': 27134, 'ccd': 28, 'field': 'SSP_WIDE', 'dateObs': '2015-03-29', 'pointing': 1183, 'filter': 'HSC-Z', 'taiObs': '2015-03-29', 'expTime': 200.0} dataId={'visit': 34338, 'ccd': 75, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34362, 'ccd': 38, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34382, 'ccd': 43, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34384, 'ccd': 88, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34402, 'ccd': 61, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34422, 'ccd': 42, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34424, 'ccd': 2, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34448, 'ccd': 79, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34450, 'ccd': 25, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34478, 'ccd': 21, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 34482, 'ccd': 50, 'field': 'SSP_WIDE', 'dateObs': '2015-07-14', 'pointing': 1290, 'filter': 'HSC-G', 'taiObs': '2015-07-14', 'expTime': 150.0} dataId={'visit': 36140, 'ccd': 30, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36170, 'ccd': 18, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36182, 'ccd': 80, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36212, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36234, 'ccd': 82, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36240, 'ccd': 0, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 36258, 'ccd': 73, 'field': 'SSP_WIDE', 'dateObs': '2015-07-21', 'pointing': 1297, 'filter': 'HSC-I', 'taiObs': '2015-07-21', 'expTime': 200.0} dataId={'visit': 34644, 'ccd': 72, 'field': 'SSP_WIDE', 'dateObs': '2015-07-15', 'pointing': 1291, 'filter': 'HSC-R', 'taiObs': '2015-07-15', 'expTime': 150.0} dataId={'visit': 34690, 'ccd': 39, 'field': 'SSP_WIDE', 'dateObs': '2015-07-15', 'pointing': 1291, 'filter': 'HSC-R', 'taiObs': '2015-07-15', 'expTime': 150.0} dataId={'visit': 34942, 'ccd': 44, 'field': 'SSP_WIDE', 'dateObs': '2015-07-16', 'pointing': 1292, 'filter': 'HSC-Y', 'taiObs': '2015-07-16', 'expTime': 200.0} dataId={'visit': 36762, 'ccd': 18, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36774, 'ccd': 24, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36792, 'ccd': 34, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36808, 'ccd': 26, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36828, 'ccd': 16, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36828, 'ccd': 73, 'field': 'SSP_WIDE', 'dateObs': '2015-07-23', 'pointing': 1299, 'filter': 'HSC-Y', 'taiObs': '2015-07-23', 'expTime': 200.0} dataId={'visit': 36428, 'ccd': 61, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0} dataId={'visit': 36432, 'ccd': 23, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0} dataId={'visit': 36492, 'ccd': 50, 'field': 'SSP_WIDE', 'dateObs': '2015-07-22', 'pointing': 1298, 'filter': 'HSC-Z', 'taiObs': '2015-07-22', 'expTime': 200.0}

Hide

Permalink

John Swinbank added a comment - 12/Feb/19 6:33 PM

It seems to me that this probably blocks release v17. Chris Morrison [X], Eric Bellm, Jim Bosch, Yusra AlSayyad — opinions?

Show

John Swinbank added a comment - 12/Feb/19 6:33 PM It seems to me that this probably blocks release v17. Chris Morrison [X] , Eric Bellm , Jim Bosch , Yusra AlSayyad — opinions?

Hide

Permalink

Jim Bosch added a comment - 13/Feb/19 5:53 AM

Agree that this is a release blocker. I'd be open to reverting to OptimisticB for just HSC to avoid impacting the v17 timeline, though a real fix would of course be even better.

Show

Jim Bosch added a comment - 13/Feb/19 5:53 AM Agree that this is a release blocker. I'd be open to reverting to OptimisticB for just HSC to avoid impacting the v17 timeline, though a real fix would of course be even better.

Hide

Permalink

Chris Morrison [X] (Inactive) added a comment - 14/Feb/19 1:29 PM

Hey, I believe I've locked in on the problem after rerunning the 60 failed ccds.

The pessimistic matcher computes an automated match tolerance. Basically this automated tolerance gets set very low in some unlucky cases and the tolerance is allowed to soften enough to get a match. By my estimates it would never be allowed larger than 1 arcsec. I also found a bug in this tolerance that let's it be set to very small, unphysical values.

I'll fix the bug and see if I can't tweak things slightly to get all 60 to succeed.

Show

Chris Morrison [X] (Inactive) added a comment - 14/Feb/19 1:29 PM Hey, I believe I've locked in on the problem after rerunning the 60 failed ccds. The pessimistic matcher computes an automated match tolerance. Basically this automated tolerance gets set very low in some unlucky cases and the tolerance is allowed to soften enough to get a match. By my estimates it would never be allowed larger than 1 arcsec. I also found a bug in this tolerance that let's it be set to very small, unphysical values. I'll fix the bug and see if I can't tweak things slightly to get all 60 to succeed.

Hide

Permalink

Chris Morrison [X] (Inactive) added a comment - 15/Feb/19 9:39 AM

Okay, so tweak I made to the distance tolerance softening iterations and was able to get all 60 ccds to succeed. As I said previously, the problem was that the match distances was never softened with enough iterations to produce a match given the distortions in HSC. I've run the same settings through validate_drp and everything checks out as fine. I'll clean up the ticket once I head into the office. John Swinbank could you find me a reviewer in the man time please?

Show

Chris Morrison [X] (Inactive) added a comment - 15/Feb/19 9:39 AM Okay, so tweak I made to the distance tolerance softening iterations and was able to get all 60 ccds to succeed. As I said previously, the problem was that the match distances was never softened with enough iterations to produce a match given the distortions in HSC. I've run the same settings through validate_drp and everything checks out as fine. I'll clean up the ticket once I head into the office. John Swinbank could you find me a reviewer in the man time please?

Hide

Permalink

Chris Morrison [X] (Inactive) added a comment - 15/Feb/19 3:01 PM - edited

Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29442/pipeline

Show

Chris Morrison [X] (Inactive) added a comment - 15/Feb/19 3:01 PM - edited Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29442/pipeline

Hide

Permalink

Colin Slater added a comment - 15/Feb/19 3:07 PM

Fixes look good to me. The transition to multiple-pattern consensus mode above 2000 stars in the refcat seems reasonable, but it might be worth pulling that number out into a config option; that would at least give it some visibility and a doc string, etc (though it seems unlikely to need to be changed much).

To record some offline discussions here: because the main change here is to allow more iterations of softening the match tolerances before giving up, we think this is unlikely to break any other CCDs, and the testing on validation_data_* hasn't shown any other breakage or degradation of the metrics.

Show

Colin Slater added a comment - 15/Feb/19 3:07 PM Fixes look good to me. The transition to multiple-pattern consensus mode above 2000 stars in the refcat seems reasonable, but it might be worth pulling that number out into a config option; that would at least give it some visibility and a doc string, etc (though it seems unlikely to need to be changed much). To record some offline discussions here: because the main change here is to allow more iterations of softening the match tolerances before giving up, we think this is unlikely to break any other CCDs, and the testing on validation_data_* hasn't shown any other breakage or degradation of the metrics.

People

Assignee:: Chris Morrison [X] (Inactive)

Reporter:: Hsin-Fang Chiang

Reviewers:: Colin Slater

Watchers:: Chris Morrison [X] (Inactive), Colin Slater, Eric Bellm, Hsin-Fang Chiang, Jim Bosch, John Swinbank

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Dates

Created:: 11/Feb/19 12:46 PM

Updated:: 20/Feb/19 2:05 PM

Resolved:: 15/Feb/19 6:16 PM

Jenkins

No builds found.

Data Management

Details

Description

Attachments

Issue Links

Activity

People

Dates

Jenkins