Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Story Points:1
-
Team:Data Facility
Description
Some of the HSC RC dataset were not included in HSC-PDR1/DM-9683//datasets/hsc/raw/ssp_pdr1 although some of them are publicly available.
Obtain them from NAOJ if possible, so to allow processing the exactly same RC dataset and facilitate QA.
Attachments
Issue Links
Activity
Hsin-Fang Chiang
created issue -
Hsin-Fang Chiang
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
Hsin-Fang Chiang
made changes -
Hsin-Fang Chiang
made changes -
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
Hsin-Fang Chiang
made changes -
Greg Daues, may you please create /datasets/hsc/raw/ssp_extra for me?
Hsin-Fang Chiang
added a comment - Greg Daues , may you please create /datasets/hsc/raw/ssp_extra for me? This directory /datasets/hsc/raw/ssp_extra has been created; it is ready to receive data from 'hchiang2'.
Greg Daues
added a comment - This directory /datasets/hsc/raw/ssp_extra has been created; it is ready to receive data from 'hchiang2'. Sorry, Greg Daues, may you please also give me the permission of editing /datasets/hsc/repo/registry.sqlite3 and adding new items inside /datasets/hsc/repo/SSP_WIDE/ and /datasets/hsc/repo/SSP_UDEEP_COSMOS? Thank you!
Hsin-Fang Chiang
added a comment - Sorry, Greg Daues , may you please also give me the permission of editing /datasets/hsc/repo/registry.sqlite3 and adding new items inside /datasets/hsc/repo/SSP_WIDE/ and /datasets/hsc/repo/SSP_UDEEP_COSMOS ? Thank you! These should now have immutable flag turned off, so I think you can proceed.
Greg Daues
added a comment - These should now have immutable flag turned off, so I think you can proceed. These 6944 files have been copied to /datasets/hsc/raw/ssp_extra, and then ingested into the butler repo using stack w_2017_15.
$ registryInfo.py /datasets/hsc/repo/ --field 'SSP_UDEEP_COSMOS' -s |
filter field expTime visit
|
HSC-G SSP_UDEEP_COSMOS 5460.0 1886^11688^11690..11712:2^29324^29326^29336^29340^29350^29352 |
HSC-I SSP_UDEEP_COSMOS 9960.0 1226..1232:2^1236..1248:2^19656..19662:2^19680..19684:2^19694..19698:2^19708..19712:2^30482..30504:2 |
HSC-R SSP_UDEEP_COSMOS 5490.0 1200..1222:2^23690..23694:2^23704^23706^23716^23718 |
HSC-Y SSP_UDEEP_COSMOS 23550.0 274..302:2^306..334:2^340..370:2^1856..1862:2^1868..1882:2^11716..11742:2^22602..22608:2^22626..22632:2^22642..22648:2^22658..22664:2 |
HSC-Z SSP_UDEEP_COSMOS 16410.0 1164..1194:2^17898..17908:2^17926..17934:2^17944..17952:2^17962^28354^28356..28402:2 |
NB0921 SSP_UDEEP_COSMOS 25230.0 23036..23056:2^23594..23606:2^24298..24310:2^25810..25816:2 |
|
|
$ registryInfo.py /datasets/hsc/repo/ --field 'SSP_WIDE' -s |
filter field expTime visit
|
HSC-G SSP_WIDE 115530.0 6292..6350:2^9838..9872:2^9880..9908:2^9912..9918:2^11562..11624:2^11630..11640:2^11646..11656:2^11664..11684:2^15186..15196:2^15210..15214:2^15222..15226:2^15234..15238:2^15264..15268:2^15272..15278:2^15282..15318:2^15324..15400:2^25824..25882:2^25886..26014:2^26018..26110:2^29362..29480:2^29484..29502:2^29508^29510^29514^29518^29520..29526:2^29588..29614:2^29618^29620^29624..29630:2^34176..34204:2^34260..34328:2^34332..34344:2^34348..34356:4^34358..34486:2^38206..38266:2^38274..38300:2^38304..38334:2^38338..38354:2^38358^38362^38364^38368^38370..38406:2^38414..38440:2^38444..38470:2^42142..42176:2^42180..42190:2^42194..42342:2^42382..42560:2^46872^46876^46878..46948:2 |
HSC-I SSP_WIDE 161100.0 1252..1318:2^1322..1346:2^7212..7238:2^7286..7314:2^7318..7322:2^7338..7360:2^7364..7374:2^7378..7404:2^7408..7416:2^7424^14078^14080..14104:2^14116..14150:2^14160..14182:2^14192..14214:2^16220..16272:2^16284^16292^16294..16380:2^19378^19382^19384..19432:2^19444..19486:2^19490..19622:2^19628..19654:2^24360..24426:2^24430..24434:2^24438..24494:2^24504..24556:2^24938..25080:2^25084^25088^25090..25150:2^30508..30528:2^30534..30550:2^30554..30594:2^30672..30726:2^35832..35918:2^35922..35954:2^35958..35974:2^36112..36122:2^36126..36266:2^37976..38072:2^45848..45854:2^45858..45876:2^45890..45916:2^45920..45932:2^45936..46008:2^46016..46024:2 |
HSC-R SSP_WIDE 102300.0 7110..7154:2^7158^7164^7166^7168^7204..7208:2^11382..11400:2^11404..11450:2^11458..11480:2^11486..11508:2^11514..11556:2^14836..14848:2^14868..14912:2^14920..14928:2^14932..14990:2^16154..16168:2^16176..16182:2^16190..16196:2^16204..16210:2^23610..23618:2^23622..23670:2^23728..23930:2^28902..28906:2^28910..28930:2^28940..28996:2^29058^29060^29064..29108:2^34030..34098:2^34582..34630:2^34634..34664:2^34668..34774:2^34808..34816:2^40752..40844:2^40848..40972:2^41010..41178:2^45158..45206:2^45210..45230:2^45234^45238^45240^45242^45246..45252:2^45256..45284:2^45288..45320:2^45324^45340^45346..45374:2 |
HSC-Y SSP_WIDE 161800.0 374..432:2^436..480:2^6370..6410:2^6460..6500:2^6512^6522^6524..6546:2^6556..6568:2^13130..13156:2^13166..13172:2^13178..13198:2^13208..13226:2^13236..13254:2^14294..14374:2^14380..14408:2^16054..16062:4^16064^16066^16082..16110:2^16118..16132:2^22438..22540:2^22544..22550:2^22554..22566:2^22572^22584^22586..22598:2^26884..26986:2^26994..27002:2^27006..27084:2^29898..29916:2^29928..29974:2^29978^29998^30000^30004^30006^30028^30036^30038^30050^30074^30076..30080:2^30084^30086^30090..30122:2^30128..30138:2^30142..30148:2^30152..30174:2^30178^30184^30186..30192:2^30204^32258^32260..32360:2^32364..32416:2^32468..32506:2^34870..34916:2^34920..34948:2^34954^36600^36604^36608^36610..36628:2^36632..36720:2^36724..36838:2^36874..36880:2^37520..37574:2^37610..37628:2^39312..39368:2^44008..44036:2^44044..44060:2^44064^44066^44074..44106:2^44110..44166:2^44176..44208:2 |
HSC-Z SSP_WIDE 173210.0 8794..8806:2^8810..8816:2^9596..9626:2^9682..9690:2^9694..9790:2^13272..13288:2^13304^13306^13340^13342^13356..13452:2^15078^15080^15084..15182:2^17654..17686:2^17690..17726:2^17730..17754:2^17758..17850:2^17854..17894:2^23108..23260:2^27088..27160:2^28432^28438^28440^31262^31264..31270:2^31274..31282:4^31284..31350:2^31370..31448:2^31468..31530:2^33762..33776:2^33780..33846:2^33850..33868:2^33872..33952:2^36040..36058:2^36062..36108:2^36324..36340:2^36344..36372:2^36376^36378^36382..36398:2^36402..36508:2^38876..39028:2^44444..44512:2^44516..44536:2^44542^44560^44590^44614^44616^44620^44622..44686:2^44690..44772:2^47042^47056^47060..47072:4^47074^47076^47080^47092^47094^47098^47112^47116^47118..47126:2^47130^47134^47136^47140^47142..47148:2^47154 |
Hsin-Fang Chiang
added a comment - These 6944 files have been copied to /datasets/hsc/raw/ssp_extra , and then ingested into the butler repo using stack w_2017_15 .
$ registryInfo.py /datasets/hsc/repo/ --field 'SSP_UDEEP_COSMOS' -s
filter field expTime visit
HSC-G SSP_UDEEP_COSMOS 5460.0 1886 ^ 11688 ^ 11690 .. 11712 : 2 ^ 29324 ^ 29326 ^ 29336 ^ 29340 ^ 29350 ^ 29352
HSC-I SSP_UDEEP_COSMOS 9960.0 1226 .. 1232 : 2 ^ 1236 .. 1248 : 2 ^ 19656 .. 19662 : 2 ^ 19680 .. 19684 : 2 ^ 19694 .. 19698 : 2 ^ 19708 .. 19712 : 2 ^ 30482 .. 30504 : 2
HSC-R SSP_UDEEP_COSMOS 5490.0 1200 .. 1222 : 2 ^ 23690 .. 23694 : 2 ^ 23704 ^ 23706 ^ 23716 ^ 23718
HSC-Y SSP_UDEEP_COSMOS 23550.0 274 .. 302 : 2 ^ 306 .. 334 : 2 ^ 340 .. 370 : 2 ^ 1856 .. 1862 : 2 ^ 1868 .. 1882 : 2 ^ 11716 .. 11742 : 2 ^ 22602 .. 22608 : 2 ^ 22626 .. 22632 : 2 ^ 22642 .. 22648 : 2 ^ 22658 .. 22664 : 2
HSC-Z SSP_UDEEP_COSMOS 16410.0 1164 .. 1194 : 2 ^ 17898 .. 17908 : 2 ^ 17926 .. 17934 : 2 ^ 17944 .. 17952 : 2 ^ 17962 ^ 28354 ^ 28356 .. 28402 : 2
NB0921 SSP_UDEEP_COSMOS 25230.0 23036 .. 23056 : 2 ^ 23594 .. 23606 : 2 ^ 24298 .. 24310 : 2 ^ 25810 .. 25816 : 2
$ registryInfo.py /datasets/hsc/repo/ --field 'SSP_WIDE' -s
filter field expTime visit
HSC-G SSP_WIDE 115530.0 6292 .. 6350 : 2 ^ 9838 .. 9872 : 2 ^ 9880 .. 9908 : 2 ^ 9912 .. 9918 : 2 ^ 11562 .. 11624 : 2 ^ 11630 .. 11640 : 2 ^ 11646 .. 11656 : 2 ^ 11664 .. 11684 : 2 ^ 15186 .. 15196 : 2 ^ 15210 .. 15214 : 2 ^ 15222 .. 15226 : 2 ^ 15234 .. 15238 : 2 ^ 15264 .. 15268 : 2 ^ 15272 .. 15278 : 2 ^ 15282 .. 15318 : 2 ^ 15324 .. 15400 : 2 ^ 25824 .. 25882 : 2 ^ 25886 .. 26014 : 2 ^ 26018 .. 26110 : 2 ^ 29362 .. 29480 : 2 ^ 29484 .. 29502 : 2 ^ 29508 ^ 29510 ^ 29514 ^ 29518 ^ 29520 .. 29526 : 2 ^ 29588 .. 29614 : 2 ^ 29618 ^ 29620 ^ 29624 .. 29630 : 2 ^ 34176 .. 34204 : 2 ^ 34260 .. 34328 : 2 ^ 34332 .. 34344 : 2 ^ 34348 .. 34356 : 4 ^ 34358 .. 34486 : 2 ^ 38206 .. 38266 : 2 ^ 38274 .. 38300 : 2 ^ 38304 .. 38334 : 2 ^ 38338 .. 38354 : 2 ^ 38358 ^ 38362 ^ 38364 ^ 38368 ^ 38370 .. 38406 : 2 ^ 38414 .. 38440 : 2 ^ 38444 .. 38470 : 2 ^ 42142 .. 42176 : 2 ^ 42180 .. 42190 : 2 ^ 42194 .. 42342 : 2 ^ 42382 .. 42560 : 2 ^ 46872 ^ 46876 ^ 46878 .. 46948 : 2
HSC-I SSP_WIDE 161100.0 1252 .. 1318 : 2 ^ 1322 .. 1346 : 2 ^ 7212 .. 7238 : 2 ^ 7286 .. 7314 : 2 ^ 7318 .. 7322 : 2 ^ 7338 .. 7360 : 2 ^ 7364 .. 7374 : 2 ^ 7378 .. 7404 : 2 ^ 7408 .. 7416 : 2 ^ 7424 ^ 14078 ^ 14080 .. 14104 : 2 ^ 14116 .. 14150 : 2 ^ 14160 .. 14182 : 2 ^ 14192 .. 14214 : 2 ^ 16220 .. 16272 : 2 ^ 16284 ^ 16292 ^ 16294 .. 16380 : 2 ^ 19378 ^ 19382 ^ 19384 .. 19432 : 2 ^ 19444 .. 19486 : 2 ^ 19490 .. 19622 : 2 ^ 19628 .. 19654 : 2 ^ 24360 .. 24426 : 2 ^ 24430 .. 24434 : 2 ^ 24438 .. 24494 : 2 ^ 24504 .. 24556 : 2 ^ 24938 .. 25080 : 2 ^ 25084 ^ 25088 ^ 25090 .. 25150 : 2 ^ 30508 .. 30528 : 2 ^ 30534 .. 30550 : 2 ^ 30554 .. 30594 : 2 ^ 30672 .. 30726 : 2 ^ 35832 .. 35918 : 2 ^ 35922 .. 35954 : 2 ^ 35958 .. 35974 : 2 ^ 36112 .. 36122 : 2 ^ 36126 .. 36266 : 2 ^ 37976 .. 38072 : 2 ^ 45848 .. 45854 : 2 ^ 45858 .. 45876 : 2 ^ 45890 .. 45916 : 2 ^ 45920 .. 45932 : 2 ^ 45936 .. 46008 : 2 ^ 46016 .. 46024 : 2
HSC-R SSP_WIDE 102300.0 7110 .. 7154 : 2 ^ 7158 ^ 7164 ^ 7166 ^ 7168 ^ 7204 .. 7208 : 2 ^ 11382 .. 11400 : 2 ^ 11404 .. 11450 : 2 ^ 11458 .. 11480 : 2 ^ 11486 .. 11508 : 2 ^ 11514 .. 11556 : 2 ^ 14836 .. 14848 : 2 ^ 14868 .. 14912 : 2 ^ 14920 .. 14928 : 2 ^ 14932 .. 14990 : 2 ^ 16154 .. 16168 : 2 ^ 16176 .. 16182 : 2 ^ 16190 .. 16196 : 2 ^ 16204 .. 16210 : 2 ^ 23610 .. 23618 : 2 ^ 23622 .. 23670 : 2 ^ 23728 .. 23930 : 2 ^ 28902 .. 28906 : 2 ^ 28910 .. 28930 : 2 ^ 28940 .. 28996 : 2 ^ 29058 ^ 29060 ^ 29064 .. 29108 : 2 ^ 34030 .. 34098 : 2 ^ 34582 .. 34630 : 2 ^ 34634 .. 34664 : 2 ^ 34668 .. 34774 : 2 ^ 34808 .. 34816 : 2 ^ 40752 .. 40844 : 2 ^ 40848 .. 40972 : 2 ^ 41010 .. 41178 : 2 ^ 45158 .. 45206 : 2 ^ 45210 .. 45230 : 2 ^ 45234 ^ 45238 ^ 45240 ^ 45242 ^ 45246 .. 45252 : 2 ^ 45256 .. 45284 : 2 ^ 45288 .. 45320 : 2 ^ 45324 ^ 45340 ^ 45346 .. 45374 : 2
HSC-Y SSP_WIDE 161800.0 374 .. 432 : 2 ^ 436 .. 480 : 2 ^ 6370 .. 6410 : 2 ^ 6460 .. 6500 : 2 ^ 6512 ^ 6522 ^ 6524 .. 6546 : 2 ^ 6556 .. 6568 : 2 ^ 13130 .. 13156 : 2 ^ 13166 .. 13172 : 2 ^ 13178 .. 13198 : 2 ^ 13208 .. 13226 : 2 ^ 13236 .. 13254 : 2 ^ 14294 .. 14374 : 2 ^ 14380 .. 14408 : 2 ^ 16054 .. 16062 : 4 ^ 16064 ^ 16066 ^ 16082 .. 16110 : 2 ^ 16118 .. 16132 : 2 ^ 22438 .. 22540 : 2 ^ 22544 .. 22550 : 2 ^ 22554 .. 22566 : 2 ^ 22572 ^ 22584 ^ 22586 .. 22598 : 2 ^ 26884 .. 26986 : 2 ^ 26994 .. 27002 : 2 ^ 27006 .. 27084 : 2 ^ 29898 .. 29916 : 2 ^ 29928 .. 29974 : 2 ^ 29978 ^ 29998 ^ 30000 ^ 30004 ^ 30006 ^ 30028 ^ 30036 ^ 30038 ^ 30050 ^ 30074 ^ 30076 .. 30080 : 2 ^ 30084 ^ 30086 ^ 30090 .. 30122 : 2 ^ 30128 .. 30138 : 2 ^ 30142 .. 30148 : 2 ^ 30152 .. 30174 : 2 ^ 30178 ^ 30184 ^ 30186 .. 30192 : 2 ^ 30204 ^ 32258 ^ 32260 .. 32360 : 2 ^ 32364 .. 32416 : 2 ^ 32468 .. 32506 : 2 ^ 34870 .. 34916 : 2 ^ 34920 .. 34948 : 2 ^ 34954 ^ 36600 ^ 36604 ^ 36608 ^ 36610 .. 36628 : 2 ^ 36632 .. 36720 : 2 ^ 36724 .. 36838 : 2 ^ 36874 .. 36880 : 2 ^ 37520 .. 37574 : 2 ^ 37610 .. 37628 : 2 ^ 39312 .. 39368 : 2 ^ 44008 .. 44036 : 2 ^ 44044 .. 44060 : 2 ^ 44064 ^ 44066 ^ 44074 .. 44106 : 2 ^ 44110 .. 44166 : 2 ^ 44176 .. 44208 : 2
HSC-Z SSP_WIDE 173210.0 8794 .. 8806 : 2 ^ 8810 .. 8816 : 2 ^ 9596 .. 9626 : 2 ^ 9682 .. 9690 : 2 ^ 9694 .. 9790 : 2 ^ 13272 .. 13288 : 2 ^ 13304 ^ 13306 ^ 13340 ^ 13342 ^ 13356 .. 13452 : 2 ^ 15078 ^ 15080 ^ 15084 .. 15182 : 2 ^ 17654 .. 17686 : 2 ^ 17690 .. 17726 : 2 ^ 17730 .. 17754 : 2 ^ 17758 .. 17850 : 2 ^ 17854 .. 17894 : 2 ^ 23108 .. 23260 : 2 ^ 27088 .. 27160 : 2 ^ 28432 ^ 28438 ^ 28440 ^ 31262 ^ 31264 .. 31270 : 2 ^ 31274 .. 31282 : 4 ^ 31284 .. 31350 : 2 ^ 31370 .. 31448 : 2 ^ 31468 .. 31530 : 2 ^ 33762 .. 33776 : 2 ^ 33780 .. 33846 : 2 ^ 33850 .. 33868 : 2 ^ 33872 .. 33952 : 2 ^ 36040 .. 36058 : 2 ^ 36062 .. 36108 : 2 ^ 36324 .. 36340 : 2 ^ 36344 .. 36372 : 2 ^ 36376 ^ 36378 ^ 36382 .. 36398 : 2 ^ 36402 .. 36508 : 2 ^ 38876 .. 39028 : 2 ^ 44444 .. 44512 : 2 ^ 44516 .. 44536 : 2 ^ 44542 ^ 44560 ^ 44590 ^ 44614 ^ 44616 ^ 44620 ^ 44622 .. 44686 : 2 ^ 44690 .. 44772 : 2 ^ 47042 ^ 47056 ^ 47060 .. 47072 : 4 ^ 47074 ^ 47076 ^ 47080 ^ 47092 ^ 47094 ^ 47098 ^ 47112 ^ 47116 ^ 47118 .. 47126 : 2 ^ 47130 ^ 47134 ^ 47136 ^ 47140 ^ 47142 .. 47148 : 2 ^ 47154
Hsin-Fang Chiang
made changes -
| Labels | datasets |
Hsin-Fang Chiang
made changes -
Greg Daues may you please lock down the folders (/datasets/hsc/raw/ssp_extra, /datasets/hsc/repo/SSP_WIDE/, /datasets/hsc/repo/SSP_UDEEP_COSMOS, /datasets/hsc/repo/registry.sqlite3)
Hsin-Fang Chiang
added a comment - Greg Daues may you please lock down the folders (/datasets/hsc/raw/ssp_extra, /datasets/hsc/repo/SSP_WIDE/, /datasets/hsc/repo/SSP_UDEEP_COSMOS, /datasets/hsc/repo/registry.sqlite3) These directories and files are now locked down / immutable.
Greg Daues
added a comment - These directories and files are now locked down / immutable. Thank you Greg!
Hsin-Fang Chiang
added a comment - Thank you Greg! Hsin-Fang Chiang
made changes -
| Resolution | Done [ 10000 ] | |
| Status | In Progress [ 3 ] | Done [ 10002 ] |
Hsin-Fang Chiang
made changes -
| Remote Link | This issue links to "Page (Confluence)" [ 15000 ] |
| Team | Process Middleware [ 10206 ] | Data Facility [ 12219 ] |
Downloading these 62 visits (*112 ccd each = 6944 files) from SMOKA:
274^276^278^280^282^284^286^288^290^292^294^296^298^300^302^306^308^310^312^314^316^320^334^342^364^366^368^370^1236^1858^1860^1862^1878^9864^9890^11742^28354^28356^28358^28360^28362^28364^28366^28368^28370^28372^28374^28376^28378^28380^28382^28384^28386^28388^28390^28392^28394^28396^28398^28400^28402^29352