Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Story Points:1
-
Team:Data Facility
Description
Store a set of HSC Bright Object Masks of version "Arcturus" to the /datasets/hsc butler root repo. These masks should become available as Butler type brightObjectMask
Also adding links for NB0816 and NB0921 filters for all tracts/patches.
Attachments
Issue Links
Activity
Hsin-Fang Chiang
created issue -
Hsin-Fang Chiang
made changes -
| Field | Original Value | New Value |
|---|---|---|
| Epic Link |
|
Hsin-Fang Chiang
made changes -
Hsin-Fang Chiang
made changes -
| Labels | datasets |
Hsin-Fang Chiang. I have created such a directory; can you check if access is suitable ?
Greg Daues
added a comment - Hsin-Fang Chiang . I have created such a directory; can you check if access is suitable ? Greg Daues: Yes. Copying data over now. Thank you!
Hsin-Fang Chiang
added a comment - Greg Daues : Yes. Copying data over now. Thank you! Hsin-Fang Chiang
made changes -
| Remote Link | This issue links to "Page (Confluence)" [ 15037 ] |
Greg Daues may you please make /datasets/hsc/repo/deepCoadd/ immutable?
Hsin-Fang Chiang
added a comment - Greg Daues may you please make /datasets/hsc/repo/deepCoadd/ immutable? Hsin-Fang Chiang
made changes -
| Reviewers | Greg Daues [ gdaues ] | |
| Status | To Do [ 10001 ] | In Review [ 10004 ] |
brightObjectMask can now be accessed from the HSC butler repo. For example,
butler = lsst.daf.persistence.Butler("/datasets/hsc/repo/") |
butler.get("brightObjectMask", tract=8284, patch='5,2', filter='HSC-I') |
Note: not every patch/tract has brightObjectMask available.
Hsin-Fang Chiang
added a comment - brightObjectMask can now be accessed from the HSC butler repo. For example,
butler = lsst.daf.persistence.Butler( "/datasets/hsc/repo/" )
butler.get( "brightObjectMask" , tract= 8284 , patch= '5,2' , filter= 'HSC-I' )
Note: not every patch/tract has brightObjectMask available. The directory /datasets/hsc/repo/deepCoadd/ and the subdirectories/contents should now be immutable, group lsst_users, readable by all, i.e., ready for use.
Greg Daues
added a comment - The directory /datasets/hsc/repo/deepCoadd/ and the subdirectories/contents should now be immutable, group lsst_users , readable by all, i.e., ready for use. I attempted the lines above for v13_0 of the stack and obtain many WARN messages; it does not look like it really worked. Is that to be expected?
Sample:
ObjectMaskCatalog WARN: Unexpected line "box(37.039951,-6.990420,0.034446d,0.003445d,0) # ID: 5178395927953628288, mag: 11.389953" at /datasets/hsc/repo/deepCoadd/BrightObjectMasks/8284/BrightObjectMask-8284-5,2-HSC-I.reg:135
|
Greg Daues
added a comment - I attempted the lines above for v13_0 of the stack and obtain many WARN messages; it does not look like it really worked. Is that to be expected?
Sample:
ObjectMaskCatalog WARN: Unexpected line "box(37.039951,-6.990420,0.034446d,0.003445d,0) # ID: 5178395927953628288, mag: 11.389953" at /datasets/hsc/repo/deepCoadd/BrightObjectMasks/8284/BrightObjectMask-8284-5,2-HSC-I.reg:135
I see those WARN messages using the v13_0 stack to retrieve them too, but a w_2017_17 stack or newer can read it fine. I guess it's because of this commit from DM-10308. Older stacks were not compatible with these object masks.
Hsin-Fang Chiang
added a comment - I see those WARN messages using the v13_0 stack to retrieve them too, but a w_2017_17 stack or newer can read it fine. I guess it's because of this commit from DM-10308 . Older stacks were not compatible with these object masks. In my tests I had trouble with the v13_0 stack, and also with a build b2913 yesterday, but now I see indeed that it is working with the w_2017_17 stack, and today b2914 shows success. So looks good.
Greg Daues
added a comment - In my tests I had trouble with the v13_0 stack, and also with a build b2913 yesterday, but now I see indeed that it is working with the w_2017_17 stack, and today b2914 shows success. So looks good. Greg Daues
made changes -
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Greg Daues Sorry, I need the permission of /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ again. I will make many new links (all within the same folder), but won't add new files.
Hsin-Fang Chiang
added a comment - Greg Daues Sorry, I need the permission of /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ again. I will make many new links (all within the same folder), but won't add new files. (I won't modify or delete any existing files/links either. So what's already immutable can stay immutable)
Hsin-Fang Chiang
added a comment - - edited (I won't modify or delete any existing files/links either. So what's already immutable can stay immutable) Hsin-Fang Chiang The directory /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ has been reopened.
Greg Daues
added a comment - Hsin-Fang Chiang The directory /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ has been reopened. Greg Daues, the folder can be locked down again. Thank you!
Hsin-Fang Chiang
added a comment - Greg Daues , the folder can be locked down again. Thank you! Hsin-Fang Chiang
made changes -
| Description | Store a set of HSC Bright Object Masks of version "Arcturus" to the {{/datasets/hsc}} butler root repo. These masks should become available as Butler type {{brightObjectMask}} |
Store a set of HSC Bright Object Masks of version "Arcturus" to the {{/datasets/hsc}} butler root repo. These masks should become available as Butler type {{brightObjectMask}}
Also adding links for {{NB0816}} and {{NB0921}} filters for all tracts/patches. |
Hsin-Fang Chiang The directory /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ and its contents are once again immutable.
Greg Daues
added a comment - Hsin-Fang Chiang The directory /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ and its contents are once again immutable. Thank you!
Done.
Hsin-Fang Chiang
added a comment - Thank you!
Done. Hsin-Fang Chiang
made changes -
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
| Team | Process Middleware [ 10206 ] | Data Facility [ 12219 ] |
Greg Daues, may I have a new folder /datasets/hsc/repo/deepCoadd/BrightObjectMasks/ and permission to add new data into it?