Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24602

extract simulated LSST data for testdata_jointcal

    Details

    • Story Points:
      2
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      Eli Rykoff told me he is working on getting some imsim/phosim data for DESC from DC2 for his own test purposes. We might be able to use the same (or nearly same) data to test LSST input to jointcal and fgcmcal.

      I think a good set of testdata would be ~5 visits with ~6 ccds each (preferably mostly the same ccds), that all overlap more or less the same region.

      We can use this ticket to track what data to use, and to incorporate it into testdata_jointcal.

        Attachments

          Issue Links

            Activity

            Hide
            erykoff Eli Rykoff added a comment -

            I have looked at the DC2 dataset shared with the project for testing on lsst-dev01, in /datasets/DC2/repoRun2.2i/rerun/w_2020_03/DM-22816, and gotten permission from DESC to have a subset of these data go on this public github repo with the following caveats: https://lsstc.slack.com/archives/C978LTJGN/p1588085857170900?thread_ts=1588024424.167600&cid=C978LTJGN . Specifically "the data is not fully validated yet, make sure that it can't be used to do science that DESC folks with DC2 projects want to do, you might still find some bugs." As this is too small a subset to do science, and it is only used for unit and integration tests, this is fine. However, while this looks like expected LSSTCam data now, real data might look different.

            I have extracted a maximally overlapping subset of 6 CCDs from 10 visits (5 r, 5 i) over tract 3828 and patches 1,1; 1,2; 2,1; 2,2. Unfortunately, due to camera rotations and dithering it is not possible to use "mostly the same CCDs". The visits (expId) and CCDs (detector) are:

            Filter r:
            456716 [61, 58, 62, 59, 57, 60]
            401660 [84, 81, 85, 82, 74, 77]
            212071 [49, 48, 46, 45, 47, 52]
            212739 [156, 159, 149, 152, 160, 157]
            202617 [136, 97, 137, 98, 135, 139]
            Filter i:
            457749 [157, 160, 158, 161, 159, 156]
            248970 [22, 23, 25, 26, 20, 21]
            228020 [115, 112, 116, 113, 114, 155]
            280217 [46, 47, 49, 50, 16, 45]
            227984 [105, 102, 106, 98, 144, 103]
            

            Note that these data cannot be read with versions of the stack between the merger of DM-23584 (part of w_2020_08) and the merger of DM-24517 (soon to be part of w_2020_18).

            Show
            erykoff Eli Rykoff added a comment - I have looked at the DC2 dataset shared with the project for testing on lsst-dev01 , in /datasets/DC2/repoRun2.2i/rerun/w_2020_03/ DM-22816 , and gotten permission from DESC to have a subset of these data go on this public github repo with the following caveats: https://lsstc.slack.com/archives/C978LTJGN/p1588085857170900?thread_ts=1588024424.167600&cid=C978LTJGN . Specifically "the data is not fully validated yet, make sure that it can't be used to do science that DESC folks with DC2 projects want to do, you might still find some bugs." As this is too small a subset to do science, and it is only used for unit and integration tests, this is fine. However, while this looks like expected LSSTCam data now, real data might look different. I have extracted a maximally overlapping subset of 6 CCDs from 10 visits (5 r, 5 i) over tract 3828 and patches 1,1; 1,2; 2,1; 2,2. Unfortunately, due to camera rotations and dithering it is not possible to use "mostly the same CCDs". The visits (expId) and CCDs (detector) are: Filter r: 456716 [61, 58, 62, 59, 57, 60] 401660 [84, 81, 85, 82, 74, 77] 212071 [49, 48, 46, 45, 47, 52] 212739 [156, 159, 149, 152, 160, 157] 202617 [136, 97, 137, 98, 135, 139] Filter i: 457749 [157, 160, 158, 161, 159, 156] 248970 [22, 23, 25, 26, 20, 21] 228020 [115, 112, 116, 113, 114, 155] 280217 [46, 47, 49, 50, 16, 45] 227984 [105, 102, 106, 98, 144, 103] Note that these data cannot be read with versions of the stack between the merger of DM-23584 (part of w_2020_08 ) and the merger of DM-24517 (soon to be part of w_2020_18 ).

              People

              • Assignee:
                erykoff Eli Rykoff
                Reporter:
                Parejkoj John Parejko
                Watchers:
                Eli Rykoff, John Parejko
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel