Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8701

Copy and ingest HSC Cosmos data

    Details

      Description

      Once RFC-266 is accepted, /gpfs/fs0/scratch/pprice/UH-Cosmos/*.fits needs to be copied into /datasets/hsc/raw/cosmos and ingested into /datasets/hsc/repo.

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Jason Alt [X], these folders in /datasets/hsc/ can be locked down once Paul Price is happy with them.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Jason Alt [X] , these folders in /datasets/hsc/ can be locked down once Paul Price is happy with them.
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            The ingestImages log is attached, in case it's useful.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - The ingestImages log is attached, in case it's useful.
            Hide
            price Paul Price added a comment -

            Looks good to me.

            pprice@lsst-dev01:/datasets/hsc/repo $ registryInfo.py /datasets/hsc/repo/ --field COSMOS -s
            filter  field                expTime visit
            HSC-G   COSMOS                7630.0 19180..19220:2
            HSC-I   COSMOS               21683.9 872..910:2^914..946:2^18184..18224:2^908060^908104^908106..908162:2^908166
            HSC-Y   COSMOS               20460.0 628..670:2^674..680:2^18476..18556:2
            HSC-Z   COSMOS                7230.0 18834..18874:2
            NB0921  COSMOS                2220.0 906120^906130^906132..906140:2
            pprice@lsst-dev01:/datasets/hsc/repo $ python -c 'from lsst.daf.persistence import Butler; print Butler("/datasets/hsc/repo").get("raw", visit=18184, ccd=49).getDimensions()'
            CameraMapper INFO: Unable to locate calibRegistry registry in root: /datasets/hsc/repo/CALIB/calibRegistry.sqlite3
            CameraMapper INFO: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3
            CameraMapper INFO: Loading Posix registry from /datasets/hsc/repo/CALIB
            (2144, 4241)
            

            Show
            price Paul Price added a comment - Looks good to me. pprice@lsst-dev01:/datasets/hsc/repo $ registryInfo.py /datasets/hsc/repo/ --field COSMOS -s filter field expTime visit HSC-G COSMOS 7630.0 19180..19220:2 HSC-I COSMOS 21683.9 872..910:2^914..946:2^18184..18224:2^908060^908104^908106..908162:2^908166 HSC-Y COSMOS 20460.0 628..670:2^674..680:2^18476..18556:2 HSC-Z COSMOS 7230.0 18834..18874:2 NB0921 COSMOS 2220.0 906120^906130^906132..906140:2 pprice@lsst-dev01:/datasets/hsc/repo $ python -c 'from lsst.daf.persistence import Butler; print Butler("/datasets/hsc/repo").get("raw", visit=18184, ccd=49).getDimensions()' CameraMapper INFO: Unable to locate calibRegistry registry in root: /datasets/hsc/repo/CALIB/calibRegistry.sqlite3 CameraMapper INFO: Unable to locate calibRegistry registry in current dir: ./calibRegistry.sqlite3 CameraMapper INFO: Loading Posix registry from /datasets/hsc/repo/CALIB (2144, 4241)
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Ha, I didn't realize there is a registryInfo.py in obs_subaru. That looks super useful!

            And searching on Jira led me to DM-4382.

            Thank you Paul.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Ha, I didn't realize there is a registryInfo.py in obs_subaru . That looks super useful! And searching on Jira led me to DM-4382 . Thank you Paul.
            Hide
            jalt Jason Alt [X] (Inactive) added a comment -

            Locked down raw/[cosmos], repo/[COSMOS|_mapper|registry.sqlite3]

            raw/newhorizons still awaiting organization.

            Show
            jalt Jason Alt [X] (Inactive) added a comment - Locked down raw/ [cosmos] , repo/ [COSMOS|_mapper|registry.sqlite3] raw/newhorizons still awaiting organization.

              People

              • Assignee:
                hchiang2 Hsin-Fang Chiang
                Reporter:
                price Paul Price
                Reviewers:
                Jason Alt [X] (Inactive), Paul Price
                Watchers:
                Hsin-Fang Chiang, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel