Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7985 Migration of scientific datasets
  3. DM-8116

/datasets/hsc/raw/newhorizons/ (source: /lsst8/newhorizons)

    XMLWordPrintable

    Details

    • Type: Technical task
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Team:
      Data Facility

      Description

      Copying /datasets/hsc/raw/newhorizons/ (source: /lsst8/newhorizons, owner: ctslater)

        Attachments

          Issue Links

          is child task of

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-7985 Migration of scientific datasets

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jalt Jason Alt [X] (Inactive) added a comment -

            One (unecessary) directory skipped:
            rsync: opendir "/nfs/lsst8/newhorizons/subaru2014/.Trashes" failed: Permission denied (13)

            Show
            jalt Jason Alt [X] (Inactive) added a comment - One (unecessary) directory skipped: rsync: opendir "/nfs/lsst8/newhorizons/subaru2014/.Trashes" failed: Permission denied (13)
            Hide
            Permalink
            jalt Jason Alt [X] (Inactive) added a comment -

            Done. Waiting on butlerization of data set before applying sharing/immutability

            Show
            jalt Jason Alt [X] (Inactive) added a comment - Done. Waiting on butlerization of data set before applying sharing/immutability
            Hide
            Permalink
            jalt Jason Alt [X] (Inactive) added a comment -

            using 'review' for state needing butlerization and/or organization of dataset before immutability

            Show
            jalt Jason Alt [X] (Inactive) added a comment - using 'review' for state needing butlerization and/or organization of dataset before immutability
            Hide
            Permalink
            jalt Jason Alt [X] (Inactive) added a comment -

            Colin to organize per rfc 249 and butler-ize

            Show
            jalt Jason Alt [X] (Inactive) added a comment - Colin to organize per rfc 249 and butler-ize
            Hide
            Permalink
            ctslater Colin Slater added a comment -

            I think this is done; images are available in /datasets/hsc/repo.

            Show
            ctslater Colin Slater added a comment - I think this is done; images are available in /datasets/hsc/repo .
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Colin Slater I don't think the newhorizons data are ingested into the butler repo /datasets/hsc/repo, what Jason meant by "butlerizing".

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Colin Slater I don't think the newhorizons data are ingested into the butler repo /datasets/hsc/repo , what Jason meant by "butlerizing".
            Hide
            Permalink
            ctslater Colin Slater added a comment -

            Looks like it's there to me?

            [ctslater@lsst-dev01 repo]$ ls /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/*02940* | head -n 3
            		 
            /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294000.fits
            		 
            /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294001.fits
            		 
            /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294002.fits


            [ctslater@lsst-dev01 repo]$ sqlite3 /datasets/hsc/repo/registry.sqlite3
            		 
            sqlite> select * from raw where visit=2940 limit 5;
            		 
            id|taiObs|expId|pointing|dataType|visit|dateObs|frameId|filter|field|pa|expTime|ccdTemp|ccd|proposal|config|autoguider
            		 
            205413|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294153|HSC-R|DEEPE02|0.0|320.0|-100.472|0|o14215|20140306.cfg|1
            		 
            207325|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294154|HSC-R|DEEPE02|0.0|320.0|-100.666|1|o14215|20140306.cfg|1
            		 
            216478|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294155|HSC-R|DEEPE02|0.0|320.0|-100.777|2|o14215|20140306.cfg|1
            		 
            201454|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294156|HSC-R|DEEPE02|0.0|320.0|-101.026|3|o14215|20140306.cfg|1
            		 
            198360|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294142|HSC-R|DEEPE02|0.0|320.0|-101.573|4|o14215|20140306.cfg|1

            Show
            ctslater Colin Slater added a comment - Looks like it's there to me? [ctslater@lsst-dev01 repo]$ ls /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/*02940* | head -n 3 /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294000.fits /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294001.fits /datasets/hsc/raw/newhorizons/subaru2014/RawNT1/HSCA00294002.fits [ctslater@lsst-dev01 repo]$ sqlite3 /datasets/hsc/repo/registry.sqlite3 sqlite> select * from raw where visit=2940 limit 5; id|taiObs|expId|pointing|dataType|visit|dateObs|frameId|filter|field|pa|expTime|ccdTemp|ccd|proposal|config|autoguider 205413|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294153|HSC-R|DEEPE02|0.0|320.0|-100.472|0|o14215|20140306.cfg|1 207325|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294154|HSC-R|DEEPE02|0.0|320.0|-100.666|1|o14215|20140306.cfg|1 216478|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294155|HSC-R|DEEPE02|0.0|320.0|-100.777|2|o14215|20140306.cfg|1 201454|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294156|HSC-R|DEEPE02|0.0|320.0|-101.026|3|o14215|20140306.cfg|1 198360|2014-06-25|HSCA00294000|906|OBJECT|2940|2014-06-25|HSCA00294142|HSC-R|DEEPE02|0.0|320.0|-101.573|4|o14215|20140306.cfg|1
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            Ha, apparently Paul did it. Sorry for the noise.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Ha, apparently Paul did it. Sorry for the noise.

              People

              Assignee:
              korrapa2 Harathi Korrapati [X] (Inactive)
              Reporter:
              jalt Jason Alt [X] (Inactive)
              Reviewers:
              Colin Slater
              Watchers:
              Colin Slater, Hsin-Fang Chiang, Jason Alt [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.