Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-268

Create a safe archive for the astrometry_net_data and other test datasets

    Details

    • Team:
      SQuaRE

      Description

      Eups package astrometry_net_data co-opted the individual versions to hold different unique datasets (i.e. not just different versions of the same dataset).

      Currently the different datasets are not backed-up nor are they installed in git repositories so the loss of a dataset thru misadventure or other failure requires the recreation of the dataset from whence it came.

      This Issue should deal with setting up a failsafe archival mechanism.

      Edit (by mjuric): As K-T points out in the first comment, this is true of all other test data sets. This issue is intended to cover the needed work for all of them.

        Attachments

          Issue Links

            Activity

            Hide
            ktl Kian-Tat Lim added a comment -

            Simon points out that camera geometry information, calibration data, and perhaps also raw data from precursor surveys also require a similar archive. If we build it right, it could also be used for discovering and sharing intermediate and result datasets from nightly, weekly, developer, mini, and full productions.

            Show
            ktl Kian-Tat Lim added a comment - Simon points out that camera geometry information, calibration data, and perhaps also raw data from precursor surveys also require a similar archive. If we build it right, it could also be used for discovering and sharing intermediate and result datasets from nightly, weekly, developer, mini, and full productions.
            Hide
            jmatt J Matt Peterson [X] (Inactive) added a comment -

            Completed through git-lfs and S3 GitHub backups.

            Show
            jmatt J Matt Peterson [X] (Inactive) added a comment - Completed through git-lfs and S3 GitHub backups.

              People

              • Assignee:
                ktl Kian-Tat Lim
                Reporter:
                robyn Robyn Allsman [X] (Inactive)
                Reviewers:
                J Matt Peterson [X] (Inactive)
                Watchers:
                J Matt Peterson [X] (Inactive), Kian-Tat Lim, Mario Juric
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel