Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-264

Move SDSS DR9 reference catalogs to /datasets

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Colin Slater has sharded the SDSS DR9 catalogs. I'd like to move them into the standard spot on /datasets which currently is refcats.

      As noted, this would end up being /datasets/refcats/htm/sdss_dr9.

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8672 /datasets/refcats/sdss_dr9 (source: /scratch/ctslater/sdss_dr9_refcat/*)

          • Done

            Activity

            Ascending order - Click to sort in descending order
            No builds found.
            krughoff Simon Krughoff created issue -
            krughoff Simon Krughoff made changes -
            Field Original Value New Value
            Link This issue is triggering DM-8672 [ DM-8672 ]
            Hide
            Permalink
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            Colin Slater has sharded the SDSS DR9 catalogs

            Does this mean that the catalog is in Qserv?

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - Colin Slater has sharded the SDSS DR9 catalogs Does this mean that the catalog is in Qserv?
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            No, they are not in Qserv. This is a simple on disk database for the reference catalogs where each shard contains the lines from the reference in a particular HTM trixel.

            Show
            krughoff Simon Krughoff added a comment - No, they are not in Qserv. This is a simple on disk database for the reference catalogs where each shard contains the lines from the reference in a particular HTM trixel.
            Hide
            Permalink
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            What database do you use? What software do you use to make the shards, search the data? I am asking because the private data storage we want to do for users. SUIT wants to save and search those data for users. We may explore SQLite.

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - What database do you use? What software do you use to make the shards, search the data? I am asking because the private data storage we want to do for users. SUIT wants to save and search those data for users. We may explore SQLite.
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            We are not using an RDBMS. It is a database in a generic sense. In other words, it's just a bunch of files stored on disk. We shard the input catalog using code I wrote in the DM stack using a python library written by Erin Sheldon (esutil).

            Show
            krughoff Simon Krughoff added a comment - We are not using an RDBMS. It is a database in a generic sense. In other words, it's just a bunch of files stored on disk. We shard the input catalog using code I wrote in the DM stack using a python library written by Erin Sheldon (esutil).
            Hide
            Permalink
            krughoff Simon Krughoff added a comment -

            Marking as adopted since there were no arguments.

            Show
            krughoff Simon Krughoff added a comment - Marking as adopted since there were no arguments.
            krughoff Simon Krughoff made changes -
            Resolution Done [ 10000 ]
            Status Proposed [ 10805 ] Adopted [ 10806 ]
            Hide
            Permalink
            jalt Jason Alt [X] (Inactive) added a comment -

            I believe the target directory is /datasets/refcats/htm/sdss_dr9. Slight derivation from the camera layouts but similar to the htm/gaia layout.

            Show
            jalt Jason Alt [X] (Inactive) added a comment - I believe the target directory is /datasets/refcats/htm/sdss_dr9. Slight derivation from the camera layouts but similar to the htm/gaia layout.
            krughoff Simon Krughoff made changes -
            Description [~ctslater] has sharded the SDSS DR9 catalogs. I'd like to move them into the standard spot on {{/datasets}} which currently is {{refcats}}. [~ctslater] has sharded the SDSS DR9 catalogs. I'd like to move them into the standard spot on {{/datasets}} which currently is {{refcats}}.

            As noted, this would end up being {{/datasets/refcats/htm/sdss_dr9}}.
            Hide
            Permalink
            hchiang2 Hsin-Fang Chiang added a comment -

            This was renamed to /datasets/refcats/htm/sdss-dr9-fink-v5b/ and immutability has been applied.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - This was renamed to /datasets/refcats/htm/sdss-dr9-fink-v5b/ and immutability has been applied.
            hchiang2 Hsin-Fang Chiang made changes -
            Status Adopted [ 10806 ] Implemented [ 11105 ]

              People

              Assignee:
              krughoff Simon Krughoff
              Reporter:
              krughoff Simon Krughoff
              Watchers:
              Hsin-Fang Chiang, Jason Alt [X] (Inactive), Simon Krughoff, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.