Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-474

Document how to create an astrometry_net_data repository

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsstSim
    • Labels:
    • Story Points:
      6
    • Sprint:
      DevOps Sprint 1, DevOps Sprint 2
    • Team:
      SQuaRE

      Description

      Document how to create astrometry_net_data index files for an arbitrary astronomical dataset, as well as how to create a repository of such data to support processing with the LSST Stack. It must be possible to limit the area of sky coverage in the index files to that appropriate for the images that the user wishes to process.

        Attachments

          Activity

          Hide
          price Paul Price added a comment -

          Is it worth mentioning hpsplit? Its use seems important for large catalogues.

          Show
          price Paul Price added a comment - Is it worth mentioning hpsplit? Its use seems important for large catalogues.
          Hide
          krughoff Simon Krughoff added a comment -

          Good point about hpsplit. It's mentioned in the astrometry.net docs, but not called out specifically in the LSST page.

          Show
          krughoff Simon Krughoff added a comment - Good point about hpsplit. It's mentioned in the astrometry.net docs, but not called out specifically in the LSST page.
          Hide
          robyn Robyn Allsman [X] (Inactive) added a comment -

          There is some generic documentation on index building; no need to repeat
          that part. Really the only LSST-specific part is re-using the same star
          trees (via "build-index -1").

          http://astrometry.net/doc/build-index.html

          cheers,
          --dstn

          Simon Krughoff<https://jira.lsstcorp.org/secure/ViewProfile.jspa?name=krughoff>

          Show
          robyn Robyn Allsman [X] (Inactive) added a comment - There is some generic documentation on index building; no need to repeat that part. Really the only LSST-specific part is re-using the same star trees (via "build-index -1"). http://astrometry.net/doc/build-index.html cheers, --dstn Simon Krughoff< https://jira.lsstcorp.org/secure/ViewProfile.jspa?name=krughoff >
          Hide
          shaw Richard Shaw [X] (Inactive) added a comment -

          Simon, thanks for the review. Responses below:

          1. Links to the astrometry.net website already existed under the subheadings:

          • Build a FITS Binary Table (one link)
          • Build Indexes (3 links)

          Another was added to the new info block (Split the Sky into Pieces) in:

          • Build Indexes

          Where else would you recommend links?

          2. Updated download command to curl.

          3. The scripts are now also attached to the page and links to download the files are embedded in the text.

          4. Regrettably, a "hide source" feature is not offered in Confluence.

          5. Files could be compressed in a variety of formats, and the supporting libraries for the various MIME types do not appear to be uniformly rich. The suggested capability may be added in a future rev of this page.

          6. Example of andConfig.py added.

          7. (from Paul) hpsplit is now mentioned in an Info block. It is not necessary for the catalog used in this tutorial, and would clutter the narrative to explain it fully. So I've mostly deferred to the Astrometry.net page. I can always embellish this tutorial later if users stumble on this point.

          Show
          shaw Richard Shaw [X] (Inactive) added a comment - Simon, thanks for the review. Responses below: 1. Links to the astrometry.net website already existed under the subheadings: Build a FITS Binary Table (one link) Build Indexes (3 links) Another was added to the new info block (Split the Sky into Pieces) in: Build Indexes Where else would you recommend links? 2. Updated download command to curl. 3. The scripts are now also attached to the page and links to download the files are embedded in the text. 4. Regrettably, a "hide source" feature is not offered in Confluence. 5. Files could be compressed in a variety of formats, and the supporting libraries for the various MIME types do not appear to be uniformly rich. The suggested capability may be added in a future rev of this page. 6. Example of andConfig.py added. 7. (from Paul) hpsplit is now mentioned in an Info block. It is not necessary for the catalog used in this tutorial, and would clutter the narrative to explain it fully. So I've mostly deferred to the Astrometry.net page. I can always embellish this tutorial later if users stumble on this point.
          Hide
          krughoff Simon Krughoff added a comment -

          Re: bullet #1

          I don't know how I missed those. Thanks for pointing out their existence. I don't think anything others are necessary.

          Show
          krughoff Simon Krughoff added a comment - Re: bullet #1 I don't know how I missed those. Thanks for pointing out their existence. I don't think anything others are necessary.

            People

            • Assignee:
              shaw Richard Shaw [X] (Inactive)
              Reporter:
              shaw Richard Shaw [X] (Inactive)
              Reviewers:
              Simon Krughoff
              Watchers:
              Dustin Lang [X] (Inactive), Mario Juric, Paul Price, Richard Shaw [X] (Inactive), Simon Krughoff
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel