Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-705

Add Decam galactic bulge survey to lsst-dev:/datasets/

    Details

    • Type: RFC
    • Status: Adopted
    • Resolution: Unresolved
    • Component/s: DM

      Description

      I am testing our pipelines on crowded fields, and wish to make the datasets generally available on lsst-dev. The HSC New Horizons observations already exist in /datasets/hsc/ on lsst-dev, but I will also be using the galactic bulge survey from DECam done by Saha et. al. The data consists of observations in six fields and five filters across two years (2013 and 2015), though I am proposing moving only the two densest fields (B1 and B2) and two filters (g and i bands). The directory /datasets/decam/ already exists, so to avoid confusion I propose to put the new data in /datasets/decam_saha/.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Is this data not already included in the `/datasets/decam` repository? I see `_internal/raw/saha_bulge/` in that directory.

            Show
            Parejkoj John Parejko added a comment - Is this data not already included in the `/datasets/decam` repository? I see `_internal/raw/saha_bulge/` in that directory.
            Hide
            ebellm Eric Bellm added a comment - - edited

            Yes, we should put the data in the same place as the other DECam datasets (/datasets/decam/_internal/raw). (It looks like only a subset of the Saha Bulge data that Ian wants is currently there.)

            I note that the _internal path comes via RFC-308; the ticket to document this in the developer guide is unassigned and still open: DM-14098

            Show
            ebellm Eric Bellm added a comment - - edited Yes, we should put the data in the same place as the other DECam datasets ( /datasets/decam/_internal/raw ). (It looks like only a subset of the Saha Bulge data that Ian wants is currently there.) I note that the _internal path comes via RFC-308 ; the ticket to document this in the developer guide is unassigned and still open: DM-14098
            Hide
            sullivan Ian Sullivan added a comment -

            I had not thought to look in _internal. That does already contain a small amount of data from the Saha galactic bulge survey, but not the set I am interested in. I will plan to add the new data to /datasets/decam/_internal/raw/saha_bulge/ rather than my originally suggested location.

            Show
            sullivan Ian Sullivan added a comment - I had not thought to look in _internal . That does already contain a small amount of data from the Saha galactic bulge survey, but not the set I am interested in. I will plan to add the new data to /datasets/decam/_internal/raw/saha_bulge/ rather than my originally suggested location.
            Hide
            mrawls Meredith Rawls added a comment -

            John beat me to it, but OK, makes sense that that's not all the data you need. I had never fully realized that `_internal` meant non-Butler format, but that's as sensible as anything... and I guess raw is raws and preprocessed is instcals.

            So, I agree with Eric that for now, your data should land in /datasets/decam/_internal/raw. Please do update the README in that same directory when you add more data.

            Please put the corresponding calibs somewhere sensible too... perhaps in /datasets/decam/_internal/calib and call it something like cpSaha_bulge (if mastercals) / saha_bulge (if raw calibs); or cp20yymm (if mastercals) / 20yymm (non-mastercals).

            At some point we should probably make a nice main root decam data directory at /datasets/decam,  parallel to /datasets/hsc, but that will require re-ingesting a whole load of stuff, and is probably outside the scope of this RFC.

            Show
            mrawls Meredith Rawls added a comment - John beat me to it, but OK, makes sense that that's not all the data you need. I had never fully realized that `_internal` meant non-Butler format, but that's as sensible as anything... and I guess raw is raws and preprocessed is instcals. So, I agree with Eric that for now, your data should land in /datasets/decam/_internal/raw. Please do update the README in that same directory when you add more data. Please put the corresponding calibs somewhere sensible too... perhaps in /datasets/decam/_internal/calib and call it something like cpSaha_bulge (if mastercals) / saha_bulge (if raw calibs); or cp20yymm (if mastercals) / 20yymm (non-mastercals). At some point we should probably make a nice main root decam data directory at /datasets/decam,  parallel to /datasets/hsc, but that will require re-ingesting a whole load of stuff, and is probably outside the scope of this RFC.
            Hide
            Parejkoj John Parejko added a comment -

            It hasn't been linked yet, so I'll note the common dataset policy regarding format and documentation: https://developer.lsst.io/services/datasets.html

            Show
            Parejkoj John Parejko added a comment - It hasn't been linked yet, so I'll note the common dataset policy regarding format and documentation: https://developer.lsst.io/services/datasets.html
            Hide
            sullivan Ian Sullivan added a comment -

            As suggested, I will add the new data to /datasets/decam/_internal/raw and the calibration products to /datasets/decam/_internal/calib.

            Show
            sullivan Ian Sullivan added a comment - As suggested, I will add the new data to /datasets/decam/_internal/raw and the calibration products to /datasets/decam/_internal/calib.

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Watchers:
                Eric Bellm, Ian Sullivan, John Parejko, John Swinbank, Meredith Rawls
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Planned End:

                  Summary Panel