Add Decam galactic bulge survey to lsst-dev:/datasets/

XMLWordPrintable

Details

• Type: RFC
• Status: Implemented
• Resolution: Done
• Component/s:
• Labels:

Description

I am testing our pipelines on crowded fields, and wish to make the datasets generally available on lsst-dev. The HSC New Horizons observations already exist in /datasets/hsc/ on lsst-dev, but I will also be using the galactic bulge survey from DECam done by Saha et. al. The data consists of observations in six fields and five filters across two years (2013 and 2015), though I am proposing moving only the two densest fields (B1 and B2) and two filters (g and i bands). The directory /datasets/decam/ already exists, so to avoid confusion I propose to put the new data in /datasets/decam_saha/.

Activity

Hide
John Parejko added a comment -

Is this data not already included in the /datasets/decam repository? I see _internal/raw/saha_bulge/ in that directory.

Show
John Parejko added a comment - Is this data not already included in the /datasets/decam repository? I see _internal/raw/saha_bulge/ in that directory.
Hide
Eric Bellm added a comment - - edited

Yes, we should put the data in the same place as the other DECam datasets (/datasets/decam/_internal/raw). (It looks like only a subset of the Saha Bulge data that Ian wants is currently there.)

I note that the _internal path comes via RFC-308; the ticket to document this in the developer guide is unassigned and still open: DM-14098

Show
Eric Bellm added a comment - - edited Yes, we should put the data in the same place as the other DECam datasets ( /datasets/decam/_internal/raw ). (It looks like only a subset of the Saha Bulge data that Ian wants is currently there.) I note that the _internal path comes via RFC-308 ; the ticket to document this in the developer guide is unassigned and still open: DM-14098
Hide
Ian Sullivan added a comment -

I had not thought to look in _internal. That does already contain a small amount of data from the Saha galactic bulge survey, but not the set I am interested in. I will plan to add the new data to /datasets/decam/_internal/raw/saha_bulge/ rather than my originally suggested location.

Show
Ian Sullivan added a comment - I had not thought to look in _internal . That does already contain a small amount of data from the Saha galactic bulge survey, but not the set I am interested in. I will plan to add the new data to /datasets/decam/_internal/raw/saha_bulge/ rather than my originally suggested location.
Hide
Meredith Rawls added a comment -

John beat me to it, but OK, makes sense that that's not all the data you need. I had never fully realized that _internal meant non-Butler format, but that's as sensible as anything... and I guess raw is raws and preprocessed is instcals.

So, I agree with Eric that for now, your data should land in /datasets/decam/_internal/raw. Please do update the README in that same directory when you add more data.

Please put the corresponding calibs somewhere sensible too... perhaps in /datasets/decam/_internal/calib and call it something like cpSaha_bulge (if mastercals) / saha_bulge (if raw calibs); or cp20yymm (if mastercals) / 20yymm (non-mastercals).

At some point we should probably make a nice main root decam data directory at /datasets/decam,  parallel to /datasets/hsc, but that will require re-ingesting a whole load of stuff, and is probably outside the scope of this RFC.

Show
Meredith Rawls added a comment - John beat me to it, but OK, makes sense that that's not all the data you need. I had never fully realized that _internal meant non-Butler format, but that's as sensible as anything... and I guess raw is raws and preprocessed is instcals. So, I agree with Eric that for now, your data should land in /datasets/decam/_internal/raw. Please do update the README in that same directory when you add more data. Please put the corresponding calibs somewhere sensible too... perhaps in /datasets/decam/_internal/calib and call it something like cpSaha_bulge (if mastercals) / saha_bulge (if raw calibs); or cp20yymm (if mastercals) / 20yymm (non-mastercals). At some point we should probably make a nice main root decam data directory at /datasets/decam,  parallel to /datasets/hsc, but that will require re-ingesting a whole load of stuff, and is probably outside the scope of this RFC.
Hide
John Parejko added a comment -

It hasn't been linked yet, so I'll note the common dataset policy regarding format and documentation: https://developer.lsst.io/services/datasets.html

Show
John Parejko added a comment - It hasn't been linked yet, so I'll note the common dataset policy regarding format and documentation: https://developer.lsst.io/services/datasets.html
Hide
Ian Sullivan added a comment -

As suggested, I will add the new data to /datasets/decam/_internal/raw and the calibration products to /datasets/decam/_internal/calib.

Show
Ian Sullivan added a comment - As suggested, I will add the new data to /datasets/decam/_internal/raw and the calibration products to /datasets/decam/_internal/calib.

People

Assignee:
Ian Sullivan
Reporter:
Ian Sullivan
Watchers:
Dino Bektesevic, Eric Bellm, Ian Sullivan, John Parejko, John Swinbank, Meredith Rawls
0 Vote for this issue
Watchers:
6 Start watching this issue

Dates

Created:
Updated:
Resolved:
Planned End:

CI Builds

No builds found.