Details
Description
The missing (or desired) fields are denoted by # FIXME
in the generation script for ObsCore data for images:
https://github.com/lsst/dax_imgserv/blob/tickets/DM-21433/integration/ci_hsc/gen_img_obscore.py
Here're the FIXME line items:
r["target_name"] = ? # FIXME: to be scheduler field ID
r["obs_id"] = ? # FIXME: to be online OBSID
r["s_fov"] = ? # FIXME: fov - the diameter of a circle around s_ra, s_dec
For reference on the above fields, see Gregory Dubois-Felsmann's notes: https://confluence.lsstcorp.org/pages/viewpage.action?spaceKey=~gpdf&title=Satisfying+ObsCore+from+the+Gen3+Butler+Schema
As of now, the script succeeded in fetching all the other fields from the ci_hsc test dataset without loading the Exposure object of the corresponding FITS file into memory, taking about ~1 min (on my low-end PC) to process and generate 68 rows of ObsCore data.
FYI, the generated CVS raw output has been attached here as reference. This raw CVS output is to be combined with SQL DDL and DML scripts to insert the data into a local PostgreSQL database, which has been done with success, albeit using psql tool.
Attachments
Issue Links
Activity
| Field | Original Value | New Value |
|---|---|---|
| Description |
The missing (or desired) fields are denoted by # FIXME
in the generation script for ObsCore data for images: [https://github.com/lsst/dax_imgserv/blob/tickets/DM-21433/integration/ci_hsc/gen_img_obscore.py] As of now, the script succeeded in fetching all the other fields from the ci_hsc test dataset without loading the Exposure object of the corresponding FITS file into memory, taking about ~1 min (on my low-end PC) to process and generate 68 rows of ObsCore data. FYI, the generated CVS raw output has been attached here for reference. This raw CVS output is to be combined with SQL DDL script to insert the data into a local PostgreSQL database, which has been done with success, albeit using psql tool. |
The missing (or desired) fields are denoted by # FIXME
in the generation script for ObsCore data for images: [https://github.com/lsst/dax_imgserv/blob/tickets/DM-21433/integration/ci_hsc/gen_img_obscore.py] Here're the FIXME line items: r["target_name"] = ? # FIXME: to be scheduler field ID r["obs_id"] = ? # FIXME: to be online OBSID r["s_fov"] = ? # FIXME: fov - the diameter of a circle around s_ra, s_dec For reference on the above fields, see [~gpdf]'s notes: [https://confluence.lsstcorp.org/pages/viewpage.action?spaceKey=~gpdf&title=Satisfying+ObsCore+from+the+Gen3+Butler+Schema] As of now, the script succeeded in fetching all the other fields from the ci_hsc test dataset without loading the Exposure object of the corresponding FITS file into memory, taking about ~1 min (on my low-end PC) to process and generate 68 rows of ObsCore data. FYI, the generated CVS raw output has been attached here as reference. This raw CVS output is to be combined with SQL DDL script to insert the data into a local PostgreSQL database, which has been done with success, albeit using psql tool. |
| Description |
The missing (or desired) fields are denoted by # FIXME
in the generation script for ObsCore data for images: [https://github.com/lsst/dax_imgserv/blob/tickets/DM-21433/integration/ci_hsc/gen_img_obscore.py] Here're the FIXME line items: r["target_name"] = ? # FIXME: to be scheduler field ID r["obs_id"] = ? # FIXME: to be online OBSID r["s_fov"] = ? # FIXME: fov - the diameter of a circle around s_ra, s_dec For reference on the above fields, see [~gpdf]'s notes: [https://confluence.lsstcorp.org/pages/viewpage.action?spaceKey=~gpdf&title=Satisfying+ObsCore+from+the+Gen3+Butler+Schema] As of now, the script succeeded in fetching all the other fields from the ci_hsc test dataset without loading the Exposure object of the corresponding FITS file into memory, taking about ~1 min (on my low-end PC) to process and generate 68 rows of ObsCore data. FYI, the generated CVS raw output has been attached here as reference. This raw CVS output is to be combined with SQL DDL script to insert the data into a local PostgreSQL database, which has been done with success, albeit using psql tool. |
The missing (or desired) fields are denoted by # FIXME
in the generation script for ObsCore data for images: [https://github.com/lsst/dax_imgserv/blob/tickets/DM-21433/integration/ci_hsc/gen_img_obscore.py] Here're the FIXME line items: r["target_name"] = ? # FIXME: to be scheduler field ID r["obs_id"] = ? # FIXME: to be online OBSID r["s_fov"] = ? # FIXME: fov - the diameter of a circle around s_ra, s_dec For reference on the above fields, see [~gpdf]'s notes: [https://confluence.lsstcorp.org/pages/viewpage.action?spaceKey=~gpdf&title=Satisfying+ObsCore+from+the+Gen3+Butler+Schema] As of now, the script succeeded in fetching all the other fields from the ci_hsc test dataset without loading the Exposure object of the corresponding FITS file into memory, taking about ~1 min (on my low-end PC) to process and generate 68 rows of ObsCore data. FYI, the generated CVS raw output has been attached here as reference. This raw CVS output is to be combined with SQL DDL and DML scripts to insert the data into a local PostgreSQL database, which has been done with success, albeit using psql tool. |
| Assignee | Tim Jenness [ tjenness ] | Kenny Lo [ kennylo ] |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Labels | ObsCore |
| Resolution | Done [ 10000 ] | |
| Status | In Progress [ 3 ] | Done [ 10002 ] |