Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_analysis
-
Labels:None
-
Story Points:24
-
Epic Link:
-
Sprint:DRP F19-6 (Nov), DRP S20-2 (Jan), DRP S20-3 (Feb), DRP S20-5 (Apr), DRP S20-6 (May), DRP F20-1 (June), DRP F20-2 (July), DRP F20-3 (Aug), DRP F20-5 (Oct)
-
Team:Data Release Production
-
Urgent?:No
Description
As a step towards converting the pipe_analysis scripts to Gen3, adapt the scripts in pipe_analysis to work with the parquet tables produced in the pipe_tasks postprocessing.py step. The reading of afwTables will be deprecated eventually, but we still leave the ability to do so for compatibility with older datasets for which the postprocessing.py step was not run (so no parquet catalog tables exist). As such, the first step here is to convert all functionality to be run with pandas DataFrames (so, if reading in afwTable catalogs, convert them immediately to pandas DataFrames). Then add a function to read in the *_obj parquet tables and add a config parameter to control which tables are read in (default it to read the parquet catalogs).