Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18353

Add pre-computed values to parquet tables output by pipe_analysis scripts

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_analysis
    • Labels:
      None
    • Templates:
    • Story Points:
      6
    • Epic Link:
    • Sprint:
      DRP S19-4, DRP S19-5
    • Team:
      Data Release Production

      Description

      In order to facilitate the interactive/dashboard QA work of Tim Morton, currently, the pipe_analysis scripts have the option to write out parquet tables for the various datasets at the visit and tract level.  Currently, they only contain the information in the source catalogs plus an extra column to indicate the deemed suitability of a given source for QA analyses, and columns to indicate the ccd & visit or patch & tract per object.  It would also be of great use for the interactive plotting to have a set of columns with pre-computed values of interest for plotting (e.g. mag differences, reference catalog comparisons, etc.) These are also the per-object values from which various (aggregated) metrics will be computed.  The strategy for now will be to add a column to the parquet tables for any value that is computed and plotted in the pipe_analysis scripts.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                lauren Lauren MacArthur
                Reviewers:
                Tim Morton
                Watchers:
                Lauren MacArthur, Tim Morton, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel