Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-13770

Write a stand-alone task to convert/consolidate source tables to parquet files for QA

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Story Points:
      6
    • Sprint:
      DRP S18-4, DRP S18-5
    • Team:
      Data Release Production

      Description

      Currently, pipe_analysis scripts have an optional writeParquetOnly flag to skip making plots and only write the parquet tables. It might be desirable to factor that functionality out into a separate task. One thing that might be interesting to test as part of this work would be what the performance is like if catalogs are just written at the patch level instead of consolidating all requested patches to a single full-tract catalog.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  tmorton Tim Morton
                  Reporter:
                  tmorton Tim Morton
                  Reviewers:
                  John Parejko
                  Watchers:
                  Hsin-Fang Chiang, Jim Bosch, John Parejko, John Swinbank, Tim Morton, Yusra AlSayyad
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  6 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel