Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18546

Enable fastparquet as a read option for ParquetTable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Story Points:
      8
    • Epic Link:
    • Team:
      Data Release Production

      Description

      As identified in DM-18353, there are some reading bugs with pyarrow that prevent "large" (don't know what this means exactly) parquet files from being read. Experimentation has shown that fastparquet can read these files successfully. This ticket is to implement an option to read a ParquetTable using fastparquet instead of pyarrow.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tmorton Tim Morton
                Reporter:
                tmorton Tim Morton
                Watchers:
                John Swinbank, Tim Morton, Yusra AlSayyad
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel