Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18624

speed up Table pickling

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw
    • Templates:
    • Story Points:
      2
    • Sprint:
      AP S19-4, AP S19-5
    • Team:
      Alert Production

      Description

      With Schema pickleable (DM-17950), I can now use ProcessPools to parallelize reading data. But pickle/unpickle of SourceCatalogs seems to be preventing that from actually providing a speed boost. I've found at least one obvious place to gain a big chunk of speed. I'll spend a little more time looking for others.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  Parejkoj John Parejko
                  Reporter:
                  Parejkoj John Parejko
                  Reviewers:
                  Nate Lust
                  Watchers:
                  Jim Bosch, John Parejko, John Swinbank, Nate Lust
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel