Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17495

Running PipelineTasks in parallel can lead to aborting on locks instead of blocking

    Details

    • Templates:
    • Story Points:
      4
    • Team:
      Data Release Production

      Description

      Running PipelineTasks with ctrl_mpexec with -j can lead to SQLite errors about the database being locked.  It's unclear whether it's just not blocking long enough or not blocking at all, and also unclear whether we need to fix this by adding our own blocking or somehow informing SQLAlchemy or SQLite to do it.

      (It's also possible we do have a genuine deadlock bug, but I doubt it.)

        Attachments

          Activity

            People

            • Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              Nate Lust
              Watchers:
              Jim Bosch, Nate Lust
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel