Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19807

Avoid savepoints in Gen3 ingest and bulk conversions

    Details

      Description

      The savepoints used to implement nested transactions are taking the bulk of the time in raw and calibration in Oracle.  See if we can implement lighter-weight, Python-side nested transactions for at least some operations.

        Attachments

          Issue Links

            Activity

            Hide
            jbosch Jim Bosch added a comment -

            It looks like it will be tricky to remove savepoints without first reimplementing operations that involve catching and ignoring certain constraint violations.   For SQLite, that involves DM-17419, and for Oracle, it involves using MERGE, I believe.

            Show
            jbosch Jim Bosch added a comment - It looks like it will be tricky to remove savepoints without first reimplementing operations that involve catching and ignoring certain constraint violations.   For SQLite, that involves DM-17419 , and for Oracle, it involves using MERGE, I believe.
            Hide
            jbosch Jim Bosch added a comment -

            I've decided that while this ticket is a good description of a problem, it doesn't reflect how we'll likely address that problem.  I'm zeroing the story points and blocking it on DM-21768, which will probably take care of it (if not, DM-21770 will).  We can then just mark this ticket as done when its blocker is.

            Show
            jbosch Jim Bosch added a comment - I've decided that while this ticket is a good description of a problem, it doesn't reflect how we'll likely address that problem.  I'm zeroing the story points and blocking it on DM-21768 , which will probably take care of it (if not, DM-21770 will).  We can then just mark this ticket as done when its blocker is.

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Watchers:
                Jim Bosch
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel