Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17496

QuantumGraph generation hits SQLite join limit

    Details

    • Templates:
    • Story Points:
      6
    • Sprint:
      DRP S19-3
    • Team:
      Data Release Production

      Description

      Nate Lust reports that trying to build a single QuantumGraph for all of the PipelineTasks in ci_hsc runs into SQLite's hard-coded maximum of 64 joins a single query.  I can think of several ways to avoid this, many involving temporary tables (I know those work differently in Oracle; I think my ideas are compatible with that).

      For the demo we'll plan on just running the pipelines in three stages, for which QuantumGraph generation works fine.  If everything else is going extremely well early in the week I may take a stab at resolving this, too.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jbosch Jim Bosch
                Reporter:
                jbosch Jim Bosch
                Reviewers:
                Andy Salnikov, Nate Lust
                Watchers:
                Andy Salnikov, Jim Bosch, Nate Lust
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel