Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17496

QuantumGraph generation hits SQLite join limit

    Details

    • Templates:
    • Story Points:
      6
    • Sprint:
      DRP S19-3
    • Team:
      Data Release Production

      Description

      Nate Lust reports that trying to build a single QuantumGraph for all of the PipelineTasks in ci_hsc runs into SQLite's hard-coded maximum of 64 joins a single query.  I can think of several ways to avoid this, many involving temporary tables (I know those work differently in Oracle; I think my ideas are compatible with that).

      For the demo we'll plan on just running the pipelines in three stages, for which QuantumGraph generation works fine.  If everything else is going extremely well early in the week I may take a stab at resolving this, too.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  jbosch Jim Bosch
                  Reporter:
                  jbosch Jim Bosch
                  Reviewers:
                  Andy Salnikov, Nate Lust
                  Watchers:
                  Andy Salnikov, Jim Bosch, Nate Lust
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  3 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel