Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-30438

Add support for uploading data IDs to temporary tables and vectorize data ID expansion

    XMLWordPrintable

    Details

    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      General idea is to add a Registry interface (or at least a SqlRegistry interface) for uploading DataCoordinateIterable contents to a temporary table, returning a DataCoordinateQueryResults.  Being able to fully use those in arbitrary new queries or QG gen would be another ticket, but this would unblock a fully-vectorized version of expandDataIds - DM-30332 takes some steps towards that, but it can't really do vectorized fetches of DimensionRecords at scale because it's stuck encoding data IDs into WHERE clauses separated by OR.

      This is something I've been planning to do for a long time but didn't have a direct ticket for.  I'm creating a ticket now because I have some code on the DM-30332 that I don't want to try to finish there (an astropy.table-backed DataCoordinateIterable that would be useful for vectorized expandDataIds) , so I'm planning to move that to the branch for this ticket even if I don't start this ticket for a while.

        Attachments

          Issue Links

            Activity

            There are no comments yet on this issue.

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:

                  Jenkins

                  No builds found.