Details
-
Type:
Story
-
Status: In Progress
-
Resolution: Unresolved
-
Fix Version/s: None
-
Component/s: daf_butler
-
Story Points:2
-
Epic Link:
-
Team:Data Release Production
-
Urgent?:No
Description
The Gen3 Registry's implied dimension dependencies (e.g. visit.physical_filter, exposure.visit) are allowed to be None, which translates nicely to NULL in SQL. Unfortunately SQL NULLs and Python Nones compare differently, and we want the latter behavior in SQL queries. That means we need to either
- add special NULL-handling logic to the query-generation code
- do our own mapping from Python None to sentinal values in all fetch/insert code
- drop support for None/NULL in implied dimensions and make the associated foreign key columns in the database NOT NULL to support this.
Attachments
Issue Links
Activity
| Labels | gen3-middleware | gen2-deprecation-blocker gen3-middleware |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
| Status | In Progress [ 3 ] | To Do [ 10001 ] |
| Summary | Handle NULL values for optional implied dependencies in query logic | Disallow NULL/None for implied dimension values in data IDs |
| Urgent? | off | |
| Description | The Gen3 Registry's implied dimension dependencies (e.g. visit.physical_filter, exposure.visit) are allowed to be None, which translates nicely to NULL in SQL. Unfortunately SQL NULLs and Python Nones compare differently, and we want the latter behavior in SQL queries. That means we need to either add special NULL-handling logic to the query-generation code, or do our own mapping from Python None to sentinal values in all fetch/insert code. Right now it feels like the former is cleaner, but I'm open to opinions. |
The Gen3 Registry's implied dimension dependencies (e.g. visit.physical_filter, exposure.visit) are allowed to be None, which translates nicely to NULL in SQL. Unfortunately SQL NULLs and Python Nones compare differently, and we want the latter behavior in SQL queries. That means we need to either
- add special NULL-handling logic to the query-generation code - do our own mapping from Python None to sentinal values in all fetch/insert code - drop support for None/NULL in implied dimensions and make the associated foreign key columns in the database NOT NULL to support this. |
| Labels | gen2-deprecation-blocker gen3-middleware | gen3-middleware gen3-registry-incompatibility |
| Status | To Do [ 10001 ] | In Progress [ 3 ] |