Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-24786

New component column in datastore is too small

    Details

    • Story Points:
      2
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      In DM-24288 the component name was added to the internal posix datastore record. Unfortunately it was length 16 and sqlite did not complain – this breaks databases that do check the length of strings. In particular "TransmissionCurve" exceeds the limit. Change the limit to 32 to fix this.

      Additionally we might want to add constraints back. Constraints were added previously in DM-18438 but subsequently removed when DatabaseDict was replaced with OpaqueData.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Constraints seem to be working now.

            One open question is whether this slows down sqlite too much. I guess we could have some type of global switch for enabling these constraints and only enable it when running tests.

            Show
            tjenness Tim Jenness added a comment - Constraints seem to be working now. One open question is whether this slows down sqlite too much. I guess we could have some type of global switch for enabling these constraints and only enable it when running tests.
            Hide
            jbosch Jim Bosch added a comment -

            Given that this is happening inside SQLite, it seems like it should be pretty fast compared relative to all of the Python preprocessing we regularly do to prepare records for insert.  It's easy enough to comment out for a one-off profile to check that someday in the future if we're unhappy with SQLite ingest times.

            Show
            jbosch Jim Bosch added a comment - Given that this is happening inside SQLite, it seems like it should be pretty fast compared relative to all of the Python preprocessing we regularly do to prepare records for insert.  It's easy enough to comment out for a one-off profile to check that someday in the future if we're unhappy with SQLite ingest times.
            Hide
            tjenness Tim Jenness added a comment -

            Jenkins revealed that our abstract_filter column was too small as well so I also fixed that.

            Show
            tjenness Tim Jenness added a comment - Jenkins revealed that our abstract_filter column was too small as well so I also fixed that.

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Reviewers:
                Jim Bosch
                Watchers:
                Dino Bektesevic, Jim Bosch, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel