Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8414

Investigate alternatives to ingest JSON blobs into the SQUASH database

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can vary widely and we don’t want to to turn all the details into a relational model.

        Attachments

          Issue Links

            Activity

            No builds found.
            afausti Angelo Fausti created issue -
            afausti Angelo Fausti made changes -
            Field Original Value New Value
            Epic Link DM-6196 [ 24712 ]
            afausti Angelo Fausti made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            afausti Angelo Fausti made changes -
            Description SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can varie widely and we don’t want to to turn all the details into a relational model.
            Hide
            afausti Angelo Fausti added a comment -

            Implemented in the ticket branch tickets/DM-8414

            Technote was updated with associated documentation: http://sqr-009.lsst.io/en/latest/#setting-up-the-database

            Show
            afausti Angelo Fausti added a comment - Implemented in the ticket branch tickets/ DM-8414 Technote was updated with associated documentation: http://sqr-009.lsst.io/en/latest/#setting-up-the-database
            afausti Angelo Fausti made changes -
            Description SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can varie widely and we don’t want to to turn all the details into a relational model. SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can vary widely and we don’t want to to turn all the details into a relational model.
            afausti Angelo Fausti made changes -
            Description SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can vary widely and we don’t want to to turn all the details into a relational model. SQuaSH uses MySQL/MariaDB for the development and production databases, it is implemented through the Django-MySQL package to add specific features of MySQL/MariaDB to Django. In particular, we are interested in the JSON data type to store the validate_drp outputs. That seems very convenient as these outputs can vary widely and we don’t want to to turn all the details into a relational model.
            Hide
            afausti Angelo Fausti added a comment -

            Also created a community post to get more feedaback on this

            https://community.lsst.org/t/using-json-data-type-to-minimize-database-schema-changes/1440

            Show
            afausti Angelo Fausti added a comment - Also created a community post to get more feedaback on this https://community.lsst.org/t/using-json-data-type-to-minimize-database-schema-changes/1440
            Hide
            afausti Angelo Fausti added a comment -

            Changes pushed to ticket branch, pending tests with MariaDB 10.1+

            Show
            afausti Angelo Fausti added a comment - Changes pushed to ticket branch, pending tests with MariaDB 10.1+
            afausti Angelo Fausti made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Done [ 10002 ]
            afausti Angelo Fausti made changes -
            Link This issue is child task of DM-7043 [ DM-7043 ]
            afausti Angelo Fausti made changes -
            Link This issue relates to DM-6330 [ DM-6330 ]
            afausti Angelo Fausti made changes -
            Link This issue relates to DM-8890 [ DM-8890 ]

              People

              Assignee:
              afausti Angelo Fausti
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.