Data ManagementIcon indicating the project type

Data Management

Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-14538

Remove ci_dataset from job model

    XMLWordPrintable

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: squash
    • Labels:
      None

      Description

      The ci_dataset field was added to the job table for performance reasons when filtering the jobs by dataset. It is also present in the env JSON field.

      Consider the removal of this field from the job table:

      1) compare the performance when querying the ci_dataset field in the Job table and querying ci_dataset in the env JSON field.

      2) have in mind that when supporting multiple environments the name of this field may change

        Attachments

          Issue Links

          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15317 Implement model to register datasets in SQuaSH

          • To Do

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-18057 QAWG-REC-37: It must be possible to submit metrics to SQuaSH from arbitrary pipeline execution environments

          • To Do

            Activity

            Descending order - Click to sort in ascending order
            Hide
            Permalink
            afausti Angelo Fausti added a comment - - edited

            Hi Krzysztof Findeisen this ticket addresses an issue in the SQuaSH MySQL database model where ci_dataset is present in two different places.

            You can see for example how a verification job from ap_verify is stored in SQuaSH
            https://squash-restful-api.lsst.codes/job/3608

            This issue is known for some time and I would like to fix it as part of the work in DM-18057.

            But the answer is yes, ci_dataset will still be mapped to an InfluxDB tag along with other metadata and it will be available through the Chronograf UI for filtering etc.

            Show
            afausti Angelo Fausti added a comment - - edited Hi Krzysztof Findeisen this ticket addresses an issue in the SQuaSH MySQL database model where ci_dataset is present in two different places. You can see for example how a verification job from ap_verify is stored in SQuaSH https://squash-restful-api.lsst.codes/job/3608 This issue is known for some time and I would like to fix it as part of the work in DM-18057 . But the answer is yes, ci_dataset will still be mapped to an InfluxDB tag along with other metadata and it will be available through the Chronograf UI for filtering etc.
            Hide
            Permalink
            krzys Krzysztof Findeisen added a comment -

            Will we still be able to query a dataset ID from Chronograf after the ci_dataset tag is removed? It's not clear from the description, and I can't find an env in the online database.

            Show
            krzys Krzysztof Findeisen added a comment - Will we still be able to query a dataset ID from Chronograf after the ci_dataset tag is removed? It's not clear from the description, and I can't find an env in the online database.

              People

              • Assignee:
                afausti Angelo Fausti
                Reporter:
                afausti Angelo Fausti
                Watchers:
                Angelo Fausti, Krzysztof Findeisen
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Summary Panel