Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26937

Profile raw data ingest

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      We are going to want to ingest a lot of raw files so we should profile raw ingest to ensure there are no obvious slowdowns.

      testdata_lsst has 450 files and I have a copy locally so first try with that.

      Checksum calculation should be disabled (and possibly I should make that the default).

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-26943 Support newer headers in imsim data

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-27007 Disable checksum calculation by default in gen3 datastore

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-26662 ap_verify import slow in Gen 3

          • Done

            Activity

            Ascending order - Click to sort in descending order
            tjenness Tim Jenness created issue -
            tjenness Tim Jenness made changes -
            Field Original Value New Value
            Watchers Jim Bosch, Michelle Gower, Robert Gruendl, Tim Jenness [ Jim Bosch, Michelle Gower, Robert Gruendl, Tim Jenness ] Hsin-Fang Chiang, Jim Bosch, Michelle Gower, Robert Gruendl, Tim Jenness [ Hsin-Fang Chiang, Jim Bosch, Michelle Gower, Robert Gruendl, Tim Jenness ]
            hchiang2 Hsin-Fang Chiang made changes -
            Link This issue relates to PREOPS-102 [ PREOPS-102 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-26943 [ DM-26943 ]
            tjenness Tim Jenness made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            tjenness Tim Jenness made changes -
            Resolution Done [ 10000 ]
            Status In Progress [ 3 ] Done [ 10002 ]
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-27007 [ DM-27007 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to DM-26662 [ DM-26662 ]

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Eric Neilsen, Hsin-Fang Chiang, James Chiang, Jim Bosch, Michelle Gower, Robert Gruendl [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.