Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26937

Profile raw data ingest

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      We are going to want to ingest a lot of raw files so we should profile raw ingest to ensure there are no obvious slowdowns.

      testdata_lsst has 450 files and I have a copy locally so first try with that.

      Checksum calculation should be disabled (and possibly I should make that the default).

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-26943 Support newer headers in imsim data

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-27007 Disable checksum calculation by default in gen3 datastore

          • Done
          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-26662 ap_verify import slow in Gen 3

          • Done

            Activity

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Eric Neilsen, Hsin-Fang Chiang, James Chiang, Jim Bosch, Michelle Gower, Robert Gruendl [X] (Inactive), Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.