Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3274

KPM Measurement: Computational Performance Metrics: OTT1, FY15

    Details

    • Type: Epic
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Epic Name:
      KPM: Computational Performance Metrics: OTT1, FY15
    • WBS:
      02C.03
    • Team:
      Alert Production
    • Cycle:
      Summer 2015

      Attachments

        Issue Links

          Activity

          Hide
          swinbank John Swinbank added a comment -

          Closing per DM-CCB of 2019-04-24 — this epic is no longer aligned with DM V&V plans.

          Show
          swinbank John Swinbank added a comment - Closing per DM-CCB of 2019-04-24 — this epic is no longer aligned with DM V&V plans.
          Hide
          krughoff Simon Krughoff added a comment - - edited

          I believe we have shown that this has been measured and is within the 240 sec expected. There were two methods taken to reach this, but they both, I believe, show that the KPM has been met.

          Assumptions:
          1. SFM, image differencing, and difference source measurement dominate the runtime of any given frame.
          2. Scaling to LSST is approximately by the ratio of pixels in LSST to the instrument used for the estimate.
          3. Runtimes for the same algorithms on a single core will decrease between now and commissioning.

          Measurements:
          1. SFM measured on lsst-dev (3.1 GHz intel Xeon) using a full focalplane of HSC data at high galactic latitude. The result is 22 sec/chip. Chips are 2K x 4K.
          2. Image differencing with no measurement on a ~2008 MacBook Pro using HITS data from DECam. The result is 60 sec/chip. Chips are 2K x 4K.
          3. Image differencing + difference source measurement using the diffim task on a ~2008 MacBook Pro using HITS data from DECam. The result is 105 sec/chip.
          4. Image differencing + difference source measurement using the diffim task on lsst-dev (3.1 GHz intel Xeon). The result is 36 sec/chip. Chips are 2Kx4K.

          Results:
          The runtime of the diffim task is suspiciously long given that image differencing + measurement should be no longer than image differencing alone and SFM. Even though the two machines used to measure the timings are not the same, the difference in clock speed does not explain the nearly 2x increase in runtime for measurement.

          1. If we don't believe the diffim task runtime,
          runtime_for_LSST = (SFM_runtime + differencing_runtime + SFM_runtime)*LSST_pix/TESTDATA_pix
          runtime_for_LSST = (22 + 60 + 22)*2 = 208 sec
          Even if the rest of the pipeline takes 20% longer to run, we still are meeting the runtime on 2015 hardware which clearly should meet the runtime on ~2020 hardware.

          2. If we take the diffim task runtime at face value,
          runtime_for_LSST = (SFM_runtime + diffimTask_runtim)*LSST_pix/TESTDATA_pix
          runtime_for_LSST = (22 + 105)*2 = 254 sec
          Though there is no overhead for other parts of the pipeline, I think this is an encouraging number since there are most likely optimizations we can make and improvement in clock speed will gain something back on ~2020 hardware.

          *3. Taking SFM and DiffIm+DiffIm_meas as measured on lsst-dev. This should be the most reliable.
          runtime_for_LSST = (SFM_runtime + diffimTask_lsst_dev_runtime)*LSST_pix/TESTDATA_pix
          runtime_for_LSST = (22 + 35)*2 = 114
          With the most expensive parts of the processing coming in at a little under half of the budgeted runtime, this is a good indication that we are meeting this KPM on 2015 hardware.

          I submit that this KPM has been measured to pass in Summer 2015.

          Show
          krughoff Simon Krughoff added a comment - - edited I believe we have shown that this has been measured and is within the 240 sec expected. There were two methods taken to reach this, but they both, I believe, show that the KPM has been met. Assumptions: 1. SFM, image differencing, and difference source measurement dominate the runtime of any given frame. 2. Scaling to LSST is approximately by the ratio of pixels in LSST to the instrument used for the estimate. 3. Runtimes for the same algorithms on a single core will decrease between now and commissioning. Measurements: 1. SFM measured on lsst-dev (3.1 GHz intel Xeon) using a full focalplane of HSC data at high galactic latitude. The result is 22 sec/chip. Chips are 2K x 4K. 2. Image differencing with no measurement on a ~2008 MacBook Pro using HITS data from DECam. The result is 60 sec/chip. Chips are 2K x 4K. 3. Image differencing + difference source measurement using the diffim task on a ~2008 MacBook Pro using HITS data from DECam. The result is 105 sec/chip. 4. Image differencing + difference source measurement using the diffim task on lsst-dev (3.1 GHz intel Xeon). The result is 36 sec/chip. Chips are 2Kx4K. Results: The runtime of the diffim task is suspiciously long given that image differencing + measurement should be no longer than image differencing alone and SFM. Even though the two machines used to measure the timings are not the same, the difference in clock speed does not explain the nearly 2x increase in runtime for measurement. 1. If we don't believe the diffim task runtime, runtime_for_LSST = (SFM_runtime + differencing_runtime + SFM_runtime)*LSST_pix/TESTDATA_pix runtime_for_LSST = (22 + 60 + 22)*2 = 208 sec Even if the rest of the pipeline takes 20% longer to run, we still are meeting the runtime on 2015 hardware which clearly should meet the runtime on ~2020 hardware. 2. If we take the diffim task runtime at face value, runtime_for_LSST = (SFM_runtime + diffimTask_runtim)*LSST_pix/TESTDATA_pix runtime_for_LSST = (22 + 105)*2 = 254 sec Though there is no overhead for other parts of the pipeline, I think this is an encouraging number since there are most likely optimizations we can make and improvement in clock speed will gain something back on ~2020 hardware. *3. Taking SFM and DiffIm+DiffIm_meas as measured on lsst-dev. This should be the most reliable. runtime_for_LSST = (SFM_runtime + diffimTask_lsst_dev_runtime)*LSST_pix/TESTDATA_pix runtime_for_LSST = (22 + 35)*2 = 114 With the most expensive parts of the processing coming in at a little under half of the budgeted runtime, this is a good indication that we are meeting this KPM on 2015 hardware. I submit that this KPM has been measured to pass in Summer 2015.

            People

            • Assignee:
              krughoff Simon Krughoff
              Reporter:
              krughoff Simon Krughoff
              Watchers:
              John Swinbank, Simon Krughoff
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel