Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7445

DCR template generation speed improvements

    XMLWordPrintable

    Details

    • Story Points:
      8
    • Sprint:
      Alert Production F16 - 9, Alert Production F16 - 10, Alert Production F16 - 11, Alert Production F16 - 11b, Alert Production F16 - 11c
    • Team:
      Alert Production

      Description

      The current DCR template generation code takes approximately 30s per pixel on a single core to run. This code needs to be optimized to run much faster.

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            This was all written using Python 2, but I'd be interested to know how close it is for compatibility with Python 3. These changes required some refactoring to make the speed improvements possible, so it touches a fair amount of the code. There is certainly additional room for speed and efficiency improvements, but it now runs in ~minutes on test data instead of ~days.

            Show
            sullivan Ian Sullivan added a comment - This was all written using Python 2, but I'd be interested to know how close it is for compatibility with Python 3. These changes required some refactoring to make the speed improvements possible, so it touches a fair amount of the code. There is certainly additional room for speed and efficiency improvements, but it now runs in ~minutes on test data instead of ~days.
            Hide
            Parejkoj John Parejko added a comment -

            See comments on github PR. Several cleanups, including some open questions about speed/memory, and a broader comment about testing.

            Show
            Parejkoj John Parejko added a comment - See comments on github PR. Several cleanups, including some open questions about speed/memory, and a broader comment about testing.
            Hide
            sullivan Ian Sullivan added a comment -

            The DCR template generation code now runs much faster on simulated data, and produces lower residuals after image differencing.

            Show
            sullivan Ian Sullivan added a comment - The DCR template generation code now runs much faster on simulated data, and produces lower residuals after image differencing.

              People

              Assignee:
              sullivan Ian Sullivan
              Reporter:
              sullivan Ian Sullivan
              Reviewers:
              John Parejko
              Watchers:
              Ian Sullivan, John Parejko
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  CI Builds

                  No builds found.