Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-7445

DCR template generation speed improvements

    Details

    • Story Points:
      8
    • Sprint:
      Alert Production F16 - 9, Alert Production F16 - 10, Alert Production F16 - 11, Alert Production F16 - 11b, Alert Production F16 - 11c
    • Team:
      Alert Production

      Description

      The current DCR template generation code takes approximately 30s per pixel on a single core to run. This code needs to be optimized to run much faster.

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            This was all written using Python 2, but I'd be interested to know how close it is for compatibility with Python 3. These changes required some refactoring to make the speed improvements possible, so it touches a fair amount of the code. There is certainly additional room for speed and efficiency improvements, but it now runs in ~minutes on test data instead of ~days.

            Show
            sullivan Ian Sullivan added a comment - This was all written using Python 2, but I'd be interested to know how close it is for compatibility with Python 3. These changes required some refactoring to make the speed improvements possible, so it touches a fair amount of the code. There is certainly additional room for speed and efficiency improvements, but it now runs in ~minutes on test data instead of ~days.
            Hide
            Parejkoj John Parejko added a comment -

            See comments on github PR. Several cleanups, including some open questions about speed/memory, and a broader comment about testing.

            Show
            Parejkoj John Parejko added a comment - See comments on github PR. Several cleanups, including some open questions about speed/memory, and a broader comment about testing.
            Hide
            sullivan Ian Sullivan added a comment -

            The DCR template generation code now runs much faster on simulated data, and produces lower residuals after image differencing.

            Show
            sullivan Ian Sullivan added a comment - The DCR template generation code now runs much faster on simulated data, and produces lower residuals after image differencing.

              People

              • Assignee:
                sullivan Ian Sullivan
                Reporter:
                sullivan Ian Sullivan
                Reviewers:
                John Parejko
                Watchers:
                Ian Sullivan, John Parejko
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: