Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21302

Produce design sketch for ip_diffim updates for Gen3

    Details

    • Story Points:
      10
    • Sprint:
      AP F19-4 (2H September), AP F19-6 (November), AP S20-1 (December)
    • Team:
      Alert Production

      Description

      Use this opportunity to simplify and update the codebase as well as adding support for the Generation 3 middleware.

      This ticket is simply to think through and write down your ideas; we'll then schedule time to discuss them with Yusra before we put them into practice.

      We discussed this ticket on 2019-11-27 and agreed that the goal is to present initial design thoughts at the Brown Bag of 2019-12-04.

        Attachments

          Issue Links

            Activity

            Hide
            gkovacs Gabor Kovacs added a comment -

            During two brown bag sessions and further discussions we agreed on the following major transition path:

            • ImageDifferenceTask will be modified to be both Gen2 and Gen3 compatible; keeping the current functionality in Gen2 and doing only
              the absolutely necessary modifications for running under Gen3.
            • ImageDifferenceTask will be frozen at this point and will perhaps retire with Gen2. All the fuzziness and possible inconsistencies lying around will be kept as they are, with minimal maintenance only.
            • A new Gen3 only image differencing will be drafted from scratch. The necessity (use cases) and the decision logic behind many functional steps will be reviewed. Particularly, algorithmic steps that have been accumulated directly in top level code in the current ImageDifferenceTask should be re-factored and moved into their respective subtasks.
            • The new image differencing will be backward incompatible, having different design and config options. This implementation will rely on the Gen3 "pipeline" concept. New dataTypes will also be introduced to reflect incompatible processing products (e.g. zogy vs. al differences)
            Show
            gkovacs Gabor Kovacs added a comment - During two brown bag sessions and further discussions we agreed on the following major transition path: ImageDifferenceTask will be modified to be both Gen2 and Gen3 compatible; keeping the current functionality in Gen2 and doing only the absolutely necessary modifications for running under Gen3. ImageDifferenceTask will be frozen at this point and will perhaps retire with Gen2. All the fuzziness and possible inconsistencies lying around will be kept as they are, with minimal maintenance only. A new Gen3 only image differencing will be drafted from scratch. The necessity (use cases) and the decision logic behind many functional steps will be reviewed. Particularly, algorithmic steps that have been accumulated directly in top level code in the current ImageDifferenceTask should be re-factored and moved into their respective subtasks. The new image differencing will be backward incompatible , having different design and config options. This implementation will rely on the Gen3 "pipeline" concept. New dataTypes will also be introduced to reflect incompatible processing products (e.g. zogy vs. al differences)
            Hide
            gkovacs Gabor Kovacs added a comment -

            Attached mindmap snapshot as of 2019-12-18 about the principles, planned tasks and considerations.

            Show
            gkovacs Gabor Kovacs added a comment - Attached mindmap snapshot as of 2019-12-18 about the principles, planned tasks and considerations.
            Hide
            swinbank John Swinbank added a comment -

            Show
            swinbank John Swinbank added a comment -

              People

              • Assignee:
                gkovacs Gabor Kovacs
                Reporter:
                swinbank John Swinbank
                Reviewers:
                John Swinbank
                Watchers:
                Gabor Kovacs, John Swinbank
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: