Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22505

Create run method in ImageDifferenceTask and move all functionality from runDataRef except butler usage

    Details

      Description

      ImageDifferenceTask will be migrated to support Gen3 middleware while remaining Gen2 executable. The goal is to create a minimal wrapper while keeping as much as possible unchanged in the code.

      The first step is to create a run() method that already takes all inputs from memory and there is no butler reference in the code from here.

        Attachments

          Issue Links

            Activity

            Hide
            sullivan Ian Sullivan added a comment -

            This looks like a good start, though I think you don't need to move quite as much code into runDataRef as you do in imageDifference.py. I had a few other suggestions for moving code blocks out of run and into new methods, but that might be beyond the scope of this ticket. I don't think it would be much work, but I will leave it up to you whether to include that now or later.

            Are there really no unit tests of ImageDifferenceTask? In the absence of tests you will need to run image differencing both on this branch and on master with different settings: subtract='AL'/'zogy'doSelectSources=True/False, doUseRegister=True/FalsedoMatchSources=True/False, doDetection=True/False. You don't need to run every combination between all these settings, but it would be good to do a run with each one turned on and off with the rest left at their defaults. 

            Show
            sullivan Ian Sullivan added a comment - This looks like a good start, though I think you don't need to move quite as much code into runDataRef as you do in imageDifference.py. I had a few other suggestions for moving code blocks out of run and into new methods, but that might be beyond the scope of this ticket. I don't think it would be much work, but I will leave it up to you whether to include that now or later. Are there really no unit tests of ImageDifferenceTask ? In the absence of tests you will need to run image differencing both on this branch and on master with different settings: subtract='AL'/'zogy' ,  doSelectSources=True/False , doUseRegister=True/False ,  doMatchSources=True/False , doDetection=True/False . You don't need to run every combination between all these settings, but it would be good to do a run with each one turned on and off with the rest left at their defaults. 
            Hide
            gkovacs Gabor Kovacs added a comment - - edited

            Run imageDifference.py in 6 scenarios, see attachments for cmd lines, configs and logs. Number of sources in log messages in all scenarios and output catalog entries were compared in scenario 1. (Outputs are always binary different due to having processing time/memory, etc. metadata and product fingerprint version and config info included.)

            Also, ap_pipe.py was run with its defaults on one decam visit.

            All review comments addressed, some postponed as TODO comments as DM-22762.

            Show
            gkovacs Gabor Kovacs added a comment - - edited Run imageDifference.py in 6 scenarios, see attachments for cmd lines, configs and logs. Number of sources in log messages in all scenarios and output catalog entries were compared in scenario 1. (Outputs are always binary different due to having processing time/memory, etc. metadata and product fingerprint version and config info included.) Also, ap_pipe.py was run with its defaults on one decam visit. All review comments addressed, some postponed as TODO comments as DM-22762 .

              People

              • Assignee:
                gkovacs Gabor Kovacs
                Reporter:
                gkovacs Gabor Kovacs
                Reviewers:
                Ian Sullivan
                Watchers:
                Gabor Kovacs, Ian Sullivan
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: