Details

    • Type: Technical task
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_astrom
    • Labels:
      None
    • Story Points:
      4
    • Sprint:
      Science Pipelines DM-S15-3
    • Team:
      Alert Production

      Description

      Dominique Boutigny has demonstrated that one reason the new astrometry task is working so poorly is that it is not selective enough about which sources it uses. This ticket is to be used to improve that situation.

      Another problem Dominique discovered is that the TAN-SIP WCS fitter needs to be iterated to work properly, and that work may also be done on this ticket.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Filed issue DM-2775 to deal with the fitting iteration.

            Show
            price Paul Price added a comment - Filed issue DM-2775 to deal with the fitting iteration.
            Hide
            rowen Russell Owen added a comment -

            Dominique Boutigny raised these additional points on HipChat in the CFHT reprocessing room:
            1- I think that it is better to leave out the saturated sources
            2- It would be better to be able to pass the list of bad flags as a configurable parameter. It would then be easier to test various combinations
            3- I don't understand how you deal with rotations in matchOptimisticB.py

            Show
            rowen Russell Owen added a comment - Dominique Boutigny raised these additional points on HipChat in the CFHT reprocessing room: 1- I think that it is better to leave out the saturated sources 2- It would be better to be able to pass the list of bad flags as a configurable parameter. It would then be easier to test various combinations 3- I don't understand how you deal with rotations in matchOptimisticB.py
            Hide
            rowen Russell Owen added a comment -

            Here are my responses to Dominique's comments:
            1 - You are right. I accidentally removed the step of culling non-good sources before running the WCS fitter. I will fix that. A test showed that by culling saturated sources we reached 18.6 mas in 3 iterations of the outer loop, which is fewer iterations than before. Culling sources that have been interpolated in their center, as Paul Price suggested, does just as well and is probably superior so I'll do that. That catches saturated sources, cosmic rays, bad pixels, etc.
            2 - I would love to make this more configurable, but it's tricky because some tests rely on more than fields that are not flags. Instead I made SourceInfo configurable by subclassing. And of course for quick tests one can simply edit the code.
            3 - This is in reference to the loop in MatchOptimisticBTask, and I am not happy about it either. Vello Tabur said it was a poor idea to match repeatedly with broader and broader parameters, but rather to tighten parameters after getting an initial match with loose parameters. I think the outer loop in AstrometryTask does a nice job of tightening parameters so I plan to try eliminating the loop in MatchOptimisticBTask.

            Show
            rowen Russell Owen added a comment - Here are my responses to Dominique's comments: 1 - You are right. I accidentally removed the step of culling non-good sources before running the WCS fitter. I will fix that. A test showed that by culling saturated sources we reached 18.6 mas in 3 iterations of the outer loop, which is fewer iterations than before. Culling sources that have been interpolated in their center, as Paul Price suggested, does just as well and is probably superior so I'll do that. That catches saturated sources, cosmic rays, bad pixels, etc. 2 - I would love to make this more configurable, but it's tricky because some tests rely on more than fields that are not flags. Instead I made SourceInfo configurable by subclassing. And of course for quick tests one can simply edit the code. 3 - This is in reference to the loop in MatchOptimisticBTask, and I am not happy about it either. Vello Tabur said it was a poor idea to match repeatedly with broader and broader parameters, but rather to tighten parameters after getting an initial match with loose parameters. I think the outer loop in AstrometryTask does a nice job of tightening parameters so I plan to try eliminating the loop in MatchOptimisticBTask.
            Hide
            rowen Russell Owen added a comment -

            I tried simplifying the loop in MatchOptimisticBTask but the results failed on CFHT visit=850587 ccd=21 (which is usually the one that fails if any of the CFHT demo CCDs fail). My conclusion: this fixing (3) will take much more thought and care. Best to leave it alone for now.

            Show
            rowen Russell Owen added a comment - I tried simplifying the loop in MatchOptimisticBTask but the results failed on CFHT visit=850587 ccd=21 (which is usually the one that fails if any of the CFHT demo CCDs fail). My conclusion: this fixing (3) will take much more thought and care. Best to leave it alone for now.
            Hide
            rowen Russell Owen added a comment -

            I tried further orthogonalizing the commits but gave up in frustration. I had already spent 4 hours cleaning up the commit history and I think it'll have to do.

            Otherwise I think I addressed the concerns. The results are quite good on the CFHT data (better than our old task) but still need to be proven on more distorted fields before I am comfortable making the new astrometry task the default in pipe_tasks.

            Show
            rowen Russell Owen added a comment - I tried further orthogonalizing the commits but gave up in frustration. I had already spent 4 hours cleaning up the commit history and I think it'll have to do. Otherwise I think I addressed the concerns. The results are quite good on the CFHT data (better than our old task) but still need to be proven on more distorted fields before I am comfortable making the new astrometry task the default in pipe_tasks.

              People

              Assignee:
              rowen Russell Owen
              Reporter:
              rowen Russell Owen
              Reviewers:
              Paul Price
              Watchers:
              Dominique Boutigny, Kian-Tat Lim, Mario Juric, Paul Price, Robert Lupton, Russell Owen, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: