Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-6562

DoubleShapeletApproximation fails to fit Gaussian PSF

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_modelfit
    • Labels:
      None
    • Story Points:
      4
    • Epic Link:
    • Sprint:
      DRP F16-6
    • Team:
      Data Release Production

      Description

      When iterating a psf in a single frame processing for ci_hsc data-product visit 903338 ccd 18, the psf determiner generates a PSF for which a shapelet approximation cannot be found. Specifically, when the PSF determination stage iterates, it converts the determined PSF from the previous step into a Gaussian with an equivalent width. The double shapelet approximation fails to fit this Gaussian.

      This ticket should make the PSF fitting robust against cases like this. Additionally it should revert the changes in ci_hsc on the DM-4202 ticket branch which were put in as a temporary workaround for this problem.

      Attached to this ticket is the PSF that is failing, and a script which can reproduce the problem.

        Attachments

        1. badpsf2
          17 kB
        2. shapeFitFail.py
          0.7 kB

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. DM-6293 Fix error in cmodel related to computing LinearTransforms

          • Done
          mentioned in

          Page Loading...

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Nate Lust, could this just be a case where there's too much degeneracy for the solver? The coefficients that come out of fitMoments() are all zeros except for the first element for each of the two components.

            If I change testProfile() in your script to:

            def testProfile(exposure, ctrl):
            		 
                import numpy as np
            		 
                psf = exposure.getPsf()
            		 
                image = psf.computeKernelImage()
            		 
                result = Algorithm.initializeResult(ctrl)
            		 
                Algorithm.fitMoments(result, ctrl, image)
            		 
                result.getComponents()[0].getCoefficients()[:] += np.random.uniform(0, 1e-3, size=6)
            		 
                result.getComponents()[1].getCoefficients()[:] += np.random.uniform(0, 1e-3, size=3)
            		 
                Algorithm.fitProfile(result, ctrl, image)
            		 
                return result

            then the script (almost always) finishes without error. If I change the upper bound of the uniform random variates to 1e-6 though, it usually still fails.

            Also, how do I record and review all the log.debug<> messages in optimizer.cc? Seems like a useful trick to know.

            Show
            jmeyers314 Joshua Meyers added a comment - Nate Lust , could this just be a case where there's too much degeneracy for the solver? The coefficients that come out of fitMoments() are all zeros except for the first element for each of the two components. If I change testProfile() in your script to: def testProfile(exposure, ctrl): import numpy as np psf = exposure.getPsf() image = psf.computeKernelImage() result = Algorithm.initializeResult(ctrl) Algorithm.fitMoments(result, ctrl, image) result.getComponents()[ 0 ].getCoefficients()[:] + = np.random.uniform( 0 , 1e - 3 , size = 6 ) result.getComponents()[ 1 ].getCoefficients()[:] + = np.random.uniform( 0 , 1e - 3 , size = 3 ) Algorithm.fitProfile(result, ctrl, image) return result then the script (almost always) finishes without error. If I change the upper bound of the uniform random variates to 1e-6 though, it usually still fails. Also, how do I record and review all the log.debug<> messages in optimizer.cc? Seems like a useful trick to know.
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Actually, I think I found it. In DoubleShapeletPsfApproxAlgorithm::fitProfile, the line

            int state = optimizer.run();

            isn't right. optimizer.run() returns the number of outer iterations used, not the state. It's probably sufficient to replace this with:

            optimizer.run();
            		 
            int state = optimizer.getState();

            Show
            jmeyers314 Joshua Meyers added a comment - Actually, I think I found it. In DoubleShapeletPsfApproxAlgorithm::fitProfile, the line int state = optimizer.run(); isn't right. optimizer.run() returns the number of outer iterations used, not the state. It's probably sufficient to replace this with: optimizer.run(); int state = optimizer.getState();
            Hide
            Permalink
            jmeyers314 Joshua Meyers added a comment -

            Well, I never managed to reproduce the original error inside the ci_hsc package (other things changed in meantime there?), but I did reproduce and fix the error in the shapeFitFail.py script attached to this ticket. ci_hsc runs happily through Jenkins too.

            The problem was, as eluded to above, that the optimizer state was being set to the output of optimizer.run(), which returns number of iterations, not the state. The particular number of iterations used for this PSF happened to have a bit set corresponding to a max iterations flag, so it looked like the fit was failing, but actually the fit worked just fine.

            Show
            jmeyers314 Joshua Meyers added a comment - Well, I never managed to reproduce the original error inside the ci_hsc package (other things changed in meantime there?), but I did reproduce and fix the error in the shapeFitFail.py script attached to this ticket. ci_hsc runs happily through Jenkins too. The problem was, as eluded to above, that the optimizer state was being set to the output of optimizer.run(), which returns number of iterations, not the state. The particular number of iterations used for this PSF happened to have a bit set corresponding to a max iterations flag, so it looked like the fit was failing, but actually the fit worked just fine.
            Hide
            Permalink
            rearmstr Bob Armstrong added a comment -

            The code looks good to me. Good catch!

            Show
            rearmstr Bob Armstrong added a comment - The code looks good to me. Good catch!

              People

              Assignee:
              jmeyers314 Joshua Meyers
              Reporter:
              nlust Nate Lust
              Watchers:
              Bob Armstrong, Joshua Meyers, Nate Lust
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.