Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-2308

Improve hacks in PhotoCalTask for dealing with reference flux uncertainties

    XMLWordPrintable

Details

    • Improvement
    • Status: To Do
    • Resolution: Unresolved
    • None
    • meas_astrom
    • None
    • Alert Production

    Description

      In reviewing DM-1576 Jim Bosch commented on two existing ugly hacks in PhotoCalTask for dealing with uncertainties in reference object flux:

      • The reference flux is not handled in a principled way
      • The square root of the flux is used as the estimated error if the reference flux error is not known

      For now I have modified the code to ignore reference flux error. I will note, however, that any catalog that is suitable to be used for accurate photometric calibration ought to have known flux errors. So I am not convinced that the ugly hack of using sqrt(flux) is a very serious issue. Still, it might be smarter to ignore the reference flux error if it is unknown? For now I will ignore it in any case.

      Attachments

        Issue Links

          Activity

            What units is the reference flux measured in? Only if it's in counts does its sqrt correspond to an error, and I think it'd be safer to set it to None than to sqrt(flux).

            If the errors in the reference catalogue are much less than those in the measurements, the reference catalogue errors can safely be neglected (yes, there are some sqrt(N)s to be considered).

            Why are we transferring the errors to the measured fluxes? I see a comment,

            # Fitting with error bars in both axes is hard, so transfer all
            # the error to src, then convert to magnitude

            (which may have originated with me, although git clears me of the most recent edit to those lines), but it's not that hard. We should bite that bullet someday.

            rhl Robert Lupton added a comment - What units is the reference flux measured in? Only if it's in counts does its sqrt correspond to an error, and I think it'd be safer to set it to None than to sqrt(flux). If the errors in the reference catalogue are much less than those in the measurements, the reference catalogue errors can safely be neglected (yes, there are some sqrt(N)s to be considered). Why are we transferring the errors to the measured fluxes? I see a comment, # Fitting with error bars in both axes is hard, so transfer all # the error to src, then convert to magnitude (which may have originated with me, although git clears me of the most recent edit to those lines), but it's not that hard. We should bite that bullet someday.
            jbosch Jim Bosch added a comment -

            rhl, to clarify, those are exactly the problems rowen found in the current code before he started modifying it. I recommended that he switch to just ignoring the reference catalog errors for now, and asked him to open this issue to "bite that bullet" in the future.

            jbosch Jim Bosch added a comment - rhl , to clarify, those are exactly the problems rowen found in the current code before he started modifying it. I recommended that he switch to just ignoring the reference catalog errors for now, and asked him to open this issue to "bite that bullet" in the future.

            jbosch Yes, I know that. So I added those comments to this ticket so we do things right when we address it.

            rhl Robert Lupton added a comment - jbosch Yes, I know that. So I added those comments to this ticket so we do things right when we address it.

            jbosch rhl [rowen : I don't understand the issue. In the current Photocal version there is no actual fit, this is just a weighted average after sigma clipping. We can just add the errors from the sources and the reference in quadrature. Or am I missing something ?

            boutigny Dominique Boutigny added a comment - jbosch rhl [ rowen : I don't understand the issue. In the current Photocal version there is no actual fit, this is just a weighted average after sigma clipping. We can just add the errors from the sources and the reference in quadrature. Or am I missing something ?

            A weighted average is a maximum likelihood fit, assuming Gaussian errors in y-only.

            We are actually fitting a straight line to points with errors in both x- and y. If you know the errors this is also a well-defined problem that there are standard solutions for. Converting to magnitudes (to make the problem linear) is a bit of a concern too.

            So this ticket is asking that we write down the problem that we really want to solve. Then figure out a solution that is close enough that we needn't worry about the errors produced by the approximations.

            rhl Robert Lupton added a comment - A weighted average is a maximum likelihood fit, assuming Gaussian errors in y-only. We are actually fitting a straight line to points with errors in both x- and y. If you know the errors this is also a well-defined problem that there are standard solutions for. Converting to magnitudes (to make the problem linear) is a bit of a concern too. So this ticket is asking that we write down the problem that we really want to solve. Then figure out a solution that is close enough that we needn't worry about the errors produced by the approximations.
            jbosch Jim Bosch added a comment -

            Reviewing for CCB old-tickets sweep: I think this is still a problem, but I don't see us prioritizing this ticket anytime soon. In DRP the results of PhotoCalTask are wholly superseded by FGCM, and in both AP and DRP we're shortly going to be overhauling the early steps of of the pipeline in such a way that I suspect we'll make bigger changes that make this ticket moot (cc erykoff). But I'll keep it around in case PhotoCalTask does survive that refactor mostly intact and we need a reminder of this defect.

            jbosch Jim Bosch added a comment - Reviewing for CCB old-tickets sweep: I think this is still a problem, but I don't see us prioritizing this ticket anytime soon. In DRP the results of PhotoCalTask are wholly superseded by FGCM, and in both AP and DRP we're shortly going to be overhauling the early steps of of the pipeline in such a way that I suspect we'll make bigger changes that make this ticket moot (cc erykoff ). But I'll keep it around in case PhotoCalTask does survive that refactor mostly intact and we need a reminder of this defect.

            People

              Unassigned Unassigned
              rowen Russell Owen
              Dominique Boutigny, Jim Bosch, John Parejko, Robert Lupton, Russell Owen
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Jenkins

                  No builds found.