Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9553

Investigate the best algorithm to compute derivatives for the Brighter-Fatter correction

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ip_isr
    • Labels:
      None
    • Story Points:
      1
    • Epic Link:
    • Sprint:
      DRP S17-5
    • Team:
      Data Release Production

      Description

      As noted in DM-9535, the method used to compute second derivatives in the Brighter-Fatter correction implementation can lead to significant (up to 1%, which is particularly significant for weak lensing considerations) differences in the trace radii of sources, with the differences becoming larger with brighter magnitudes (see figures in DM-9535). This ticket is to investigate the optimal algorithm to use for the Brighter-Fatter correction.

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            I don't think this ticket is immediately critical for LSST (although it certainly needs resolving), but it likely will be for future HSC data releases — we'd better make sure Robert Lupton is aware of it.

            Show
            swinbank John Swinbank added a comment - I don't think this ticket is immediately critical for LSST (although it certainly needs resolving), but it likely will be for future HSC data releases — we'd better make sure Robert Lupton is aware of it.
            Hide
            swinbank John Swinbank added a comment -

            Just for clarity: we actually have several months of time set aside in the long term plan for figuring out how to handle brighter-fatter. My assumption (and Lauren MacArthur, please correct me if I'm wrong) is that this ticket is not supposed to capture all that effort, but simply to figure out what the best approach is in the short term (ie, on the timescale of the next HSC software release).

            Show
            swinbank John Swinbank added a comment - Just for clarity: we actually have several months of time set aside in the long term plan for figuring out how to handle brighter-fatter. My assumption (and Lauren MacArthur , please correct me if I'm wrong) is that this ticket is not supposed to capture all that effort, but simply to figure out what the best approach is in the short term (ie, on the timescale of the next HSC software release).
            Hide
            lauren Lauren MacArthur added a comment -

            Indeed. The algorithm selected on this ticket should result in behaviour that is at least as good as that seen with the HSC stack. That may end up being a straight adoption of what is there now, but some effort will me made (likely by Bob Armstrong) to homogenize the method for computing the derivative between the 1st & 2nd (they currently use different methods).

            Show
            lauren Lauren MacArthur added a comment - Indeed. The algorithm selected on this ticket should result in behaviour that is at least as good as that seen with the HSC stack. That may end up being a straight adoption of what is there now, but some effort will me made (likely by Bob Armstrong ) to homogenize the method for computing the derivative between the 1st & 2nd (they currently use different methods).
            Hide
            swinbank John Swinbank added a comment -

            According to the discussion at the HSC call of 207-03-14, Bob Armstrong will not make an effort to homogenize methods (as described by Lauren MacArthur above), but will simply revert to the old (HSC) way of doing things.

            That's fine in the short term, but we need to figure out whether there's a larger problem here that we need to address in the intermediate term (based on the assumption that the whole system needs revising in the long term). Bob Armstrong, can you comment on that please?

            Show
            swinbank John Swinbank added a comment - According to the discussion at the HSC call of 207-03-14, Bob Armstrong will not make an effort to homogenize methods (as described by Lauren MacArthur above), but will simply revert to the old (HSC) way of doing things. That's fine in the short term, but we need to figure out whether there's a larger problem here that we need to address in the intermediate term (based on the assumption that the whole system needs revising in the long term). Bob Armstrong , can you comment on that please?
            Hide
            rearmstr Bob Armstrong added a comment -

            Yes, this was seen as a short term fix only since we would like to use the older code for the next HSC processing. We certainly still need to investigate why the newer version is not working and decide how to fix it.

            Show
            rearmstr Bob Armstrong added a comment - Yes, this was seen as a short term fix only since we would like to use the older code for the next HSC processing. We certainly still need to investigate why the newer version is not working and decide how to fix it.
            Hide
            swinbank John Swinbank added a comment -

            Ok — could you please create the follow-up ticket when you're working on this? Thanks!

            Show
            swinbank John Swinbank added a comment - Ok — could you please create the follow-up ticket when you're working on this? Thanks!
            Hide
            lauren Lauren MacArthur added a comment -

            Give me a sec to rebase and I'll push the branch I was using with the update (revert!).

            Show
            lauren Lauren MacArthur added a comment - Give me a sec to rebase and I'll push the branch I was using with the update (revert!).
            Hide
            lauren Lauren MacArthur added a comment -

            Branch is now pushed.

            Show
            lauren Lauren MacArthur added a comment - Branch is now pushed.
            Hide
            swinbank John Swinbank added a comment -

            Bob ArmstrongLauren MacArthur suggests that we assign this ticket to her, since she already has a fix on that branch, and have you act as reviewer. If you object, do shout!

            Show
            swinbank John Swinbank added a comment - Bob Armstrong — Lauren MacArthur suggests that we assign this ticket to her, since she already has a fix on that branch, and have you act as reviewer. If you object, do shout!
            Hide
            lauren Lauren MacArthur added a comment -

            Bob Armstrong, do you have time to review this?

            Show
            lauren Lauren MacArthur added a comment - Bob Armstrong , do you have time to review this?
            Hide
            rearmstr Bob Armstrong added a comment -

            Yep, I'll take a look.

            Show
            rearmstr Bob Armstrong added a comment - Yep, I'll take a look.
            Hide
            rearmstr Bob Armstrong added a comment -

            Looks good.

            Show
            rearmstr Bob Armstrong added a comment - Looks good.
            Hide
            lauren Lauren MacArthur added a comment -

            Thanks Bob Armstrong. I ran every combination of Jenkins and it passed all it currently can, i.e. through ci_hsc on all py2 os' (centos6&7 and OS X), but only lsst_distrib on py3 (since ci_hsc hasn't been py3'd yet).

            Show
            lauren Lauren MacArthur added a comment - Thanks Bob Armstrong . I ran every combination of Jenkins and it passed all it currently can, i.e. through ci_hsc on all py2 os' (centos6&7 and OS X), but only lsst_distrib on py3 (since ci_hsc hasn't been py3'd yet).

              People

              Assignee:
              rearmstr Bob Armstrong
              Reporter:
              lauren Lauren MacArthur
              Reviewers:
              Bob Armstrong
              Watchers:
              Bob Armstrong, John Swinbank, Lauren MacArthur
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.