# Port safe coadd clipping from HSC

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
18
• Sprint:
Science Pipelines DM-W16-1, Science Pipelines DM-W16-2
• Team:
Data Release Production

## Description

We have an algorithm on the HSC fork that modifies AssembleCoaddTask to clip outliers in a much safer way, based on detecting contiguous regions in the difference between a non-clipped coadd and an aggressively-clipped coadd, and only rejecting pixels that are outliers in a single epoch.

One complication for this code transfer is that some of the coadd code has been refactored on the HSC side, and there may be code in hscPipe that duplicates much of what's in pipe_tasks. We may need help from Paul Price to resolve those inconsistencies before tackling this issue.

Once that's done, HSC code can be transferred from the following issues:

## Activity

Hide
John Swinbank added a comment -

Nate Lust The revised code on meas_base is better, thanks. There are still a couple of issues which I think need attention: you should eliminate the duplicate typedefs and take another crack at the comment in the fail() function.

Beyond that, there are a lot of minor spacing and formatting issues. I feel almost bad about pulling you up on these trivial issues, but they do need fixing. I wonder if there are linters that could help you track down these issues in C++ code automatically – might be worth investigating. (There are certainly code formatters, like clang-format, but you want something that'll point out mistakes rather than attempt to reformat the whole file.)

Show
John Swinbank added a comment - Nate Lust The revised code on meas_base is better, thanks. There are still a couple of issues which I think need attention: you should eliminate the duplicate typedefs and take another crack at the comment in the fail() function. Beyond that, there are a lot of minor spacing and formatting issues. I feel almost bad about pulling you up on these trivial issues, but they do need fixing. I wonder if there are linters that could help you track down these issues in C++ code automatically – might be worth investigating. (There are certainly code formatters, like clang-format , but you want something that'll point out mistakes rather than attempt to reformat the whole file.)
Hide
John Swinbank added a comment -

By the way, I'm assuming DM-4201 was a response to my request for a tech debt tickets covering this work? It's a start, but could you make it much more explicit about what actually needs doing, please.

Show
John Swinbank added a comment - By the way, I'm assuming DM-4201 was a response to my request for a tech debt tickets covering this work? It's a start, but could you make it much more explicit about what actually needs doing, please.
Hide
John Swinbank added a comment -

Nate Lust Three tiny whitespace tweaks on your latest meas_base iteration – almost perfect. I suggest you just take care of them when squashing history. I am happy for you to merge, but equally willing to take a look at your cleaned up history first if you like.

Show
John Swinbank added a comment - Nate Lust Three tiny whitespace tweaks on your latest meas_base iteration – almost perfect. I suggest you just take care of them when squashing history. I am happy for you to merge, but equally willing to take a look at your cleaned up history first if you like.
Hide
Nate Lust added a comment -

This work has been completed and merged to master in: meas_base, pipe_tasks, and obs_subaru.

Show
Nate Lust added a comment - This work has been completed and merged to master in: meas_base, pipe_tasks, and obs_subaru.
Hide
John Swinbank added a comment -

Thanks Nate Lust. As a final step, could you please write a brief (no more than a couple of sentences) summary of the new functionality you have enabled in the LSST stack and add it to https://confluence.lsstcorp.org/display/DM/Data+Release+Production+WIP+W16+release+notes ? Thanks!

Show
John Swinbank added a comment - Thanks Nate Lust . As a final step, could you please write a brief (no more than a couple of sentences) summary of the new functionality you have enabled in the LSST stack and add it to https://confluence.lsstcorp.org/display/DM/Data+Release+Production+WIP+W16+release+notes ? Thanks!

## People

• Assignee:
Nate Lust
Reporter:
Jim Bosch
Reviewers:
Bob Armstrong, Jim Bosch, John Swinbank
Watchers:
Bob Armstrong, Jim Bosch, John Swinbank, Nate Lust