# Columnar defect identifcation

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s: None
• Labels:
None
• Story Points:
8
• Team:
Data Release Production

## Description

The system for identifying defects on DM-17490 could be more effective if it could simply flag whole (or partial) columns as bad when they contain obviously-suspicious patterns (e.g. alternating on/off pixels).

## Attachments

1. FindDefects_may22.py
19 kB
2. FindDefects.py
18 kB
3. gap_columnar.pdf
42 kB
4. image-2019-05-17-12-10-19-793.png
43 kB
5. image-2019-05-17-12-10-44-970.png
43 kB
6. image-2019-05-17-12-16-18-807.png
44 kB
7. image-2019-05-22-16-49-07-235.png
55 kB
8. image-2019-05-22-16-49-55-692.png
48 kB
9. no_gap_columnar.pdf
35 kB

## Activity

Hide
Andrés Alejandro Plazas Malagón added a comment - - edited

To address what is described in this ticket, I added a few lines of code at the end of the function find_hot_and_cold_pixels in the code described in DM-19334. Updated version of the code: FindDefects.py

TS8 data, detector S10, amp C5:

After:

^ ^

Show
Andrés Alejandro Plazas Malagón added a comment - - edited To address what is described in this ticket, I added a few lines of code at the end of the function find_hot_and_cold_pixels in the code described in  DM-19334 . Updated version of the code:  FindDefects.py TS8 data, detector S10, amp C5:   After:   ^ ^
Hide
Merlin Fisher-Levine added a comment -

Columns are often bad from a point up or down (depending on readout direction, i.e. which half of the chip they're on) and then good for the rest.

I think this code should be able to work out, for a column that's got a line of pixels on the cusp of being bad but then goes good again, how big the bbox needs to be, as opposed to just masking the whole thing.

I realise it kind of does this already, given that it takes min and max y on that column, but a single pixel near the edge would result in the whole thing being marked bad.

Does that sound like a feasible update?

Show
Merlin Fisher-Levine added a comment - Columns are often bad from a point up or down (depending on readout direction, i.e. which half of the chip they're on) and then good for the rest. I think this code should be able to work out, for a column that's got a line of pixels on the cusp of being bad but then goes good again, how big the bbox needs to be, as opposed to just masking the whole thing. I realise it kind of does this already, given that it takes min and max y on that column, but a single pixel near the edge would result in the whole thing being marked bad. Does that sound like a feasible update?
Hide
Andrés Alejandro Plazas Malagón added a comment - - edited

I added a few extra lines to the same of the code (function find_hot_and_cold_pixels) to address Merlin's comment above. Updated code: FindDefects_may22.py

Example:

1) Masking all the pixels between min_y and max_y on a particular column, even if there are relatively large gaps of "good" pixels in between:

2) If there's a gap of "good pixels" larger than the new parameter "gap_threshold" (30 in this example), then don't mask them:

Full PDFs:

Show
Andrés Alejandro Plazas Malagón added a comment - - edited I added a few extra lines to the same of the code (function find_hot_and_cold_pixels) to address Merlin's comment above. Updated code:  FindDefects_may22.py   Example:  1) Masking all the pixels between min_y and max_y on a particular column, even if there are relatively large gaps of "good" pixels in between:    2) If there's a gap of "good pixels" larger than the new parameter "gap_threshold" (30 in this example), then don't mask them:      Full PDFs:    no_gap_columnar.pdf   gap_columnar.pdf
Hide
Merlin Fisher-Levine added a comment - - edited

Haven't looked at the code, but that doesn't sound like a bad approach for now. Could you file a new ticket that says to add this as a feature to the findDefectsTask in cp_pipe please? (and post the relevant code on that ticket too)

Show
Merlin Fisher-Levine added a comment - - edited Haven't looked at the code, but that doesn't sound like a bad approach for now. Could you file a new ticket that says to add this as a feature to the findDefectsTask in cp_pipe please? (and post the relevant code on that ticket too)
Hide
Merlin Fisher-Levine added a comment -

Looks good. Just post the code on DM-19903 and then mark as Done

Show
Merlin Fisher-Levine added a comment - Looks good. Just post the code on DM-19903 and then mark as Done
Hide
Andrés Alejandro Plazas Malagón added a comment -

Code posted on DM-19903

Show
Andrés Alejandro Plazas Malagón added a comment - Code posted on  DM-19903

## People

• Assignee:
Andrés Alejandro Plazas Malagón
Reporter:
John Swinbank
Reviewers:
Merlin Fisher-Levine
Watchers:
Andrés Alejandro Plazas Malagón, John Swinbank, Merlin Fisher-Levine