Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-19903

Add columnar defect identification to findDefectsTask in cp_pipe please

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Add the extra code developed in DM-19336  to the `findDefectsTask` in cp_pipe (written as part of DM-19709).  

        Attachments

          Issue Links

            Activity

            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -
            • The obs_lsst part of this ticket--which added LATISS correction files--was merged in October 2019.
            • I have implemented the tests and other comments suggested by the reviewers in git hub, and the cp_part of the ticket has been merged.
            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - The obs_lsst part of this ticket-- which added LATISS correction files --was merged in October 2019. I have implemented the tests and other comments suggested by the reviewers in git hub, and the cp_part of the ticket has been merged.
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            After the implementation of DM-24556, some of the expected boxes in the unit tests (test/test_defects.py of cp_pipe) need to change because the code will now mark, for example, two independent boxes of defects that are adjacent as only one. This affects the tests 10-15, in particular where we have "blobs" to the sides of the columns.

            Thus, the new expected defects are as follows:

            test_maskBlocks_blobs_one_side_good_less_than_threshold:

             expectedDefects = [Box2I(corner=Point2I(60, 1), dimensions=Extent2I(1, 29)),
                                                Box2I(corner=Point2I(61, 2), dimensions=Extent2I(2, 12))]
            

            test_maskBlocks_blobs_other_side_good_less_than_threshold

             expectedDefects = [Box2I(corner=Point2I(70, 1), dimensions=Extent2I(1, 29)),
                                                Box2I(corner=Point2I(68, 2), dimensions=Extent2I(2, 12))]
            

            test_maskBlocks_blob_both_sides_good_less_than_threshold

             expectedDefects = [Box2I(corner=Point2I(75, 1), dimensions=Extent2I(1, 29)),
                                                Box2I(corner=Point2I(73, 2), dimensions=Extent2I(2, 12)),
                                                Box2I(corner=Point2I(76, 2), dimensions=Extent2I(2, 12))]
            

            test_maskBlocks_blob_one_side_good_greater_than_threshold

                    expectedDefects = [Box2I(corner=Point2I(80, 1), dimensions=Extent2I(1, 29)),
                                          Box2I(corner=Point2I(81, 2), dimensions=Extent2I(2, 2)),
                                          Box2I(corner=Point2I(81, 8), dimensions=Extent2I(2, 8))]
            

            test_maskBlocks_other_side_good_greater_than_threshold

                    expectedDefects = [Box2I(corner=Point2I(87, 1), dimensions=Extent2I(1, 29)),
                                       Box2I(corner=Point2I(85, 2), dimensions=Extent2I(2, 2)),
                                       Box2I(corner=Point2I(85, 8), dimensions=Extent2I(2, 8))]
            

            test_maskBlocks_both_sides_good_greater_than_threshold

                    expectedDefects = [Box2I(corner=Point2I(93, 1), dimensions=Extent2I(1, 34)),
                                       Box2I(corner=Point2I(91, 2), dimensions=Extent2I(2, 7)),
                                       Box2I(corner=Point2I(91, 18), dimensions=Extent2I(2, 9)),
                                       Box2I(corner=Point2I(94, 2), dimensions=Extent2I(2, 7)),
                                       Box2I(corner=Point2I(94, 18), dimensions=Extent2I(2, 9))]
            

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - After the implementation of DM-24556 , some of the expected boxes in the unit tests ( test/test_defects.py of cp_pipe ) need to change because the code will now mark, for example, two independent boxes of defects that are adjacent as only one. This affects the tests 10-15, in particular where we have "blobs" to the sides of the columns. Thus, the new expected defects are as follows: test_maskBlocks_blobs_one_side_good_less_than_threshold : expectedDefects = [Box2I(corner = Point2I( 60 , 1 ), dimensions = Extent2I( 1 , 29 )), Box2I(corner = Point2I( 61 , 2 ), dimensions = Extent2I( 2 , 12 ))] test_maskBlocks_blobs_other_side_good_less_than_threshold expectedDefects = [Box2I(corner = Point2I( 70 , 1 ), dimensions = Extent2I( 1 , 29 )), Box2I(corner = Point2I( 68 , 2 ), dimensions = Extent2I( 2 , 12 ))] test_maskBlocks_blob_both_sides_good_less_than_threshold expectedDefects = [Box2I(corner = Point2I( 75 , 1 ), dimensions = Extent2I( 1 , 29 )), Box2I(corner = Point2I( 73 , 2 ), dimensions = Extent2I( 2 , 12 )), Box2I(corner = Point2I( 76 , 2 ), dimensions = Extent2I( 2 , 12 ))] test_maskBlocks_blob_one_side_good_greater_than_threshold expectedDefects = [Box2I(corner = Point2I( 80 , 1 ), dimensions = Extent2I( 1 , 29 )), Box2I(corner = Point2I( 81 , 2 ), dimensions = Extent2I( 2 , 2 )), Box2I(corner = Point2I( 81 , 8 ), dimensions = Extent2I( 2 , 8 ))] test_maskBlocks_other_side_good_greater_than_threshold expectedDefects = [Box2I(corner = Point2I( 87 , 1 ), dimensions = Extent2I( 1 , 29 )), Box2I(corner = Point2I( 85 , 2 ), dimensions = Extent2I( 2 , 2 )), Box2I(corner = Point2I( 85 , 8 ), dimensions = Extent2I( 2 , 8 ))] test_maskBlocks_both_sides_good_greater_than_threshold expectedDefects = [Box2I(corner = Point2I( 93 , 1 ), dimensions = Extent2I( 1 , 34 )), Box2I(corner = Point2I( 91 , 2 ), dimensions = Extent2I( 2 , 7 )), Box2I(corner = Point2I( 91 , 18 ), dimensions = Extent2I( 2 , 9 )), Box2I(corner = Point2I( 94 , 2 ), dimensions = Extent2I( 2 , 7 )), Box2I(corner = Point2I( 94 , 18 ), dimensions = Extent2I( 2 , 9 ))]
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Might it not be better to just test for the pixels within the mask, as these really should be invariant under bbox representations? Also, why are we marking test updates on a long-closed ticket?

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Might it not be better to just test for the pixels within the mask, as these really should be invariant under bbox representations? Also, why are we marking test updates on a long-closed ticket?
            Hide
            plazas Andrés Alejandro Plazas Malagón added a comment -

            Yes, we could look at the pixels too.
            I thought that this was the most natural place to place these updates that had to be made after DM-24556 since they are directly related to this ticket. I could open a new ticket and paste the update there too, either way, it is fine with me, just let me know.

            Show
            plazas Andrés Alejandro Plazas Malagón added a comment - Yes, we could look at the pixels too. I thought that this was the most natural place to place these updates that had to be made after DM-24556 since they are directly related to this ticket. I could open a new ticket and paste the update there too, either way, it is fine with me, just let me know.
            Hide
            swinbank John Swinbank added a comment -

            I believe that all the relevant changes were made on DM-24556, which is fine. There's no need for a new ticket. Posting a note here about changes made on another ticket is unnecessary, but harmless.

            Show
            swinbank John Swinbank added a comment - I believe that all the relevant changes were made on DM-24556 , which is fine. There's no need for a new ticket. Posting a note here about changes made on another ticket is unnecessary, but harmless.

              People

              • Assignee:
                plazas Andrés Alejandro Plazas Malagón
                Reporter:
                plazas Andrés Alejandro Plazas Malagón
                Reviewers:
                John Parejko
                Watchers:
                Andrés Alejandro Plazas Malagón, John Swinbank, Merlin Fisher-Levine, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel