Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-32402

Add cells to patches in skymaps

    XMLWordPrintable

    Details

      Description

      As part of metadetect (and other uses!) we will be doing coadds in small edge-free coadd "cells". This ticket adds cells to patches in skymaps as a configuration option.

      When this configuration is chosen, the patches are built up in units of cells, and the tracts are built as units of patches, so all patches and tracts contain an integer number of cells (default is 20 cells per patch in the inner region). All patches have an extra ring of cells around the inner region which sets the overlap boundary. This is very helpful to ensure that neighboring patches will have a subset of pixels that are exactly identical (unlike the current "legacy" tract building method).

        Attachments

        1. patch_for_DC2_cells_new_zoom.png
          patch_for_DC2_cells_new_zoom.png
          33 kB
        2. patch_for_DC2_cells_new.png
          patch_for_DC2_cells_new.png
          57 kB
        3. patch_for_DC2_cells_zoom.png
          patch_for_DC2_cells_zoom.png
          33 kB
        4. patch_for_DC2_cells.png
          patch_for_DC2_cells.png
          57 kB
        5. patch_for_HSC_cells_new_zoom.png
          patch_for_HSC_cells_new_zoom.png
          33 kB
        6. patch_for_HSC_cells_new.png
          patch_for_HSC_cells_new.png
          56 kB
        7. patch_for_HSC_cells_zoom.png
          patch_for_HSC_cells_zoom.png
          33 kB
        8. patch_for_HSC_cells.png
          patch_for_HSC_cells.png
          57 kB
        9. patch_for_HSC_cells-1.png
          patch_for_HSC_cells-1.png
          57 kB
        10. plot_cells_example.py
          6 kB
        11. tract_for_DC2_cells_new_zoom.png
          tract_for_DC2_cells_new_zoom.png
          30 kB
        12. tract_for_DC2_cells_new.png
          tract_for_DC2_cells_new.png
          51 kB
        13. tract_for_DC2_cells_zoom.png
          tract_for_DC2_cells_zoom.png
          29 kB
        14. tract_for_DC2_cells.png
          tract_for_DC2_cells.png
          50 kB
        15. tract_for_DC2_legacy_zoom.png
          tract_for_DC2_legacy_zoom.png
          31 kB
        16. tract_for_DC2_legacy.png
          tract_for_DC2_legacy.png
          34 kB
        17. tract_for_HSC_cells_new_zoom.png
          tract_for_HSC_cells_new_zoom.png
          32 kB
        18. tract_for_HSC_cells_new.png
          tract_for_HSC_cells_new.png
          41 kB
        19. tract_for_HSC_cells_zoom.png
          tract_for_HSC_cells_zoom.png
          31 kB
        20. tract_for_HSC_cells.png
          tract_for_HSC_cells.png
          45 kB
        21. tract_for_HSC_legacy_zoom.png
          tract_for_HSC_legacy_zoom.png
          28 kB
        22. tract_for_HSC_legacy.png
          tract_for_HSC_legacy.png
          43 kB

          Issue Links

            Activity

            Hide
            erykoff Eli Rykoff added a comment - - edited

            Here is some code which will plot a tract + patches and patch + cells: plot_cells_example.py

            First, for DC2. The old version is called "legacy", the new version is "cells". The blue dots show the "inner" region of the tract. Looking at the legacy and cell version, the skymap tiling + pixel size just happens to give a lot more "extra" overlap with the cells tiling:

            And zooming in on the corner you can see that the legacy patches don't have an outer boundary, but the cell patches do:

            Looking at the cells in a patch of course we don't have a legacy version. There is one cell of buffer all the way around:

            Now for HSC, with the HSC pixel scale the cell version gives less wasted overlap. Huh!

            Zooming in:

            And the cells in a patch look very similar to DC2, as they should!

            Show
            erykoff Eli Rykoff added a comment - - edited Here is some code which will plot a tract + patches and patch + cells: plot_cells_example.py First, for DC2. The old version is called "legacy", the new version is "cells". The blue dots show the "inner" region of the tract. Looking at the legacy and cell version, the skymap tiling + pixel size just happens to give a lot more "extra" overlap with the cells tiling: And zooming in on the corner you can see that the legacy patches don't have an outer boundary, but the cell patches do: Looking at the cells in a patch of course we don't have a legacy version. There is one cell of buffer all the way around: Now for HSC, with the HSC pixel scale the cell version gives less wasted overlap. Huh! Zooming in: And the cells in a patch look very similar to DC2, as they should!
            Hide
            erykoff Eli Rykoff added a comment -

            Following up on some discussion, I now see that the DC2 skymap has the same settings as the HSC skymap except for the pixel scale. So I believe the configs were just copied over, with the new pixel scale, and the patches just came out of that. This is definitely something we can revisit for the real LSST survey.

            Show
            erykoff Eli Rykoff added a comment - Following up on some discussion, I now see that the DC2 skymap has the same settings as the HSC skymap except for the pixel scale. So I believe the configs were just copied over, with the new pixel scale, and the patches just came out of that. This is definitely something we can revisit for the real LSST survey.
            Hide
            erykoff Eli Rykoff added a comment - - edited

            After discussion with Arun Kannawadi we realized that the previous idea that the cells border extends beyond the patch outer boundary doesn't work: if we want to be able to use patch warps as inputs then these need to cover all the area that is covered by the patch outer bounding box. These are what the bounds look like now:

            Also, after discussions with Arun Kannawadi and Jim Bosch I have added a bunch of features to the tracts, patches, and cells. In particular, I added properties (so that you can do .innerBBox instead of .getInnerBBox()); I added a reference to the tract wcs to the patches and cells (so you can get polygons and do conversions without the tractInfo!); and I added the sequential patch and cell index to PatchInfo and CellInfo (so you can just access these with .sequentialIndex rather than computing them using the parent).

            Show
            erykoff Eli Rykoff added a comment - - edited After discussion with Arun Kannawadi we realized that the previous idea that the cells border extends beyond the patch outer boundary doesn't work: if we want to be able to use patch warps as inputs then these need to cover all the area that is covered by the patch outer bounding box. These are what the bounds look like now: Also, after discussions with Arun Kannawadi  and Jim Bosch  I have added a bunch of features to the tracts, patches, and cells. In particular, I added properties (so that you can do .innerBBox instead of .getInnerBBox() ); I added a reference to the tract wcs to the patches and cells (so you can get polygons and do conversions without the tractInfo !); and I added the sequential patch and cell index to PatchInfo and CellInfo (so you can just access these with .sequentialIndex rather than computing them using the parent).
            Hide
            erykoff Eli Rykoff added a comment -

            I think this is ready for review, and I'd appreciate it if both of you can take a look. I'm not sure how to split it up, though, just that I'd like two sets of interested eyes on the changes.

            Show
            erykoff Eli Rykoff added a comment - I think this is ready for review, and I'd appreciate it if both of you can take a look. I'm not sure how to split it up, though, just that I'd like two sets of interested eyes on the changes.
            Hide
            jbosch Jim Bosch added a comment -

            I've left some minor comments on the PR and marked it as Approved; you've already addressed my more substantive ones.  Removing myself from the Jira reviewers list.

            Show
            jbosch Jim Bosch added a comment - I've left some minor comments on the PR and marked it as Approved; you've already addressed my more substantive ones.  Removing myself from the Jira reviewers list.
            Hide
            kannawad Arun Kannawadi added a comment -

            I've left a number of minor comments on the PR, but overall looks great! Marking the review complete for now, but I plan have one more quick look after the changes are incorporated.

            Show
            kannawad Arun Kannawadi added a comment - I've left a number of minor comments on the PR, but overall looks great! Marking the review complete for now, but I plan have one more quick look after the changes are incorporated.
            Hide
            erykoff Eli Rykoff added a comment -
            Show
            erykoff Eli Rykoff added a comment - All comments have been addressed, and all the ci's passed: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/35351/pipeline

              People

              Assignee:
              erykoff Eli Rykoff
              Reporter:
              erykoff Eli Rykoff
              Reviewers:
              Arun Kannawadi
              Watchers:
              Arun Kannawadi, Eli Rykoff, Jim Bosch
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.