Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-17425

Investigate use of smaller patch sizes

    Details

    • Type: Story
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Team:
      External

      Description

      Deep observations and improved sky subtraction combine to link detections over a large fraction of the coadd patches, which makes things difficult for the deblender, resulting in large memory usage and very long execution times.

      Ikeda-san is investigating the use of smaller patch sizes, which artificially truncates the blends. I have a patch for the deblender that does the same kind of thing there. We need to validate these approaches. We are especially concerned that we not degrade the photometry.

        Attachments

        1. det_compare_4k_2k_revisedDetection.png
          det_compare_4k_2k_revisedDetection.png
          3.32 MB
        2. fig_deltam_large1_position.pdf
          2.76 MB
        3. patchsize_2k4k-1.pdf
          525 kB
        4. patchsize_4k_2k_det.png
          patchsize_4k_2k_det.png
          2.10 MB
        5. patchsize_twocolor_v2.pdf
          237 kB
        6. patchsize_twocolor.pdf
          263 kB
        7. patchsize_v4.pdf
          589 kB
        8. patciszie_v5.pdf
          485 kB

          Activity

          Hide
          masayuki.tanaka Masayuki Tanaka added a comment -

          Ikeda-kun looked at the stellar sequence on the gri color-color diagram in UD-COSMOS.  He processed a 4k x 4k patch, cross-matched with the HST/ACS data to select stars, and plotted the matched stars (page 1).  He did the same for 4 x (2k x 2k) patches, covering the same area, on page 2.  The scatter is clearly larger in the latter plot, indicating the smaller patch size degrades the photometry.  We have not yet understood why.

          Show
          masayuki.tanaka Masayuki Tanaka added a comment - Ikeda-kun looked at the stellar sequence on the gri color-color diagram in UD-COSMOS.  He processed a 4k x 4k patch, cross-matched with the HST/ACS data to select stars, and plotted the matched stars (page 1).  He did the same for 4 x (2k x 2k) patches, covering the same area, on page 2.  The scatter is clearly larger in the latter plot, indicating the smaller patch size degrades the photometry.  We have not yet understood why.
          Hide
          ikedahr Hiroyuki Ikeda added a comment - - edited

          The stellar sequence on the gri color-color diagram (which we uploaded previously) is made by the faint sources (i_mag<25). In addition to this, we only use the data in one patch. To investigate why the scatter for 2k x 2k patches is larger, we focus on the bright sources (i_mag<23) by using the data in several patches.

          We checked the stellar sequence on the gri color-color diagram by using the bright sources (i_mag<23). We confirmed that the scatter for 2k x 2k patches roughly consistent with that for 4k x 4k patches.

          patchsize_twocolor_v2.pdf

          Show
          ikedahr Hiroyuki Ikeda added a comment - - edited The stellar sequence on the gri color-color diagram (which we uploaded previously) is made by the faint sources (i_mag<25). In addition to this, we only use the data in one patch. To investigate why the scatter for 2k x 2k patches is larger, we focus on the bright sources (i_mag<23) by using the data in several patches. We checked the stellar sequence on the gri color-color diagram by using the bright sources (i_mag<23). We confirmed that the scatter for 2k x 2k patches roughly consistent with that for 4k x 4k patches. patchsize_twocolor_v2.pdf
          Hide
          ikedahr Hiroyuki Ikeda added a comment - - edited

          We also checked stellar sequence on the gri color-color diagram by using the psf magnitude. 

          We find the best fit line by using the 4k x 4k catalog (tract=9813, 81 patches) and we assume that this best fit line is the most reliable. Using this best fit line, we  calculate the scatter around this best fit line for 2k x 2k (all and center), 4k x 4k versions (all and center)  and confirmed that the scatter of 2k x 2k looks similar to that of 4k x 4k.

          patchsize_v4.pdf

          Show
          ikedahr Hiroyuki Ikeda added a comment - - edited We also checked stellar sequence on the gri color-color diagram by using the psf magnitude.  We find the best fit line by using the 4k x 4k catalog (tract=9813, 81 patches) and we assume that this best fit line is the most reliable. Using this best fit line, we  calculate the scatter around this best fit line for 2k x 2k (all and center), 4k x 4k versions (all and center)  and confirmed that the scatter of 2k x 2k looks similar to that of 4k x 4k. patchsize_v4.pdf
          Hide
          price Paul Price added a comment -

          At our meeting, we agreed we need to check the CModel magnitudes, and that the increased number of patches doesn't break the object IDs numbering.

          This may also be correlated with the backgrounds, so we need to validate this once the backgrounds have been updated.

          Show
          price Paul Price added a comment - At our meeting, we agreed we need to check the CModel magnitudes, and that the increased number of patches doesn't break the object IDs numbering. This may also be correlated with the backgrounds, so we need to validate this once the backgrounds have been updated.
          Hide
          ikedahr Hiroyuki Ikeda added a comment -

          We also checked stellar sequence on the gri color-color diagram by using the cmodel magnitude. We confirmed that the scatter of 2k x 2k looks similar to that of 4k x 4k.

          patciszie_v5.pdf

          Show
          ikedahr Hiroyuki Ikeda added a comment - We also checked stellar sequence on the gri color-color diagram by using the cmodel magnitude. We confirmed that the scatter of 2k x 2k looks similar to that of 4k x 4k. patciszie_v5.pdf
          Hide
          ikedahr Hiroyuki Ikeda added a comment -

          Previously, we found that some objects (~6.6%) have 

          delta_m (cmodel mag (4K) - model mag (2K))>1.0.

          patchsize_2k4k-1.pdf

          We also check the footprint for 4K and 2K (delta_mag>1mag) to study why delta_m > 1. As shown in 3page of the above pdf file, the footprint looks same.

          But, modelfit_CModel_ellipse_{xx,yy,xy} for 2K > 100.

          Show
          ikedahr Hiroyuki Ikeda added a comment - Previously, we found that some objects (~6.6%) have  delta_m (cmodel mag (4K) - model mag (2K))>1.0. patchsize_2k4k-1.pdf We also check the footprint for 4K and 2K (delta_mag>1mag) to study why delta_m > 1. As shown in 3page of the above pdf file, the footprint looks same. But, modelfit_CModel_ellipse_{xx,yy,xy} for 2K > 100.
          Hide
          ikedahr Hiroyuki Ikeda added a comment -

          We are still discussing. Because we just noticed that the detected sources around bright sources for 2k is somewhat different from the 4k. Therefore, we will check the difference of the detected sources around bright sources for 2k and 4k when we apply the revised detection algorithm that is made by mineo-san.

          Show
          ikedahr Hiroyuki Ikeda added a comment - We are still discussing. Because we just noticed that the detected sources around bright sources for 2k is somewhat different from the 4k. Therefore, we will check the difference of the detected sources around bright sources for 2k and 4k when we apply the revised detection algorithm that is made by mineo-san.
          Hide
          ikedahr Hiroyuki Ikeda added a comment - - edited

          We checked the difference of the detected sources around bright sources for 2k and 4k when we apply the revised detection algorithm that is made by mineo-san.

          WE confirmed that detected sources around bright sources for 2k is slightly different from the 4k even if we apply the revised detection algorithm that is made by mineo-san.

           

          In addition, we will check the difference of the detected sources after the sky subtraction algorithm is improved.

          Show
          ikedahr Hiroyuki Ikeda added a comment - - edited We checked the difference of the detected sources around bright sources for 2k and 4k when we apply the revised detection algorithm that is made by mineo-san. WE confirmed that detected sources around bright sources for 2k is slightly different from the 4k even if we apply the revised detection algorithm that is made by mineo-san.   In addition, we will check the difference of the detected sources after the sky subtraction algorithm is improved.

            People

            • Assignee:
              ikedahr Hiroyuki Ikeda
              Reporter:
              price Paul Price
              Watchers:
              Hiroyuki Ikeda, Masayuki Tanaka, Paul Price
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Summary Panel