Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-26950

Reprocess and Compare RC2 with Increased Sky Correction bgModel2 Values

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Story Points:
      12
    • Epic Link:
    • Team:
      Data Release Production
    • Urgent?:
      No

      Description

      Current image processing runs the skyCorrection.py routines to produce an improved background map. This process, in brief, 1) adds back on the original calexpBackground to its calexp (essentially undoing any visit-level background subtraction); 2) bins all CCDs for a single visit into 8192x8192 pixel superpixels, fits an Akima spline to these clipped-mean values and subtracts this bgModel; 3) fits and subtracts a sky frame across the entire focal plane; and 4) performs an additional background model subtraction (bgModel2) as in 2), but using smaller superpixel sizes of 256x256 pixels.

      In Aihara et al. 2019, the bin scale of this bgModel step is given as 1024x1024 pixels. Since then, this step has split into two: an initial large-scale subtraction of 8192x8192 pixels (step 2 above) and a second medium-scale subtraction of 256x256 pixels (step 4 above). This ticket explores the effects of increasing this medium-scale bin size to 1024x1024 pixels as indicated in the Aihara et al. 2019 paper.

      All three RC2 tracts have been reprocessed with the following config modifications:

      config.bgModel2.xSize=15.36
      config.bgModel2.ySize=15.36
      

      Beginning with w_2020_34/DM-26441 data, these processing steps have been run:

      1. skyCorrection.py
      2. coaddDriver.py
      3. multiBandDriver.py
      4. coaddAnalysis.py
      5. compareCoaddAnalysis.py

      As a prior reference dataset, I'm again refering to the data processing in w_2020_34/DM-26441. For this round of testing, all 26 visits which contribute to the GAMA HSC-I band tract (9615) were reprocessed. Further analysis of relevant figures is found in the comments below.

      In summary, it appears that the current configuration produces more optimal outputs than the proposed change (i.e., bgModel2 set at the current 256 pixel value is superior to the proposed 1024 pixel value). This is due to the coadd outputs looking to be more stable, and producing fewer stacking edge artefacts. No further action is recommended in modifying bgModel2 at this time.

      All outputs are available on lsst-devl at: /project/lskelvin/bgModel2, on the RSP at: /home/lskelvin/bgModel2, or at this web link: https://lsst.ncsa.illinois.edu/~lskelvin/project/bgModel2.

      PS: for context, two community posts on sky subtraction and HSC default sky correction may be useful.

        Attachments

        1. skyobj1024.png
          skyobj1024.png
          122 kB
        2. skyobj256.png
          skyobj256.png
          122 kB
        3. t9615p33-1024.png
          t9615p33-1024.png
          543 kB
        4. t9615p33-256.png
          t9615p33-256.png
          533 kB
        5. t9615p33-Diff.png
          t9615p33-Diff.png
          171 kB
        6. t9615p55-1024.png
          t9615p55-1024.png
          516 kB
        7. t9615p55-256.png
          t9615p55-256.png
          530 kB
        8. t9615p55-Diff.png
          t9615p55-Diff.png
          204 kB
        9. t9615p77-1024.png
          t9615p77-1024.png
          433 kB
        10. t9615p77-256.png
          t9615p77-256.png
          465 kB
        11. t9615p77-Diff.png
          t9615p77-Diff.png
          177 kB
        12. v1258-skyCorr1024.png
          v1258-skyCorr1024.png
          304 kB
        13. v1258-skyCorr256.png
          v1258-skyCorr256.png
          329 kB
        14. v1258-skyCorrDiff.png
          v1258-skyCorrDiff.png
          170 kB

          Issue Links

            Activity

            Hide
            lskelvin Lee Kelvin added a comment - - edited

            The first set of figures shown here show the effect of increasing bgModel2 from the default 256 pixels to 1024 pixels on the output skyCorr image. The first figure shows the default 256 pixel output; the second shows the new 1024 pixel output; and the final figure shows the difference image between these two. These data are generated following a re-run of skyCorrection.py.

            Large differences are evident between these two frames, with significantly more large scale fluctuations found in the 1024 visit image than in the 256 visit image. Further, the visit edge artifacts appear to be amplified in the 1024 pixel case, emphasising their relative flux and ultimately contributing to further science image contamination down the line.



            Show
            lskelvin Lee Kelvin added a comment - - edited The first set of figures shown here show the effect of increasing bgModel2 from the default 256 pixels to 1024 pixels on the output skyCorr image. The first figure shows the default 256 pixel output; the second shows the new 1024 pixel output; and the final figure shows the difference image between these two. These data are generated following a re-run of skyCorrection.py . Large differences are evident between these two frames, with significantly more large scale fluctuations found in the 1024 visit image than in the 256 visit image. Further, the visit edge artifacts appear to be amplified in the 1024 pixel case, emphasising their relative flux and ultimately contributing to further science image contamination down the line.
            Hide
            lskelvin Lee Kelvin added a comment -

            Output differences at coadd level for tract 9613, patch 3,3. Images below show the entire patch produced using a bgModel2 value of 256 pixels (top), 1024 pixels (middle) and the difference image (bottom). Note that the two non-difference images are scale matched, as shown via the color bar along the right hand side of each panel.

            The impact of the differences noted above in the skyCorr image become apparent here, with the overall background sky level appearing much more flat and well characterized in the 256 pixel scenario than that of the 1024 pixel scenario. Notably, distinct low troughs have emerged in the 1024 pixel processing, an output which likely is not desirable here. Furthermore, strong horizontal and vertical striping - an artefict of the coaddition process which should be minimized if the visit-level background subtraction is sufficient - has been emphasized here, leaving unwanted signatures on the resultant coadds. This 'plaid' patchworking is also apparent on the difference imaging shown here.



            Show
            lskelvin Lee Kelvin added a comment - Output differences at coadd level for tract 9613, patch 3,3. Images below show the entire patch produced using a bgModel2 value of 256 pixels (top), 1024 pixels (middle) and the difference image (bottom). Note that the two non-difference images are scale matched, as shown via the color bar along the right hand side of each panel. The impact of the differences noted above in the skyCorr image become apparent here, with the overall background sky level appearing much more flat and well characterized in the 256 pixel scenario than that of the 1024 pixel scenario. Notably, distinct low troughs have emerged in the 1024 pixel processing, an output which likely is not desirable here. Furthermore, strong horizontal and vertical striping - an artefict of the coaddition process which should be minimized if the visit-level background subtraction is sufficient - has been emphasized here, leaving unwanted signatures on the resultant coadds. This 'plaid' patchworking is also apparent on the difference imaging shown here.
            Hide
            lskelvin Lee Kelvin added a comment -

            Output deepCoadd imaging as above, but for tract 9615 patch 5,5. Similar flocculent disturbances in the background sky as noted above are also found here. In addition, the mirror ghost at ~[1000,1000] (see the difference image, where this feature is especially clear) is particularly evident in the 1024 pixel bgModel2 processing, yet has been successfully removed from the 256 pixel processing. This is likely testament to the fact that a smaller 256 pixel mesh size is more readily able to process and remove such artifacts than the equivalent 1024 pixel processing.



            Show
            lskelvin Lee Kelvin added a comment - Output deepCoadd imaging as above, but for tract 9615 patch 5,5. Similar flocculent disturbances in the background sky as noted above are also found here. In addition, the mirror ghost at ~ [1000,1000] (see the difference image, where this feature is especially clear) is particularly evident in the 1024 pixel bgModel2 processing, yet has been successfully removed from the 256 pixel processing. This is likely testament to the fact that a smaller 256 pixel mesh size is more readily able to process and remove such artifacts than the equivalent 1024 pixel processing.
            Hide
            lskelvin Lee Kelvin added a comment -

            Output deepCoadd imaging as above, but for tract 9615 patch 7,7. This patch was chosen to stress-test processing, owing to the large and challenging bright star in the lower corner of the field of view. As expected, large differences appear between the 256 pixel and 1024 pixel version outputs, with the background sky appearing flatter and more well defined in the former than in the latter of these two. The 1024 pixel bgModel2 processing appears to have biased down the entirety of the left-hand side of this patch, severely compromising photometry for and faint and/or compact sources in this region.



            Show
            lskelvin Lee Kelvin added a comment - Output deepCoadd imaging as above, but for tract 9615 patch 7,7. This patch was chosen to stress-test processing, owing to the large and challenging bright star in the lower corner of the field of view. As expected, large differences appear between the 256 pixel and 1024 pixel version outputs, with the background sky appearing flatter and more well defined in the former than in the latter of these two. The 1024 pixel bgModel2 processing appears to have biased down the entirety of the left-hand side of this patch, severely compromising photometry for and faint and/or compact sources in this region.
            Hide
            lskelvin Lee Kelvin added a comment - - edited

            Example sky object analysis plots produced from runs of pipe_analysis over the two repos generated here. These plots show the flux contained within apertures of varying size and definition (see the inset legend) centred on sky objects. The left panel shows the overall flux within those apertures, and the right panel shows this flux scaled by the error in that measurement. Of particular interest are the CircApRad9pix apertures in orange, as a 9 pixel aperture is closest to the fiducial sky object 'aperture' of 8 pixels.

            Looking at the right-hand fluxErr-scaled panels, the mean has decreased from flux/fluxErr = 0.15 down to 0.14 as we move from 256 pixel bgModel2 processing to 1024 pixel processing. This is a small change, and (as discussed next) well within 1-sigma and so not particularly noteworthy. However, the standard deviation has increased from ~1.75 to 1.87, an increase of ~8%. This increase in the scatter of sky object flux directly maps onto the increase in background sky fluctuation visually noted above. For many science use cases, and particularly for low surface brightness science, it is particularly important for the residual background sky to be a) as close to zero as possible, and b) as flat and unvarying as possible. The fact that switching from 256 pixel bgModel2 processing to 1024 pixel processing increases the relative fluctuation in the background map by ~8% is troubling, and likely a cause for concern if 1024 pixel processing were to be adopted as standard.


            The full suite of pipe_analysis outputs, including plots generated with both coaddAnalysis.py and compareCoaddAnalysis.py, can be found at these links: 256-pixel processing; 1024-pixel processing.

            Show
            lskelvin Lee Kelvin added a comment - - edited Example sky object analysis plots produced from runs of pipe_analysis over the two repos generated here. These plots show the flux contained within apertures of varying size and definition (see the inset legend) centred on sky objects. The left panel shows the overall flux within those apertures, and the right panel shows this flux scaled by the error in that measurement. Of particular interest are the CircApRad9pix apertures in orange, as a 9 pixel aperture is closest to the fiducial sky object 'aperture' of 8 pixels. Looking at the right-hand fluxErr -scaled panels, the mean has decreased from flux / fluxErr = 0.15 down to 0.14 as we move from 256 pixel bgModel2 processing to 1024 pixel processing. This is a small change, and (as discussed next) well within 1-sigma and so not particularly noteworthy. However, the standard deviation has increased from ~1.75 to 1.87, an increase of ~8%. This increase in the scatter of sky object flux directly maps onto the increase in background sky fluctuation visually noted above. For many science use cases, and particularly for low surface brightness science, it is particularly important for the residual background sky to be a) as close to zero as possible, and b) as flat and unvarying as possible. The fact that switching from 256 pixel bgModel2 processing to 1024 pixel processing increases the relative fluctuation in the background map by ~8% is troubling, and likely a cause for concern if 1024 pixel processing were to be adopted as standard. The full suite of pipe_analysis outputs, including plots generated with both coaddAnalysis.py and compareCoaddAnalysis.py , can be found at these links: 256-pixel processing ; 1024-pixel processing .
            Hide
            lskelvin Lee Kelvin added a comment -

            In summary, this investigation has reprocessed tract 9615 in two different modes to test the impact of modifying the bgModel2 parameter in the skyCorr sky correction routines. The bgModel2 value controls the mesh size of the second round of background sky subtraction which occurs after the initial sky frame has been fit and subtracted from a visit. The current default value is 256 pixels (i.e., a box of 256x256 pixels in dimension). This investigation looked into whether or not this value should be increased to 1024 pixels, bringing it into line with what was previously suggested in Aihara et al. 2019.

            The bgModel2 = 1024 pixel outputs are significantly distinct from current default 256 pixel processing, with the resultant coadded imaging a) suffering from more severe fluctuation in the background sky map, b) exhibiting more extreme horizontal and vertical artefacts as a result of less desirable visit-level subtraction, and c) containing new mirror reflection artifacts which were not previously or currently present. Using sky objects as a measure of the quality of the background sky solution, we find that the background sky is ~8% more variable when using a bgModel2 value of 1024 pixels than remaining with the current default value of 256 pixels. For this reason, and to avoid the increased number of artifacts as detailed above, we recommend that the current default value for bgModel2 of 256 pixels be maintained for the time being.

            Show
            lskelvin Lee Kelvin added a comment - In summary, this investigation has reprocessed tract 9615 in two different modes to test the impact of modifying the bgModel2 parameter in the skyCorr sky correction routines. The bgModel2 value controls the mesh size of the second round of background sky subtraction which occurs after the initial sky frame has been fit and subtracted from a visit. The current default value is 256 pixels (i.e., a box of 256x256 pixels in dimension). This investigation looked into whether or not this value should be increased to 1024 pixels, bringing it into line with what was previously suggested in Aihara et al. 2019. The bgModel2 = 1024 pixel outputs are significantly distinct from current default 256 pixel processing, with the resultant coadded imaging a) suffering from more severe fluctuation in the background sky map, b) exhibiting more extreme horizontal and vertical artefacts as a result of less desirable visit-level subtraction, and c) containing new mirror reflection artifacts which were not previously or currently present. Using sky objects as a measure of the quality of the background sky solution, we find that the background sky is ~8% more variable when using a bgModel2 value of 1024 pixels than remaining with the current default value of 256 pixels. For this reason, and to avoid the increased number of artifacts as detailed above, we recommend that the current default value for bgModel2 of 256 pixels be maintained for the time being.

              People

              Assignee:
              lskelvin Lee Kelvin
              Reporter:
              lskelvin Lee Kelvin
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Lee Kelvin, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.