Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4834

Preliminaries for LSST vs HSC pipeline comparison through coadd processing

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
      None

      Description

      This is the equivalent of DM-3942 but through coadd processing.

      Relevant HSC tickets include:

        Attachments

          Issue Links

            Activity

            Hide
            swinbank John Swinbank added a comment -

            We think this is already ported; we should check.

            Show
            swinbank John Swinbank added a comment - We think this is already ported; we should check.
            Hide
            price Paul Price added a comment -

            This is already ported:

            price@price-laptop:~/LSST/obs/subaru (master=) $ git log -1 fa40ad2
            commit fa40ad292b26d450bd9b30de01b5a48b39ff753e
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Thu Mar 10 14:42:44 2016 -0500
             
                config: reduce coadd subregionSize
                
                The HSC SSP Ultra-Deep currently has a total of 85 exposures in y-band,
                so that reading 2k square blocks exceeds 4 GB/core. Reducing the block
                size reduces the memory use at the expense of a bit of read inefficiency.
                This choice (200 rows, for a 4k patch width) should give us plenty of
                room for growth.
                
                To attempt to gain back some read efficiency, use full rows (the 10k
                width set here is much larger than the typical patch size).
                
                Implemented as an obs_subaru override instead of changing the default in
                pipe_tasks because the default is fine for most other applications.
            

            Show
            price Paul Price added a comment - This is already ported: price@price-laptop:~/LSST/obs/subaru (master=) $ git log -1 fa40ad2 commit fa40ad292b26d450bd9b30de01b5a48b39ff753e Author: Paul Price <price@astro.princeton.edu> Date: Thu Mar 10 14:42:44 2016 -0500   config: reduce coadd subregionSize The HSC SSP Ultra-Deep currently has a total of 85 exposures in y-band, so that reading 2k square blocks exceeds 4 GB/core. Reducing the block size reduces the memory use at the expense of a bit of read inefficiency. This choice (200 rows, for a 4k patch width) should give us plenty of room for growth. To attempt to gain back some read efficiency, use full rows (the 10k width set here is much larger than the typical patch size). Implemented as an obs_subaru override instead of changing the default in pipe_tasks because the default is fine for most other applications.

              People

              • Assignee:
                lauren Lauren MacArthur
                Reporter:
                swinbank John Swinbank
                Watchers:
                John Swinbank, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel