Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11345

Redefine HSC "RC" dataset for bi-weeklies processing


    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Templates:
    • Story Points:
    • Epic Link:
    • Sprint:
      DRP F17-5, DRP F17-6, DRP S18-1, DRP S18-2
    • Team:
      Data Release Production


      We are currently processing what is known as the HSC RC dataset, where RC ==> Release Candidate, on a bi-weekly basis (for all even numbered weeklies). See https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC+dataset for full details. The purpose of this processing is to have a fully processed dataset as the pipeline evolves and be able track any quality/performance changes (for better or for worse - to properly praise/bless the former and catch & fix/revert the latter). Clearly, the dataset involved should be representative of the widest range of data quality/observing conditions expected, and is currently available to us through public HSC data. The RC dataset we have been using is not satisfactory in this respect. This issue is to remedy this situation by redefining (and perhaps renaming? RC ==> CI ??) the set of HSC tracts/visits to include in our bi-weeklies processing.

      A list of known data "characteristics" that this dataset should cover include tracts that suffer from the known issues to date, including:

      • a very well-behaved tract
      • a tract that includes visits that have proven challenging for astrometric fitting (see DM-11090)
      • a tract that includes visits that have proven challenging for photometric fitting (see DM-11927)
      • a tract including "too-good-seeing" visits (see DM-9855)
      • a tract including a wide range in seeing conditions (to test PSF-matching, see DM-8289)
      • tracts covering a range of "challenges" for deblending (possibly those identified in DM-9168)
      • a tract at the survey's edge (to check for issues arising due to patchy coverage - verify if 8766 WIDE still fills this bill)
      • a tract including galaxies suitable for assessing total magnitudes for "big" galaxies (list of examples along with "best" total magnitude estimates to be compared against being compiled by HSC member Song Huang)
      • a tract with observations in both the old i and new i2 HSC filters (see RFC-390)

      We will have to keep this dataset to a reasonable size (perhaps Hsin-Fang Chiang could comment on what "reasonable" means to her in numbers?) While we will certainly want to add pathological areas as we find/gain access to them (e.g. crowded fields), the set we define and agree upon here will be considered the canonical one until a need to expand/redefine is identified.


        1. PsfDist_UD_COSMOS_9813.png
          21 kB
          Lauren MacArthur
        2. PsfDist_WIDE_GAMMA15H_9615.png
          19 kB
          Lauren MacArthur
        3. PsfDist_WIDE_VVDS_9697.png
          18 kB
          Lauren MacArthur
        4. tracts_patches_UD_COSMOS_HSC-I.png
          97 kB
          Lauren MacArthur
        5. visits_UD_COSMOS_9813_I.png
          270 kB
          Lauren MacArthur
        6. visits_W_VVDS_9697_I.png
          317 kB
          Lauren MacArthur
        7. visits_WIDE_GAMA15H_9615_I.png
          286 kB
          Lauren MacArthur
        8. W-GAMA15H_i_area.png
          53 kB
          Lauren MacArthur
        9. WIDE_GAMA15H_9615_1,5_1,6-gri.png
          2.96 MB
          Lauren MacArthur
        10. W-VVDS_i_area.png
          51 kB
          Lauren MacArthur

          Issue Links



              • Assignee:
                lauren Lauren MacArthur
                lauren Lauren MacArthur
                Jim Bosch
                Bob Armstrong, Hsin-Fang Chiang, Ian Sullivan, Jeffrey Carlin, Jim Bosch, John Swinbank, Lauren MacArthur, Paul Price, Tim Morton
              • Votes:
                0 Vote for this issue
                9 Start watching this issue


                • Created:

                  Summary Panel