Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11345

Redefine HSC "RC" dataset for bi-weeklies processing

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Templates:
    • Story Points:
      8
    • Epic Link:
    • Sprint:
      DRP F17-5, DRP F17-6, DRP S18-1, DRP S18-2
    • Team:
      Data Release Production

      Description

      We are currently processing what is known as the HSC RC dataset, where RC ==> Release Candidate, on a bi-weekly basis (for all even numbered weeklies). See https://confluence.lsstcorp.org/display/DM/Reprocessing+of+the+HSC+RC+dataset for full details. The purpose of this processing is to have a fully processed dataset as the pipeline evolves and be able track any quality/performance changes (for better or for worse - to properly praise/bless the former and catch & fix/revert the latter). Clearly, the dataset involved should be representative of the widest range of data quality/observing conditions expected, and is currently available to us through public HSC data. The RC dataset we have been using is not satisfactory in this respect. This issue is to remedy this situation by redefining (and perhaps renaming? RC ==> CI ??) the set of HSC tracts/visits to include in our bi-weeklies processing.

      A list of known data "characteristics" that this dataset should cover include tracts that suffer from the known issues to date, including:

      • a very well-behaved tract
      • a tract that includes visits that have proven challenging for astrometric fitting (see DM-11090)
      • a tract that includes visits that have proven challenging for photometric fitting (see DM-11927)
      • a tract including "too-good-seeing" visits (see DM-9855)
      • a tract including a wide range in seeing conditions (to test PSF-matching, see DM-8289)
      • tracts covering a range of "challenges" for deblending (possibly those identified in DM-9168)
      • a tract at the survey's edge (to check for issues arising due to patchy coverage - verify if 8766 WIDE still fills this bill)
      • a tract including galaxies suitable for assessing total magnitudes for "big" galaxies (list of examples along with "best" total magnitude estimates to be compared against being compiled by HSC member Song Huang)
      • a tract with observations in both the old i and new i2 HSC filters (see RFC-390)
      • PLEASE ADD YOUR FAVORITE PATHOLOGY HERE

      We will have to keep this dataset to a reasonable size (perhaps Hsin-Fang Chiang could comment on what "reasonable" means to her in numbers?) While we will certainly want to add pathological areas as we find/gain access to them (e.g. crowded fields), the set we define and agree upon here will be considered the canonical one until a need to expand/redefine is identified.

        Attachments

        1. PsfDist_UD_COSMOS_9813.png
          PsfDist_UD_COSMOS_9813.png
          21 kB
        2. PsfDist_WIDE_GAMMA15H_9615.png
          PsfDist_WIDE_GAMMA15H_9615.png
          19 kB
        3. PsfDist_WIDE_VVDS_9697.png
          PsfDist_WIDE_VVDS_9697.png
          18 kB
        4. tracts_patches_UD_COSMOS_HSC-I.png
          tracts_patches_UD_COSMOS_HSC-I.png
          97 kB
        5. visits_UD_COSMOS_9813_I.png
          visits_UD_COSMOS_9813_I.png
          270 kB
        6. visits_W_VVDS_9697_I.png
          visits_W_VVDS_9697_I.png
          317 kB
        7. visits_WIDE_GAMA15H_9615_I.png
          visits_WIDE_GAMA15H_9615_I.png
          286 kB
        8. W-GAMA15H_i_area.png
          W-GAMA15H_i_area.png
          53 kB
        9. WIDE_GAMA15H_9615_1,5_1,6-gri.png
          WIDE_GAMA15H_9615_1,5_1,6-gri.png
          2.96 MB
        10. W-VVDS_i_area.png
          W-VVDS_i_area.png
          51 kB

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  lauren Lauren MacArthur
                  Reporter:
                  lauren Lauren MacArthur
                  Reviewers:
                  Jim Bosch
                  Watchers:
                  Bob Armstrong, Hsin-Fang Chiang, Ian Sullivan, Jim Bosch, John Swinbank, Lauren MacArthur, Paul Price, Tim Morton
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  8 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel