Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22954

Define DC2.2i tracts to be included in the validation dataset

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Define DC2.2i tracts to be included in the validation dataset. 

        Attachments

          Issue Links

            Activity

            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Based on the recommendations from DESC (big thanks to Heather Kelly James Chiang Johann Cohen-Tanugi!) we are planning to use tracts 3828 and 3829 as our validation dataset. Both tracts are typical WFD tracts. From year 1, tract=3828 has 166 visits, and tract=3829 has 168 visits.  They sum to 217 visits for both tracts.
            tract=3828 is also the standard test tract on the DESC side; it would be good to use the same test tract for easier comparisons.  When the year 2 data become available we would like to include them too.

            There are also data from deep drilling fields available.  For example tract=4849 is the deepest tract, has 2471 visits, and is the standard DDF test field. These could be useful for really deep coadds, stress tests for hardware resources, and so on. 
            My opinion is if this is to test the processing I would start with WFD, as it's simpler. Also if we have limited resources (people+machine) the WFD tracts mean a faster turnaround. I'm thinking this mostly in the context of the prospective rapid stack changes this month and a representative tract for the Algorithm workshop, but likely miss many other considerations. Eventually we'll probably want the DDF tract too, but probably not so important right now.  Yusra AlSayyad what are your thoughts?  

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Based on the recommendations from DESC (big thanks to Heather Kelly James Chiang Johann Cohen-Tanugi !) we are planning to use tracts 3828 and 3829 as our validation dataset. Both tracts are typical WFD tracts. From year 1, tract=3828 has 166 visits, and tract=3829 has 168 visits.  They sum to 217 visits for both tracts. tract=3828 is also the standard test tract on the DESC side; it would be good to use the same test tract for easier comparisons.  When the year 2 data become available we would like to include them too. There are also data from deep drilling fields available.  For example tract=4849 is the deepest tract, has 2471 visits, and is the standard DDF test field. These could be useful for really deep coadds, stress tests for hardware resources, and so on.  My opinion is if this is to test the processing I would start with WFD, as it's simpler. Also if we have limited resources (people+machine) the WFD tracts mean a faster turnaround. I'm thinking this mostly in the context of the prospective rapid stack changes this month and a representative tract for the Algorithm workshop, but likely miss many other considerations. Eventually we'll probably want the DDF tract too, but probably not so important right now.  Yusra AlSayyad what are your thoughts?  
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            (Attaching the visit list for tract=3828^3829) 

            Show
            hchiang2 Hsin-Fang Chiang added a comment - (Attaching the visit list for tract=3828^3829) 
            Hide
            yusra Yusra AlSayyad added a comment -

            I agree with your assessment. 

            Show
            yusra Yusra AlSayyad added a comment - I agree with your assessment. 
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            I'm adding a file visits_y1y2_t3828y3829.txt with visits IDs from both y1 and y2 for these two tracts

            Show
            hchiang2 Hsin-Fang Chiang added a comment - I'm adding a file visits_y1y2_t3828y3829.txt with visits IDs from both y1 and y2 for these two tracts
            Hide
            hchiang2 Hsin-Fang Chiang added a comment -

            Currently we have 345 visits from y1 and partial y2; I put a file at `/datasets/DC2/repoRun2.2i/ids_filter_visits.txt` that contains the visit IDs for each filter.

            There may be more y2 data available from DESC later. But I think we have a good set to start with now. I'm gonna close this ticket.

            Show
            hchiang2 Hsin-Fang Chiang added a comment - Currently we have 345 visits from y1 and partial y2; I put a file at `/datasets/DC2/repoRun2.2i/ids_filter_visits.txt` that contains the visit IDs for each filter. There may be more y2 data available from DESC later. But I think we have a good set to start with now. I'm gonna close this ticket.

              People

              Assignee:
              hchiang2 Hsin-Fang Chiang
              Reporter:
              lguy Leanne Guy
              Reviewers:
              Yusra AlSayyad
              Watchers:
              Hsin-Fang Chiang, Leanne Guy, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Due:
                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.