Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-15331

Revisit how dataset subsets are defined in lsst.verify when subsets are implented in Gen3 middleware

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: squash, verify
    • Labels:
      None

      Description

      Currenlty in verify we add that as query metadata in specifications:

       
      specs/validate_drp/cfht_ugrizy/base.yaml
       
        dataset_repo_url: 'https://github.com/lsst/validation_data_cfht.git'
        dataset_name: 'validation_data_cfht'
        visits: [849375, 850587]
        ccds: [12, 13, 14, 21, 22, 23]
      
      

      Simon Krughoff:
      I am really concerned about putting actual visits and ccds in the spec. I'd much rather specify a tag on the dataset that specifies the subset to be used. That tag can have a central definition. Since we don't actually use this info (as far as I know), I'd prefer to leave it out rather than having it in two places that can get out of sync.

      I know tags for subsets isn't currently possible, but it will be available in the GEN3 system.

      Angelo Fausti
      Right, I'll open a ticket to revisit this when time comes. At least adding visits and ccds to query metadata in specs is consistent with adding filter_name for now. This is also related with the definition of datasets we need in SQuaSH.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Is this ticket still relevant? Where did these specifications go when validate_drp was deprecated? ccds and filter_name are not consistent with gen3 dataIds (we use detector and physical_filter).

            What is the gen3 subset functionality that is being asked for?

            Show
            tjenness Tim Jenness added a comment - Is this ticket still relevant? Where did these specifications go when validate_drp was deprecated? ccds and filter_name are not consistent with gen3 dataIds (we use detector and physical_filter). What is the gen3 subset functionality that is being asked for?
            Hide
            jsick Jonathan Sick added a comment -

            I don't think this is relevant anymore now that we've been using lsst.verify with generation 3 middleware for a while and ticket hasn't come up.

            Show
            jsick Jonathan Sick added a comment - I don't think this is relevant anymore now that we've been using lsst.verify with generation 3 middleware for a while and ticket hasn't come up.

              People

              Assignee:
              krughoff Simon Krughoff
              Reporter:
              afausti Angelo Fausti
              Watchers:
              Angelo Fausti, Jonathan Sick, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.