Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-23428

gen3 ingest of auxtel data fails due to duplicate visit.id values

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Won't Fix
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_lsst
    • Labels:
      None
    • Team:
      Data Access and Database
    • Urgent?:
      No

      Description

      Ingesting auxTel data from 2020-01-28 into a gen3 repository fails, because many of the GROUPID fields in the files are the same.  Manually removing the files from the ingest set results in a working repo, although one missing the majority of the exposures.

        Attachments

          Issue Links

          is FF-depended by

          Improvement - An improvement or enhancement to an existing feature or task. DM-23302 Add ci_cpp package to allow calibration product generation to be tested

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-23445 Move visit definition out of raw ingest for Gen3

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            GROUPID is allowed to be the same – it means they are in the same visit. The problem is that gen3 doesn't do visit_id vs exposure_id properly.

            Show
            tjenness Tim Jenness added a comment - GROUPID is allowed to be the same – it means they are in the same visit. The problem is that gen3 doesn't do visit_id vs exposure_id properly.
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            I guess that means this is a Won't Fix, then? Tim Jenness, could you link to the ticket on Gen3 handling of visit_id vs exposure_id, please?

            Show
            swinbank John Swinbank added a comment - I guess that means this is a Won't Fix, then? Tim Jenness , could you link to the ticket on Gen3 handling of visit_id vs exposure_id, please?
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            I think it's entirely possible that butler ingest gets confused if there is more than one exposure per visit. DM-23208 is meant to use the exposure_group properly but DM-23445 is the real fix.

            Show
            tjenness Tim Jenness added a comment - I think it's entirely possible that butler ingest gets confused if there is more than one exposure per visit. DM-23208 is meant to use the exposure_group properly but DM-23445 is the real fix.
            Hide
            Permalink
            swinbank John Swinbank added a comment -

            Thanks!

            Show
            swinbank John Swinbank added a comment - Thanks!
            Hide
            Permalink
            czw Christopher Waters added a comment -

            I'm watching both of those fix tickets, and will confirm they fix the issue I had when they land. Thanks!

            Show
            czw Christopher Waters added a comment - I'm watching both of those fix tickets, and will confirm they fix the issue I had when they land. Thanks!

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              czw Christopher Waters
              Watchers:
              Christopher Waters, Jim Bosch, John Swinbank, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: