Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-22708

Fix decam gen3 ingest

    Details

    • Templates:
    • Story Points:
      8
    • Sprint:
      AP S20-2 (January)
    • Team:
      Alert Production
    • Urgent?:
      No

      Description

      It turns out the work I did on decam for DM-20763 was not complete: I only tested that one dataId could be retrieved from the ingested raw data, but there are two ccds in that file in two different HDUs and the second one is not actually getting ingested. The gen2 CameraMapper path for a decam raw looks like decam%(visit)07d.fits.fz[%(hdu)d], so the hdu number is baked in there. As far as I can tell, gen3 doesn't have any way to encode hdu number. Maybe this has to be a further specialization in the RawFormatter. File paths that go into butler.ingest have to actually exist, so we will have to do something to the raw paths that we pass into `butler.ingest`.

      The gen2 butler registry has an `hdu` field that encodes which hdu to get that raw from. It doesn't look like there's an equivalent `hdu` field anywhere in the gen3 registry: could we add an extra `hdu` field to the `posix_datastore_records` table? That's my best guess as to where it should live.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Reviewers:
                Tim Jenness
                Watchers:
                Colin Slater, Jim Bosch, John Parejko, John Swinbank, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel