Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-21049

Ensure automatic ingest of BOT data taken at SLAC and rsync'ed to NCSA

    Details

    • Type: Improvement
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: butler
    • Labels:
      None
    • Templates:
    • Team:
      Data Facility

      Description

      HI, we will soon (maybe second week of September) be starting to take new BOT data at SLAC, with the expectation that this data will be rsynch'ed to NCSA (the so called 9-raft testing at SLAC). This is a follow-on to early "2-ETU" testing at SLAC, where the data was rsynch'ed to NCSA, but as far as I know not ingested. I have seen discussion from various people which seems to indicate it should be possible to automate the ingest process (using Gen2 butler), so the goal of this ticket is to increase the probability that the right people are talking to each other to maximize the probability of this ingest working in time for 9-raft data (hence the large number of people added to the watchers list, sorry if I missed people, or added people who should not be involved). The 2-ETU data is as far as I know still not ingested, so it could also be used as a test case.

      One additional wrinkle, although we expect to have 9 science rafts and 4 corner rafts installed (+-1) due to disk space/performance reasons we may choose to not read out all rafts for all images, in which case there will be no FITS files for the rafts which are not read out. I am unclear if this is easy for the butler to handle.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tjohnson Tony Johnson
              Watchers:
              Aaron Roodman, Eric Charles, Eric Morganson, Glenn Morris, James Chiang, Kian-Tat Lim, Margaret Gelman, Michelle Butler, Michelle Gower, Robert Gruendl, Robert Lupton, Seth Digel, Tim Jenness, Tony Johnson, Wil O'Mullane
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:

                Summary Panel