Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38205

Implement post-ingest update of raw regions in obscore

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: daf_butler
    • Labels:

      Description

      Obscore table has been added to embargo repo at USDF and is updated regularly during ingest. One thing still missing is the procedure to update regions for raw exposures. Raws do not get their exposures because visits are defined after raws are ingested. We need an after-burner task to perform that as a separate step. There is actually a CLI command already implemented for doing that, but we need something different:

      • Calling CLI command is slow, we want to implement the same thing as a sub-task of the ingest task. There could be further optimizations in this case, as we may have a list of ingested exposure ID and do not need to query registry for that list.
      • The code that implements that CLI command is in dax_obscore and that package is not in lsst_distrib, we need something that can be imported from existing dependencies (daf_butler) or add dax_obscore to obs_base dependencies.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            I'm fine with adding dax_obscore as a obs_base dependency.

            I assume you mean to add something that runs as part of visit definition, not part of ingest.

            Show
            tjenness Tim Jenness added a comment - I'm fine with adding dax_obscore as a obs_base dependency. I assume you mean to add something that runs as part of visit definition, not part of ingest.
            Hide
            salnikov Andy Salnikov added a comment -

            It can be run as a part of visit definition or right after it, I need to check how it can be implemented more efficiently.

            Show
            salnikov Andy Salnikov added a comment - It can be run as a part of visit definition or right after it, I need to check how it can be implemented more efficiently.
            Hide
            salnikov Andy Salnikov added a comment -

            Tim Jenness, could you check my changes when you have a minute. No rush, can wait until after JTM.

            Show
            salnikov Andy Salnikov added a comment - Tim Jenness , could you check my changes when you have a minute. No rush, can wait until after JTM. https://github.com/lsst/daf_butler/pull/803 https://github.com/lsst/obs_base/pull/446
            Hide
            tjenness Tim Jenness added a comment -

            Sorry for the delay. Looks good.

            Show
            tjenness Tim Jenness added a comment - Sorry for the delay. Looks good.

              People

              Assignee:
              salnikov Andy Salnikov
              Reporter:
              salnikov Andy Salnikov
              Reviewers:
              Tim Jenness
              Watchers:
              Andy Salnikov, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.