Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-38205

Implement post-ingest update of raw regions in obscore

    XMLWordPrintable

Details

    • Story
    • Status: Done
    • Resolution: Done
    • None
    • daf_butler

    Description

      Obscore table has been added to embargo repo at USDF and is updated regularly during ingest. One thing still missing is the procedure to update regions for raw exposures. Raws do not get their exposures because visits are defined after raws are ingested. We need an after-burner task to perform that as a separate step. There is actually a CLI command already implemented for doing that, but we need something different:

      • Calling CLI command is slow, we want to implement the same thing as a sub-task of the ingest task. There could be further optimizations in this case, as we may have a list of ingested exposure ID and do not need to query registry for that list.
      • The code that implements that CLI command is in dax_obscore and that package is not in lsst_distrib, we need something that can be imported from existing dependencies (daf_butler) or add dax_obscore to obs_base dependencies.

      Attachments

        Issue Links

          Activity

            tjenness Tim Jenness added a comment -

            I'm fine with adding dax_obscore as a obs_base dependency.

            I assume you mean to add something that runs as part of visit definition, not part of ingest.

            tjenness Tim Jenness added a comment - I'm fine with adding dax_obscore as a obs_base dependency. I assume you mean to add something that runs as part of visit definition, not part of ingest.

            It can be run as a part of visit definition or right after it, I need to check how it can be implemented more efficiently.

            salnikov Andy Salnikov added a comment - It can be run as a part of visit definition or right after it, I need to check how it can be implemented more efficiently.

            tjenness, could you check my changes when you have a minute. No rush, can wait until after JTM.

            salnikov Andy Salnikov added a comment - tjenness , could you check my changes when you have a minute. No rush, can wait until after JTM. https://github.com/lsst/daf_butler/pull/803 https://github.com/lsst/obs_base/pull/446
            tjenness Tim Jenness added a comment -

            Sorry for the delay. Looks good.

            tjenness Tim Jenness added a comment - Sorry for the delay. Looks good.

            People

              salnikov Andy Salnikov
              salnikov Andy Salnikov
              Tim Jenness
              Andy Salnikov, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Jenkins

                  No builds found.