Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-11040

Implement persistence of DIAObjects and DIASources

    Details

    • Story Points:
      10
    • Sprint:
      Alert Production F17 - 8, Alert Production F17 - 9
    • Team:
      Alert Production

      Description

      Create abstraction layer interface to persist DIAObjects and DIASource into a data structure. Data structure will either be the L1DB prototype or the butler.

        Attachments

          Activity

          Hide
          cmorrison Chris Morrison added a comment -

          Matching algorithm is in review it's time to start looking at at persisting DIAObjects and sources. The current plan is to use the prototype L1 database from Andy Salnikov. This ticket will involve writing AssociationTask to tie everything together. Loading and writing to the L1 database will either be done by AssociationTask or a SubTask.

          Show
          cmorrison Chris Morrison added a comment - Matching algorithm is in review it's time to start looking at at persisting DIAObjects and sources. The current plan is to use the prototype L1 database from Andy Salnikov . This ticket will involve writing AssociationTask to tie everything together. Loading and writing to the L1 database will either be done by AssociationTask or a SubTask.
          Hide
          cmorrison Chris Morrison added a comment -

          Mainly a review for the implementation of AssociationDBTask and it's unittest. Files to look at are tests/test_association_db_sqlite.py and python/lsst/ap/association/assoc_db_sqlite.py

          This is mainly to set up a prototype, placeholder database for ap_verify and ap_pipe. Finalizing AssociationTask has been pushed to ticket DM-11747.

          Show
          cmorrison Chris Morrison added a comment - Mainly a review for the implementation of AssociationDBTask and it's unittest. Files to look at are tests/test_association_db_sqlite.py and python/lsst/ap/association/assoc_db_sqlite.py This is mainly to set up a prototype, placeholder database for ap_verify and ap_pipe. Finalizing AssociationTask has been pushed to ticket DM-11747 .
          Hide
          krzys Krzysztof Findeisen added a comment -

          I've taken a look and made some comments. I realize some of my requests will probably be deferred to DM-11747 or some other ticket, but I think clearing up the following points is essential:

          1. Better use of the database (in particular, using joins with the dia_objects_to_dia_soruces table, and ensuring data integrity).
          2. Clarifying which methods will actually be called by other classes, and making sure those can be treated as self-contained/self-consistent operations
          3. Testing whether the system behaves as expected in the unit tests
          Show
          krzys Krzysztof Findeisen added a comment - I've taken a look and made some comments. I realize some of my requests will probably be deferred to DM-11747 or some other ticket, but I think clearing up the following points is essential: Better use of the database (in particular, using joins with the dia_objects_to_dia_soruces table, and ensuring data integrity). Clarifying which methods will actually be called by other classes, and making sure those can be treated as self-contained/self-consistent operations Testing whether the system behaves as expected in the unit tests
          Hide
          cmorrison Chris Morrison added a comment -

          Incorporated first round of reviewer comments into the code. Ready for second review pass.

          Show
          cmorrison Chris Morrison added a comment - Incorporated first round of reviewer comments into the code. Ready for second review pass.
          Hide
          krzys Krzysztof Findeisen added a comment -

          Looks pretty good. Feel free to merge once you've addressed the SQL/testing questions.

          Show
          krzys Krzysztof Findeisen added a comment - Looks pretty good. Feel free to merge once you've addressed the SQL/testing questions.
          Hide
          cmorrison Chris Morrison added a comment -

          Finished review and merged to master.

          Show
          cmorrison Chris Morrison added a comment - Finished review and merged to master.

            People

            • Assignee:
              cmorrison Chris Morrison
              Reporter:
              cmorrison Chris Morrison
              Reviewers:
              Krzysztof Findeisen
              Watchers:
              Chris Morrison, Eric Bellm, Krzysztof Findeisen, Simon Krughoff
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Summary Panel