Uploaded image for project: 'Request For Comments'
  1. Request For Comments
  2. RFC-482

Retire use of Boost.Serialization persistence before Gen3 Butler transition

    XMLWordPrintable

    Details

    • Type: RFC
    • Status: Implemented
    • Resolution: Done
    • Component/s: DM
    • Labels:
      None

      Description

      Boost.Serialization-based persistence currently lives on in the stack only in serialization of PropertyList and PropertySet, and in the pipelines I know of, only when saving Task metadata.

      Several people have proposed switching to YAML-based persistence for these datasets, and (I believe) some partial implementations are underway.

      Because Boost-based serialization is closely tied to daf_persistence code we're hoping to fully replace, I would like to transition to YAML for these datasets before we get to Gen3, and for Gen3 to not to support reading Task metadata persisted in that format (unlike all other Gen2 datasets, which we do intend to support reading via Gen3).

       

        Attachments

          Issue Links

          is triggering

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-14503 Finish implementing YAML storage for PropertyList/PropertySet

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15082 Switch to YamlStorage instead of BoostStorage in all obs packages

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-14504 Remove daf_persistence hooks in PropertySet, PropertyList, and afw objects

          • Invalid
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15593 daf_persistence segfaults on Princeton tiger2 cluster

          • Done

          RFC - An issue whose primary purpose is to advertise an intent and allow stakeholders to comment. RFC-222 Remove most formatter and boost::serialization code

          • Implemented

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-4927 ci_hsc doesn't produce readable processCcd_metadata

          • Invalid

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            +1 That sounds like a great idea.

            Show
            Parejkoj John Parejko added a comment - +1 That sounds like a great idea.
            Hide
            Permalink
            ktl Kian-Tat Lim added a comment -

            One partial implementation: https://github.com/lsst/daf_persistence/commit/6ed767db014fbcec7478ff7d616de3b0fc691b14

            Show
            ktl Kian-Tat Lim added a comment - One partial implementation: https://github.com/lsst/daf_persistence/commit/6ed767db014fbcec7478ff7d616de3b0fc691b14
            Hide
            Permalink
            ctslater Colin Slater added a comment -

            Yes please.

            Show
            ctslater Colin Slater added a comment - Yes please.
            Hide
            Permalink
            jbosch Jim Bosch added a comment - - edited

            Adopted.

            T/CAMs and free agents: please feel free to help us get you a Gen3 Butler sooner by finding someone to do this work.  If you don't, one of us will do it instead of working on the Gen3 Butler for a week or so .

            Show
            jbosch Jim Bosch added a comment - - edited Adopted. T/CAMs and free agents: please feel free to help us get you a Gen3 Butler sooner by finding someone to do this work.  If you don't, one of us will do it instead of working on the Gen3 Butler for a week or so  .
            Hide
            Permalink
            tjenness Tim Jenness added a comment -

            Jim Bosch all triggered tickets are completed. Does this mean we can mark this RFC as implemented? Also, does this mean we can remove the tests of boost::serialization in daf_persistence that are causing the problems in DM-15593?

            Show
            tjenness Tim Jenness added a comment - Jim Bosch all triggered tickets are completed. Does this mean we can mark this RFC as implemented? Also, does this mean we can remove the tests of boost::serialization in daf_persistence that are causing the problems in DM-15593 ?
            Hide
            Permalink
            jbosch Jim Bosch added a comment -

            Just marked as implemented.  I think those tests are probably gone already; moving further inspection and conversation to that ticket.

            Show
            jbosch Jim Bosch added a comment - Just marked as implemented.  I think those tests are probably gone already; moving further inspection and conversation to that ticket.

              People

              Assignee:
              jbosch Jim Bosch
              Reporter:
              jbosch Jim Bosch
              Watchers:
              Colin Slater, Jim Bosch, John Parejko, John Swinbank, Kian-Tat Lim, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Planned End:

                  Jenkins

                  No builds found.