Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20019

Fix pickling of String Fields

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw

      Description

      While finishing DM-19616, I noticed a problem with the multiprocessing Pool and a schema that contained a String Field. I added a test to afw that reproduces this error, and I will fix it on this ticket.

      E           lsst.pex.exceptions.wrappers.LengthError: 
      E             File "src/table/FieldBase.cc", line 72, in lsst::afw::table::FieldBase<std::__cxx11::basic_string<char> >::FieldBase(int)
      E               Size must be provided when constructing a string field. {0}
      E           lsst::pex::exceptions::LengthError: 'Size must be provided when constructing a string field.'
      

      My guess is that I need to tweak the pybind11 py::pickle code in schema.cc, possibly special-casing the String FieldBase type.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            Krzysztof Findeisen: do you mind taking on this small-ish review (~100 lines)? If you can suggest a better solution to the Field/Key specializations here, I'd be interested to see it. Jim Bosch suggested this approach.

            Show
            Parejkoj John Parejko added a comment - Krzysztof Findeisen : do you mind taking on this small-ish review (~100 lines)? If you can suggest a better solution to the Field/Key specializations here, I'd be interested to see it. Jim Bosch suggested this approach.
            Show
            Parejkoj John Parejko added a comment - Jenkins run: https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29934/pipeline
            Hide
            krzys Krzysztof Findeisen added a comment -

            Sorry, I can't think of anything better; the Key and Field definitions are just too special-casey.

            Show
            krzys Krzysztof Findeisen added a comment - Sorry, I can't think of anything better; the Key and Field definitions are just too special-casey.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for the quick review. I cleaned up following your suggestions.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the quick review. I cleaned up following your suggestions. Merged and done.

              People

              • Assignee:
                Parejkoj John Parejko
                Reporter:
                Parejkoj John Parejko
                Reviewers:
                Krzysztof Findeisen
                Watchers:
                Jim Bosch, John Parejko, John Swinbank, Krzysztof Findeisen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel