Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-20019

Fix pickling of String Fields

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: afw

      Description

      While finishing DM-19616, I noticed a problem with the multiprocessing Pool and a schema that contained a String Field. I added a test to afw that reproduces this error, and I will fix it on this ticket.

      E           lsst.pex.exceptions.wrappers.LengthError: 
      E             File "src/table/FieldBase.cc", line 72, in lsst::afw::table::FieldBase<std::__cxx11::basic_string<char> >::FieldBase(int)
      		 
      E               Size must be provided when constructing a string field. {0}
      		 
      E           lsst::pex::exceptions::LengthError: 'Size must be provided when constructing a string field.'

      My guess is that I need to tweak the pybind11 py::pickle code in schema.cc, possibly special-casing the String FieldBase type.

        Attachments

          Issue Links

          blocks

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-19616 Make IngestIndexReferenceObjectsTask multiprocessing capable

          • Done

            Activity

            Ascending order - Click to sort in descending order
            Parejkoj John Parejko created issue -
            Parejkoj John Parejko made changes -
            Field Original Value New Value
            Link This issue blocks DM-19616 [ DM-19616 ]
            Parejkoj John Parejko made changes -
            Description While finishing DM-19616, I noticed a problem with the multiprocessing Pool and a schema that contained a {{String}} {{Field}}. I added a test to afw that reproduces this error, and I will fix it on this ticket.

            {code}}
            E lsst.pex.exceptions.wrappers.LengthError:
            E File "src/table/FieldBase.cc", line 72, in lsst::afw::table::FieldBase<std::__cxx11::basic_string<char> >::FieldBase(int)
            E Size must be provided when constructing a string field. {0}
            E lsst::pex::exceptions::LengthError: 'Size must be provided when constructing a string field.'
            {code}

            My guess is that I need to tweak the pybind11 {{py::pickle}} code in {{schema.cc}}, possibly special-casing the String FieldBase type.             		While finishing DM-19616, I noticed a problem with the multiprocessing Pool and a schema that contained a {{String}} {{Field}}. I added a test to afw that reproduces this error, and I will fix it on this ticket.

            {code}
            E lsst.pex.exceptions.wrappers.LengthError:
            E File "src/table/FieldBase.cc", line 72, in lsst::afw::table::FieldBase<std::__cxx11::basic_string<char> >::FieldBase(int)
            E Size must be provided when constructing a string field. {0}
            E lsst::pex::exceptions::LengthError: 'Size must be provided when constructing a string field.'
            {code}

            My guess is that I need to tweak the pybind11 {{py::pickle}} code in {{schema.cc}}, possibly special-casing the String FieldBase type.
            Parejkoj John Parejko made changes -
            Reviewers Krzysztof Findeisen [ krzys ]
            Status To Do [ 10001 ] In Review [ 10004 ]
            krzys Krzysztof Findeisen made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Parejkoj John Parejko made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            swinbank John Swinbank made changes -
            Epic Link DM-19979 [ 307530 ]

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Krzysztof Findeisen
              Watchers:
              Jim Bosch, John Parejko, John Swinbank, Krzysztof Findeisen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.