Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9873

PropertySet does not support values of None

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: daf_base
    • Labels:
      None

      Description

      In DM-5466, we needed to pass the results from ParseTask.getInfo() to the butler as a dataId. This is normally valid, since both are dictionaries, and even though getInfo() often contains extraneous entries that aren't relevant, the butler will usually ignore them. However, when parsing calibration files this dictionary contains some values that are set to None, since they will be filled in later. These extraneous keys are then placed in ButlerLocation.additionalData (butlerLocation.py:221), which throws an exception as it is a PropertySet and does not support python None as a value.

      DM-5466 has a work-around that strips these None values from the dictionary, but this is inelegant. The main driver for excluding None from PropertySet seems to compatibility with FITS headers. This seems like an unwarranted mixing of data model and persistence formats. Unless there is some advantage to not being able to store None in our dictionary-like objects, it seems preferable to shift the burden of accommodating FITS's peculiarities onto the persistence layer rather than PropertySet.

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            Colin Slater do you know what I need to reverse from DM-5466 to test that None now works?

            Show
            tjenness Tim Jenness added a comment - Colin Slater do you know what I need to reverse from DM-5466 to test that None now works?
            Hide
            ctslater Colin Slater added a comment -

            I can't locate anything that looks like it would have been the culprit for filing this ticket; DM-5466 went through a few permutations so the final code that was merged might not have had the same problem we saw originally.

            Show
            ctslater Colin Slater added a comment - I can't locate anything that looks like it would have been the culprit for filing this ticket; DM-5466 went through a few permutations so the final code that was merged might not have had the same problem we saw originally.
            Hide
            tjenness Tim Jenness added a comment -

            Jim Bosch thanks for agreeing to look at this.

            Show
            tjenness Tim Jenness added a comment - Jim Bosch thanks for agreeing to look at this.
            Hide
            jbosch Jim Bosch added a comment -

            I think the changes for me to look at were on DM-18864, right?  Or should I look at the ones here, too?

            Show
            jbosch Jim Bosch added a comment - I think the changes for me to look at were on DM-18864 , right?  Or should I look at the ones here, too?
            Hide
            tjenness Tim Jenness added a comment -

            The changes were to look at this one. I referred you to DM-18864 because of my recent addition there to work around the problem here based on the Slack discussion this morning.

            Show
            tjenness Tim Jenness added a comment - The changes were to look at this one. I referred you to DM-18864 because of my recent addition there to work around the problem here based on the Slack discussion this morning.

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                ctslater Colin Slater
                Reviewers:
                Jim Bosch, Kian-Tat Lim
                Watchers:
                Colin Slater, Jim Bosch, John Swinbank, Kian-Tat Lim, Meredith Rawls, Simon Krughoff, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel