Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8825

cannot run singleFrameDriver on hsc data with python3 due to "__builtin__.str" in *Mapper.paf

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Epic Link:
    • Sprint:
      Alert Production S17 - 1
    • Team:
      Alert Production

      Description

      When attempting to reprocess some hsc data, I ran into a number of problems (one fixed in DM-8822). In this case, some of the python type definition in the obs_subaru Mapper.paf files are not valid: __builtin__.str doesn't exist on python3. Fortunately because of our use of futurize, we have builtins.str.

      Although I don't plan to write it as part of this ticket, having a python2+3 validation system for the policy files would be good.

        Attachments

          Issue Links

            Activity

            Hide
            Parejkoj John Parejko added a comment -

            I don't know, but if you put "ci_hsc" in jenkins, it fails in a couple of minutes. I haven't tried running it locally.

            Show
            Parejkoj John Parejko added a comment - I don't know, but if you put "ci_hsc" in jenkins, it fails in a couple of minutes. I haven't tried running it locally.
            Hide
            price Paul Price added a comment -

            This looks fine, but please ensure it passes ci_hsc (it's now fixed) before merging.

            Show
            price Paul Price added a comment - This looks fine, but please ensure it passes ci_hsc (it's now fixed) before merging.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for the ci_hsc fix. Butler run succeeded after a rebase:

            https://ci.lsst.codes/job/stack-os-matrix/19856/

            I still want to hear from Nate Pease about whether and how they dealt with this in other products.

            Show
            Parejkoj John Parejko added a comment - Thanks for the ci_hsc fix. Butler run succeeded after a rebase: https://ci.lsst.codes/job/stack-os-matrix/19856/ I still want to hear from Nate Pease about whether and how they dealt with this in other products.
            Hide
            npease Nate Pease added a comment -

            I haven't dealt with _builtin_.str in policy files before. This solution seems fine to me, if it's passing tests on Jenkins in py2 and py3.

            Show
            npease Nate Pease added a comment - I haven't dealt with _ builtin _.str in policy files before. This solution seems fine to me, if it's passing tests on Jenkins in py2 and py3.
            Hide
            Parejkoj John Parejko added a comment -

            Thanks for the comments.

            As I noted above, ci_hsc passed Jenkins on py2, but ci_hsc doesn't run on py3 yet, so the only "proof" that this worked is that I was able to reprocess hsc data with a py3 stack. We may have to revisit this issue during the py3 port of ci_hsc.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the comments. As I noted above, ci_hsc passed Jenkins on py2, but ci_hsc doesn't run on py3 yet, so the only "proof" that this worked is that I was able to reprocess hsc data with a py3 stack. We may have to revisit this issue during the py3 port of ci_hsc. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Nate Pease, Paul Price
              Watchers:
              Jim Bosch, John Parejko, Nate Pease, Paul Price, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: