Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-8825

cannot run singleFrameDriver on hsc data with python3 due to "__builtin__.str" in *Mapper.paf

    XMLWordPrintable

    Details

    • Story Points:
      1
    • Epic Link:
    • Sprint:
      Alert Production S17 - 1
    • Team:
      Alert Production

      Description

      When attempting to reprocess some hsc data, I ran into a number of problems (one fixed in DM-8822). In this case, some of the python type definition in the obs_subaru Mapper.paf files are not valid: __builtin__.str doesn't exist on python3. Fortunately because of our use of futurize, we have builtins.str.

      Although I don't plan to write it as part of this ticket, having a python2+3 validation system for the policy files would be good.

        Attachments

          Issue Links

          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8841 ci_hsc is broken

          • Done
          relates to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-7324 Port ci_hsc to Python 3

          • Done

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-8822 Fix octal umask handling in ctrl_pool

          • Done

            Activity

            Ascending order - Click to sort in descending order
            4 older comments
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            I don't know, but if you put "ci_hsc" in jenkins, it fails in a couple of minutes. I haven't tried running it locally.

            Show
            Parejkoj John Parejko added a comment - I don't know, but if you put "ci_hsc" in jenkins, it fails in a couple of minutes. I haven't tried running it locally.
            Hide
            Permalink
            price Paul Price added a comment -

            This looks fine, but please ensure it passes ci_hsc (it's now fixed) before merging.

            Show
            price Paul Price added a comment - This looks fine, but please ensure it passes ci_hsc (it's now fixed) before merging.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the ci_hsc fix. Butler run succeeded after a rebase:

            https://ci.lsst.codes/job/stack-os-matrix/19856/

            I still want to hear from Nate Pease [X] about whether and how they dealt with this in other products.

            Show
            Parejkoj John Parejko added a comment - Thanks for the ci_hsc fix. Butler run succeeded after a rebase: https://ci.lsst.codes/job/stack-os-matrix/19856/ I still want to hear from Nate Pease [X] about whether and how they dealt with this in other products.
            Hide
            Permalink
            npease Nate Pease [X] (Inactive) added a comment -

            I haven't dealt with _builtin_.str in policy files before. This solution seems fine to me, if it's passing tests on Jenkins in py2 and py3.

            Show
            npease Nate Pease [X] (Inactive) added a comment - I haven't dealt with _ builtin _.str in policy files before. This solution seems fine to me, if it's passing tests on Jenkins in py2 and py3.
            Hide
            Permalink
            Parejkoj John Parejko added a comment -

            Thanks for the comments.

            As I noted above, ci_hsc passed Jenkins on py2, but ci_hsc doesn't run on py3 yet, so the only "proof" that this worked is that I was able to reprocess hsc data with a py3 stack. We may have to revisit this issue during the py3 port of ci_hsc.

            Merged and done.

            Show
            Parejkoj John Parejko added a comment - Thanks for the comments. As I noted above, ci_hsc passed Jenkins on py2, but ci_hsc doesn't run on py3 yet, so the only "proof" that this worked is that I was able to reprocess hsc data with a py3 stack. We may have to revisit this issue during the py3 port of ci_hsc. Merged and done.

              People

              Assignee:
              Parejkoj John Parejko
              Reporter:
              Parejkoj John Parejko
              Reviewers:
              Nate Pease [X] (Inactive), Paul Price
              Watchers:
              Jim Bosch, John Parejko, Nate Pease [X] (Inactive), Paul Price, Simon Krughoff, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.