Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4954

Building issues with ci_hsc on OSX with SIP

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: ci_hsc
    • Labels:
      None
    • Team:
      Data Release Production

      Description

      Because the scons file in ci_hsc mixes lsst code with build code, simply typing scons to build the package on a system with sip will fail. The work around is to not let scons find the python version, but to specify it manually. The workaround is trivial, anyone on os x with sip can simply type:

      `which python` `which scons`
      

      on the command line to build the package

        Attachments

          Issue Links

            Activity

            Hide
            tjenness Tim Jenness added a comment -

            I'd rather the fix was to re-architect ci_hsc. I really don't think that scons should be running the validation directly. There should be a validation script that can be run. You can run it with scons if you want but I'm pretty sure the CI system we end up with for this is going to want to be running the integration tests outside of a scons build. You can't even guarantee that the python being used for scons is the python that you are using to run the code. Have I misunderstood how ci_hsc works?

            Show
            tjenness Tim Jenness added a comment - I'd rather the fix was to re-architect ci_hsc . I really don't think that scons should be running the validation directly. There should be a validation script that can be run. You can run it with scons if you want but I'm pretty sure the CI system we end up with for this is going to want to be running the integration tests outside of a scons build. You can't even guarantee that the python being used for scons is the python that you are using to run the code. Have I misunderstood how ci_hsc works?
            Hide
            swinbank John Swinbank added a comment -

            ci_hsc runs a bunch of interdependent data reduction and analysis steps, mostly by executing things from pipe_tasks. Using a tool like scons is a good way to do this, since we care about the dependencies between the steps rather than details about execution order or parallelization: a simple Python or shell script would be much less convenient (by the same logic, we don't just use simple scripts for the build system).

            That said, the architecture for test suites like this is clearly up to SQuaRE (with the caveat that our requirements need to be taken into account, of course). Since delivering ci_hsc in DM-3633, we've not had any feedback as to alternative approaches that would be preferred, so, as we move into the final stage of the HSC merge, we're trying to do the best with what we've got.

            Show
            swinbank John Swinbank added a comment - ci_hsc runs a bunch of interdependent data reduction and analysis steps, mostly by executing things from pipe_tasks . Using a tool like scons is a good way to do this, since we care about the dependencies between the steps rather than details about execution order or parallelization: a simple Python or shell script would be much less convenient (by the same logic, we don't just use simple scripts for the build system). That said, the architecture for test suites like this is clearly up to SQuaRE (with the caveat that our requirements need to be taken into account, of course). Since delivering ci_hsc in DM-3633 , we've not had any feedback as to alternative approaches that would be preferred, so, as we move into the final stage of the HSC merge, we're trying to do the best with what we've got.
            Hide
            swinbank John Swinbank added a comment -

            I think the doc updates on DM-4959 are all we're planning to do with respect to this in the foreseeable future. Nate Lust, are you happy to shut this down when that issue goes in?

            (This absolutely does not preclude a comprehensive redesign of ci_hsc in the future, of course!)

            Show
            swinbank John Swinbank added a comment - I think the doc updates on DM-4959 are all we're planning to do with respect to this in the foreseeable future. Nate Lust , are you happy to shut this down when that issue goes in? (This absolutely does not preclude a comprehensive redesign of ci_hsc in the future, of course!)
            Hide
            nlust Nate Lust added a comment -

            Yes, I am happy with doing all the work on 4959 and closing this ticket

            Show
            nlust Nate Lust added a comment - Yes, I am happy with doing all the work on 4959 and closing this ticket
            Hide
            swinbank John Swinbank added a comment -

            This work will be done on DM-4959.

            Show
            swinbank John Swinbank added a comment - This work will be done on DM-4959 .

              People

              • Assignee:
                Unassigned
                Reporter:
                nlust Nate Lust
                Watchers:
                John Swinbank, Joshua Hoblitt, Nate Lust, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel