Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10490

Cache camera in HscMapper

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_subaru
    • Labels:
      None
    • Story Points:
      1
    • Team:
      Data Release Production

      Description

      The lion's share of the time in instantiating a butler goes to building the camera object, because that involves parsing a Config with many lines, and a stat call for each line to get the traceback. Multiple butler instantiations (e.g., for multiprocessing or pipe_drivers) could benefit from caching the camera once it's built.

        Attachments

          Issue Links

            Activity

            Hide
            price Paul Price added a comment -

            Merlin Fisher-Levine, are you able to review this, please?

            It hasn't gone all the way through ci_hsc yet because that's broken on master; I will make sure it passes before merging.

            price@pap-laptop:~/LSST/obs_subaru (tickets/DM-10490=) $ git sub
            commit 31e1df31151bf23d2cf7d1cbdf306f328d0a7b96
            Author: Paul Price <price@astro.princeton.edu>
            Date:   Tue May 9 11:39:00 2017 -0400
             
                HscMapper: cache the camera object
                
                The lion's share of the time in instantiating a butler goes to
                building the camera object, because that involves parsing a Config
                with many lines, and a stat call for each line to get the traceback.
                We cache the camera once it's built so that subsequent butler
                instantiations (e.g., for multiprocessing or pipe_drivers) don't
                suffer the same overhead.
             
             python/lsst/obs/hsc/hscMapper.py | 22 ++++++++++++++++++++++
             tests/testCamera.py              |  4 +++-
             tests/testDistortion.py          |  4 +++-
             tests/testRepository.py          |  5 ++++-
             4 files changed, 32 insertions(+), 3 deletions(-)
            

            Show
            price Paul Price added a comment - Merlin Fisher-Levine , are you able to review this, please? It hasn't gone all the way through ci_hsc yet because that's broken on master; I will make sure it passes before merging. price@pap-laptop:~/LSST/obs_subaru (tickets/DM-10490=) $ git sub commit 31e1df31151bf23d2cf7d1cbdf306f328d0a7b96 Author: Paul Price <price@astro.princeton.edu> Date: Tue May 9 11:39:00 2017 -0400   HscMapper: cache the camera object The lion's share of the time in instantiating a butler goes to building the camera object, because that involves parsing a Config with many lines, and a stat call for each line to get the traceback. We cache the camera once it's built so that subsequent butler instantiations (e.g., for multiprocessing or pipe_drivers) don't suffer the same overhead.   python/lsst/obs/hsc/hscMapper.py | 22 ++++++++++++++++++++++ tests/testCamera.py | 4 +++- tests/testDistortion.py | 4 +++- tests/testRepository.py | 5 ++++- 4 files changed, 32 insertions(+), 3 deletions(-)
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            For the record, this will be my first review of any significance, but sure

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - For the record, this will be my first review of any significance, but sure
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Would getting to it on Thursday morning be OK, or is it more urgent than that?

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Would getting to it on Thursday morning be OK, or is it more urgent than that?
            Hide
            price Paul Price added a comment -

            I was hoping to get this into the next HSC release (aiming for maybe Thursday?), so if you can get to it today that would be ideal. If you can't get to it by tomorrow, please let me know and I'll ask someone else.

            Show
            price Paul Price added a comment - I was hoping to get this into the next HSC release (aiming for maybe Thursday?), so if you can get to it today that would be ideal. If you can't get to it by tomorrow, please let me know and I'll ask someone else.
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            No problem, I'll just do it now then

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - No problem, I'll just do it now then
            Hide
            mfisherlevine Merlin Fisher-Levine added a comment -

            Looks good to merge

            Show
            mfisherlevine Merlin Fisher-Levine added a comment - Looks good to merge
            Hide
            price Paul Price added a comment -

            Thanks Merlin.

            Jenkins passed.

            Merged to master.

            Show
            price Paul Price added a comment - Thanks Merlin. Jenkins passed . Merged to master.

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Merlin Fisher-Levine
                Watchers:
                Merlin Fisher-Levine, Paul Price
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: