Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-5911

Fix circular references in Mapper objects

    Details

    • Type: Improvement
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: obs_base
    • Labels:
      None
    • Templates:
    • Story Points:
      1
    • Team:
      Architecture

      Description

      Whilst running tests with pytest and the new file descriptor leak checker it became clear that Mapper objects were not freeing their resources when they were deleted. In particular, the registry objects remained and the associated sqlite database files were opened. This led to pytest running out of file descriptors when large test suites were being executed.

      The problem turns out to be the dynamically created map functions. These are created as functions (not bound methods) attached to an instance. Since they are not bound methods the instance object (self) has to be passed in to closure. This leads to self containing a reference to a function that contains a reference to self and this prevents the Mapper from ever being garbage collected (leading to all the resources being retained).

      A short term fix is pass the mappers into the closures using weakref.

      Eventually it would be nice to consistently make the map_ items bound methods rather than attaching them as functions but that is beyond the scope of this ticket.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tjenness Tim Jenness
                Reporter:
                tjenness Tim Jenness
                Watchers:
                Kian-Tat Lim, Nate Pease, Tim Jenness
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel