Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-33803

logInfo returns platform-specific units

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Invalid
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: utils
    • Labels:
    • Story Points:
      2
    • Team:
      Architecture
    • Urgent?:
      No

      Description

      Lee Kelvin was talking about analyzing the contents of the task metadata to look at memory usage and noted that his numbers had to be scaled by 1024.

      This horrifies me because it now seems that we are storing task metadata (and have been for years) where you need to know where you ran it in order to work out how much memory it was using.

      In another part of utils we have:

      unit = u.kibibyte if platform.system() == "Linux" else u.byte
      

      so the fix to logInfo is trivial. What I don't know is how I can fix it without everyone's scripts breaking. We can't let this situation persist though and nothing in the code says 1024 byte units. MacOS returns bytes (and I don't know what freebsd returns).

      Do I put a version number into the logInfo data structure? Do I put a unit string in there?

        Attachments

          Issue Links

            Activity

            Hide
            krzys Krzysztof Findeisen added a comment -

            This is a duplicate of DM-20970. Note that the utils code is incomplete, as there is at least a third case (SunOS).

            Show
            krzys Krzysztof Findeisen added a comment - This is a duplicate of DM-20970 . Note that the utils code is incomplete, as there is at least a third case (SunOS) .
            Hide
            ktl Kian-Tat Lim added a comment -

            I think you need a version even if you have a unit string, so putting in a version and standardizing the units sounds best.

            Show
            ktl Kian-Tat Lim added a comment - I think you need a version even if you have a unit string, so putting in a version and standardizing the units sounds best.
            Hide
            tjenness Tim Jenness added a comment -

            Thanks for pointing out the duplication. Now I'm wondering if we can justify putting some smarts into the thing we store in TaskMetadata to allow the units to be attached. Currently TaskMetadata is effectively a special dict that has no idea about content but I'll see what happens if I store an astropy Quantity in there.

            Show
            tjenness Tim Jenness added a comment - Thanks for pointing out the duplication. Now I'm wondering if we can justify putting some smarts into the thing we store in TaskMetadata to allow the units to be attached. Currently TaskMetadata is effectively a special dict that has no idea about content but I'll see what happens if I store an astropy Quantity in there.

              People

              Assignee:
              tjenness Tim Jenness
              Reporter:
              tjenness Tim Jenness
              Watchers:
              Kian-Tat Lim, Krzysztof Findeisen, Lee Kelvin, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins Builds

                  No builds found.