Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18895

Use std::ptrdiff_t as index type in jointcal Eigen objects

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: jointcal
    • Labels:
      None
    • Story Points:
      1
    • Sprint:
      AP S19-5, AP S19-6
    • Team:
      External

      Description

      Paul Price noticed that the use of unsigned (== uint32, usually) as the index type for Eigen matrices in jointcal might be a problem:

      • Eigen docs appear to want a signed integer;
      • our number of sources in some of our processing is approaching 2^31.

      There's no hard evidence that this is what's causing the segfault we're seeing when processing the (larger, still-proprietary) HSC UltraDeep, and since unsigned should be able to handle 2^32, I'm personally skeptical that this is the problem.  But it should be easy to fix, and a good thing to do regardless, especially for the first reason.

      std::ptrdiff_t is probably the right choice to use instead - that will be a signed integer as large as the pointer type (i.e. int64, usually), and hence the largest unsigned type that's actually usable as an index.

       

        Attachments

          Issue Links

            Activity

            jbosch Jim Bosch created issue -
            Parejkoj John Parejko made changes -
            Field Original Value New Value
            Status To Do [ 10001 ] In Progress [ 3 ]
            Parejkoj John Parejko made changes -
            Watchers Jim Bosch, John Swinbank [ Jim Bosch, John Swinbank ] Jim Bosch, John Swinbank, Paul Price, Yusra AlSayyad [ Jim Bosch, John Swinbank, Paul Price, Yusra AlSayyad ]
            Parejkoj John Parejko made changes -
            Reviewers Paul Price [ price ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Parejkoj John Parejko made changes -
            Sprint AP S19-5 [ 833 ]
            Priority Undefined [ 10000 ] Major [ 3 ]
            yusra Yusra AlSayyad made changes -
            Link This issue blocks DM-17743 [ DM-17743 ]
            jbosch Jim Bosch made changes -
            Attachment cholmod-longs.patch [ 37848 ]
            Parejkoj John Parejko made changes -
            Reviewers Paul Price [ price ] John Parejko [ parejkoj ]
            Parejkoj John Parejko made changes -
            Assignee John Parejko [ parejkoj ] Paul Price [ price ]
            Parejkoj John Parejko made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            swinbank John Swinbank made changes -
            Sprint AP S19-5 [ 833 ] AP S19-5, AP S19-6 [ 833, 834 ]
            swinbank John Swinbank made changes -
            Team Alert Production [ 10300 ] External [ 12117 ]
            price Paul Price made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]

              People

              Assignee:
              price Paul Price
              Reporter:
              jbosch Jim Bosch
              Reviewers:
              John Parejko
              Watchers:
              Jim Bosch, John Parejko, John Swinbank, Paul Price, Tim Jenness, Yusra AlSayyad
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: