Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-4036

Change from boost::math

    XMLWordPrintable

    Details

      Description

      Most boost::math contents (not including pi) are now available in standard C++. Please convert the code accordingly.

      In addition to the packages listed above, boost/math is used in "partition" a package I don't recognize and not a component JIRA accepts.

        Attachments

          Issue Links

            Activity

            No builds found.
            rowen Russell Owen created issue -
            rowen Russell Owen made changes -
            Field Original Value New Value
            Link This issue relates to RFC-100 [ RFC-100 ]
            Hide
            tjenness Tim Jenness added a comment - - edited

            partition is a Qserv dependency for "Spherical data partitioning and duplication utilities.". I think I'll leave it to Jacek Becla to comment on whether he wants it handled in this ticket.

            Show
            tjenness Tim Jenness added a comment - - edited partition is a Qserv dependency for "Spherical data partitioning and duplication utilities.". I think I'll leave it to Jacek Becla to comment on whether he wants it handled in this ticket.
            Hide
            jbecla Jacek Becla added a comment -

            I'd suggest to split it into several stories, e.g., qserv migration should be a dedicated story. We will handle it sometimes soon as time permits in between already scheduled stories.

            Show
            jbecla Jacek Becla added a comment - I'd suggest to split it into several stories, e.g., qserv migration should be a dedicated story. We will handle it sometimes soon as time permits in between already scheduled stories.
            tjenness Tim Jenness made changes -
            Link This issue is triggered by RFC-100 [ RFC-100 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to RFC-100 [ RFC-100 ]
            rowen Russell Owen made changes -
            Description Most or all boost::math contents are now available in standard C++. Please convert the code accordingly.

            In addition to the packages listed above, boost/math is used in a "partition" a package I don't recognize and not a component JIRA accepts.
            Most or all boost::math contents are now available in standard C++. Please convert the code accordingly.

            In addition to the packages listed above, boost/math is used in "partition" a package I don't recognize and not a component JIRA accepts.
            rowen Russell Owen made changes -
            Description Most or all boost::math contents are now available in standard C++. Please convert the code accordingly.

            In addition to the packages listed above, boost/math is used in "partition" a package I don't recognize and not a component JIRA accepts.
            Most boost::math contents (not including pi) are now available in standard C++. Please convert the code accordingly.

            In addition to the packages listed above, boost/math is used in "partition" a package I don't recognize and not a component JIRA accepts.
            swinbank John Swinbank made changes -
            Labels SciencePipelines
            jbecla Jacek Becla made changes -
            Team Alert Production [ 10300 ]
            tjenness Tim Jenness made changes -
            Link This issue relates to DM-5880 [ DM-5880 ]
            pschella Pim Schellart [X] (Inactive) made changes -
            Assignee Pim Schellart [ pschella ]
            pschella Pim Schellart [X] (Inactive) made changes -
            Sprint DRP X16-3 [ 217 ]
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            A quick check though the codebase shows limited direct usage of boost::math.
            The following are used and have replacements in C++11:

            • boost::math::atanh
            • boost::math::erfc
            • boost::math::fpclassify
            • boost::math::isfinite
            • boost::math::isinf
            • boost::math::isnan
            • boost::math::isnormal
            • boost::math::iround (yes, but in the form of std::lround)

            and the following do not.

            • boost::math::binomial_coefficient<double>
            • boost::math::constants::pi<float>
            • boost::math::cyl_bessel_j
            • boost::math::erf_inv
            • boost::math::erfc_inv
            • boost::math::gamma_q_inv
            • boost::math::tgamma_delta_ratio
            • boost::math::tgamma_ratio
            • boost::math::unchecked_factorial<double>

            Shall we convert the first group and postpone the latter until a proper replacement library has been identified?

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - A quick check though the codebase shows limited direct usage of boost::math . The following are used and have replacements in C++11: boost::math::atanh boost::math::erfc boost::math::fpclassify boost::math::isfinite boost::math::isinf boost::math::isnan boost::math::isnormal boost::math::iround (yes, but in the form of std::lround ) and the following do not. boost::math::binomial_coefficient<double> boost::math::constants::pi<float> boost::math::cyl_bessel_j boost::math::erf_inv boost::math::erfc_inv boost::math::gamma_q_inv boost::math::tgamma_delta_ratio boost::math::tgamma_ratio boost::math::unchecked_factorial<double> Shall we convert the first group and postpone the latter until a proper replacement library has been identified?
            Hide
            tjenness Tim Jenness added a comment - - edited

            doing what we can gets my vote. At least that way we end up with a smaller list of "bits of boost we do use".

            Show
            tjenness Tim Jenness added a comment - - edited doing what we can gets my vote. At least that way we end up with a smaller list of "bits of boost we do use".
            swinbank John Swinbank made changes -
            Epic Link DM-5346 [ 23141 ]
            swinbank John Swinbank made changes -
            Team Alert Production [ 10300 ] Data Release Production [ 10301 ]
            Hide
            ktl Kian-Tat Lim added a comment -

            But the proposal to only convert the first group doesn't enable us to stop building boost. It forces developers to look for a std:: function first and fall back to boost:: if not present, and furthermore, even if you consider that a good thing (which I tend to agree with), there's no good enforcement if someone misses the function in std:: and uses the boost:: version instead.

            All that said, I am OK with the proposal. It is beyond the scope of RFC-100 and so perhaps shouldn't be discussed solely on a ticket. I would also suggest that something like the lists above as well as a pointer to standard library documentation should be added to the "Using Boost" developer documentation.

            Show
            ktl Kian-Tat Lim added a comment - But the proposal to only convert the first group doesn't enable us to stop building boost. It forces developers to look for a std:: function first and fall back to boost:: if not present, and furthermore, even if you consider that a good thing (which I tend to agree with), there's no good enforcement if someone misses the function in std:: and uses the boost:: version instead. All that said, I am OK with the proposal. It is beyond the scope of RFC-100 and so perhaps shouldn't be discussed solely on a ticket. I would also suggest that something like the lists above as well as a pointer to standard library documentation should be added to the "Using Boost" developer documentation.
            Hide
            rowen Russell Owen added a comment -

            Surely we all agree that if a math function we need is in std:: then that is the version that should be used. I agree with Kian-Tat Lim that this will be hard to enforce (as long as we continue to also use boost::math) but failure to enforce should not cause any harm as long as the functions are correct in boost and std.

            In the long term we should consider alternatives to boost::math (such as trying to find a well supported 3rd party library that does the job, writing our own, or doing without). But that seems out of scope for this ticket.

            Show
            rowen Russell Owen added a comment - Surely we all agree that if a math function we need is in std:: then that is the version that should be used. I agree with Kian-Tat Lim that this will be hard to enforce (as long as we continue to also use boost::math) but failure to enforce should not cause any harm as long as the functions are correct in boost and std. In the long term we should consider alternatives to boost::math (such as trying to find a well supported 3rd party library that does the job, writing our own, or doing without). But that seems out of scope for this ticket.
            pschella Pim Schellart [X] (Inactive) made changes -
            Status To Do [ 10001 ] In Progress [ 3 ]
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Replaced all boost::math functions with std replacements. The only exception (for now) is boost::math::erfc because the inverse function is not in std and I thought this might make the code a little awkward in places (using forward from standard and inverse from boost). Easy to change if wanted though.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Replaced all boost::math functions with std replacements. The only exception (for now) is boost::math::erfc because the inverse function is not in std and I thought this might make the code a little awkward in places (using forward from standard and inverse from boost). Easy to change if wanted though.
            pschella Pim Schellart [X] (Inactive) made changes -
            Reviewers Jim Bosch, Russell Owen [ jbosch, rowen ]
            Status In Progress [ 3 ] In Review [ 10004 ]
            Hide
            rowen Russell Owen added a comment -

            Review of DM-4036

            I agree with your decision to leave boost::math::erfc alone.

            I'm glad you were able to get rid of utils/ieee.h.

            The one thing I would like you to fix (unfortunately in most of the files)
            is to regroup the imports correctly (first standard headers, then 3rd party headers,
            then finally LSST headers). Auto replacement is fine as a first step, but then
            please look at each modified source file to make sure the grouping is still correct.

            One question about the files in Jointcal: I see some of them still include boost/make_shared.hpp.
            I thought you switched from boost to std for shared_ptr in an earlier ticket?

            I'll leave it to Jim Bosch (the second reviewer) to mark this ticket as reviewed when he is ready.

            Show
            rowen Russell Owen added a comment - Review of DM-4036 I agree with your decision to leave boost::math::erfc alone. I'm glad you were able to get rid of utils/ieee.h . The one thing I would like you to fix (unfortunately in most of the files) is to regroup the imports correctly (first standard headers, then 3rd party headers, then finally LSST headers). Auto replacement is fine as a first step, but then please look at each modified source file to make sure the grouping is still correct. One question about the files in Jointcal: I see some of them still include boost/make_shared.hpp . I thought you switched from boost to std for shared_ptr in an earlier ticket? I'll leave it to Jim Bosch (the second reviewer) to mark this ticket as reviewed when he is ready.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Issues fixed, just waiting for Jim Bosch to sign off.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Issues fixed, just waiting for Jim Bosch to sign off.
            Hide
            jbosch Jim Bosch added a comment -

            Some of the header reordering that Russell Owen looks like it went too far (changing things that were fine already) and in some cases was incorrect (e.g. putting ndarray at the top).

            Also, please follow up on K-T's suggestion to modify the Boost usage section of the developer docs accordingly. I'm not actually sure what the workflow for changes to that is; if no one here chimes in, ask in the SQuaRE room on HipChat.

            Show
            jbosch Jim Bosch added a comment - Some of the header reordering that Russell Owen looks like it went too far (changing things that were fine already) and in some cases was incorrect (e.g. putting ndarray at the top). Also, please follow up on K-T's suggestion to modify the Boost usage section of the developer docs accordingly. I'm not actually sure what the workflow for changes to that is; if no one here chimes in, ask in the SQuaRE room on HipChat.
            jbosch Jim Bosch made changes -
            Status In Review [ 10004 ] Reviewed [ 10101 ]
            Hide
            ktl Kian-Tat Lim added a comment -

            Workflow is 1) decide if there is likely to be any controversy, 2) if so, file RFC and get it adopted, 3) file ticket for updating dev guide, 4) submit ticket for review by System Architect (me), 5) get approval, 6) merge.

            Show
            ktl Kian-Tat Lim added a comment - Workflow is 1) decide if there is likely to be any controversy, 2) if so, file RFC and get it adopted, 3) file ticket for updating dev guide, 4) submit ticket for review by System Architect (me), 5) get approval, 6) merge.
            Hide
            tjenness Tim Jenness added a comment -

            In this case I think RFC-100 and the discussion on this ticket count as pre-approval so I don't think an additional RFC is required specifically for the boost::math update to the developer guide.

            Show
            tjenness Tim Jenness added a comment - In this case I think RFC-100 and the discussion on this ticket count as pre-approval so I don't think an additional RFC is required specifically for the boost::math update to the developer guide.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Ok lesson learned, I will not try to reorder includes via scripts in the future. Have reset the commit and applied minimal changes manually instead. Running final Jenkins run now before merge.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Ok lesson learned, I will not try to reorder includes via scripts in the future. Have reset the commit and applied minimal changes manually instead. Running final Jenkins run now before merge.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            As for updating the "Using Boost" documentation section. I think we should make it a little more general then just dealing with changes for array, tuple, math etc. But I will file an RFC for that.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - As for updating the "Using Boost" documentation section. I think we should make it a little more general then just dealing with changes for array , tuple , math etc. But I will file an RFC for that.
            Hide
            pschella Pim Schellart [X] (Inactive) added a comment -

            Merge complete (after final Jenkins CI run). RFC-185 has been created to update DM Developer Guide as requested by Kian-Tat Lim.

            Show
            pschella Pim Schellart [X] (Inactive) added a comment - Merge complete (after final Jenkins CI run). RFC-185 has been created to update DM Developer Guide as requested by Kian-Tat Lim .
            pschella Pim Schellart [X] (Inactive) made changes -
            Resolution Done [ 10000 ]
            Status Reviewed [ 10101 ] Done [ 10002 ]
            Hide
            tjenness Tim Jenness added a comment -

            I'll just note in passing that the removal of the IEEE routines from utils have the side effect that the testExecutables test no longer tests anything. I'll put a shell script back in so we can test the executable test discovery code (although the obvious point is that we don't test that we actually ran any tests).

            Show
            tjenness Tim Jenness added a comment - I'll just note in passing that the removal of the IEEE routines from utils have the side effect that the testExecutables test no longer tests anything. I'll put a shell script back in so we can test the executable test discovery code (although the obvious point is that we don't test that we actually ran any tests).
            tjenness Tim Jenness made changes -
            Link This issue is triggering DM-6306 [ DM-6306 ]

              People

              Assignee:
              pschella Pim Schellart [X] (Inactive)
              Reporter:
              rowen Russell Owen
              Reviewers:
              Jim Bosch, Russell Owen
              Watchers:
              Jacek Becla, Jim Bosch, John Swinbank, Kian-Tat Lim, Pim Schellart [X] (Inactive), Russell Owen, Serge Monkewitz, Tim Jenness
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.