Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-9393

make jointcal's results not dependent on the order of the input data

    XMLWordPrintable

    Details

      Description

      While finalizing DM-8552, I discovered that jointcal's results depend slightly on the order of the input data. I fixed the fact that the common tangent plane was only being set to the first input image, but some of the differences persisted. I hacked in sorting of the dataRefs when the tract calculation happens in jointcal/dataIds.py, which makes the py2/py3 results the same (how this was discovered), but it would be good to understand why this is occurring in the first place.

      As an example to test this, if you reverse the input dataRefs list (e.g. jointcal.py:255 becomes for ref in reversed(dataRefs):), you change the number of selected fittedStars by one in JointcalTestLSSTSim.testJointcalTask_10_visits. Here's the wayward star:

      x -0.0343595 y 0.0550873 flux 4580.45  mcount 1 removed for no refstar
      x -0.0343595 y 0.0550873 flux 4580.45  mcount 1 too few measurements: 1
      

        Attachments

          Issue Links

            Activity

            Hide
            astier Pierre Astier added a comment -

            The results may depend on the order of the input image because the image catalogs are incrementally aggregated into
            a set of FittedStars. The effect should be marginal though, especially if the input WCS's are precise.
            I would naively expect that marginal matches will go away later as outliers.

            Show
            astier Pierre Astier added a comment - The results may depend on the order of the input image because the image catalogs are incrementally aggregated into a set of FittedStars. The effect should be marginal though, especially if the input WCS's are precise. I would naively expect that marginal matches will go away later as outliers.
            Hide
            Parejkoj John Parejko added a comment -

            Note: see reference in dataIds.py referencing this ticket for some sorted() calls that can be removed once this is figured out.

            Show
            Parejkoj John Parejko added a comment - Note: see reference in dataIds.py referencing this ticket for some sorted() calls that can be removed once this is figured out.
            Hide
            swinbank John Swinbank added a comment -

            Reviewed for DM-CCB, 2020-08-03.

            Show
            swinbank John Swinbank added a comment - Reviewed for DM-CCB, 2020-08-03.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              Parejkoj John Parejko
              Watchers:
              Dominique Boutigny, Eric Bellm, John Parejko, John Swinbank, Pierre Astier, Simon Krughoff
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.