Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3935

Measurement plugin errors

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: pipe_tasks
    • Labels:
      None
    • Story Points:
      6
    • Sprint:
      DRP X16-1, DRP X16-2
    • Team:
      Data Release Production

      Description

      When doing measurements on coadds, several errors are thrown within the measurement plugins.

      Error in base_GaussianFlux.measure on record 283467884979: Input shape is singular
      

      Error in base_GaussianFlux.measure on record 283467883979: 
        File "src/SdssShape.cc", line 842, in static lsst::meas::base::FluxResult lsst::meas::base::SdssShapeAlgorithm::computeFixedMomentsFlux(const ImageT&, const lsst::afw::geom::ellipses::Quadrupole&, const Point2D&) [with ImageT = lsst::afw::image::MaskedImage<float, short unsigned int, float>; lsst::afw::geom::Point2D = lsst::afw::geom::Point<double, 2>]
          Error from calcmom {0}
      lsst::pex::exceptions::RuntimeError: 'Error from calcmom'
      

      The measurements were done on tiger, and the command used was:

      measureCoaddSources.py /tigress/HSC/HSC/rerun/nate/old_clip/ --output=/tigress/HSC/HSC/rerun/nate/old_clip/ -C /home/nlust/options_temp.py --id tract=0 patch=2,2 filter=HSC-I^HSC-R 
      

      The data can be found within the rerun directory specified as the input to the command. The data was created using the commands:

      assembleCoadd.py  --legacyCoadd /tigress/HSC/HSC/rerun/nate/ --output=/tigress/HSC/HSC/rerun/nate/old_clip --id tract=0 patch=2,2 filter=HSC-R --selectId visit=1208 ccd=56^64^72 --selectId visit=1206 ccd=64^65^72^73^79^80 --selectId visit=1212 ccd=64^65^72^73^79^80 --selectId visit=23704 ccd=64^65^72^73 --selectId visit=23706 ccd=56^64^72 --selectId visit=23694 ccd=64^65^72^73^79^80 --selectId visit=1204 ccd=64^65^72^73^79^80 --selectId visit=1220 ccd=63^64^71^72^78^79 --selectId visit=1218 ccd=56^57^64^65^72^73 --selectId visit=23718 ccd=64^65^72^73^79^80 --selectId visit=23692 ccd=64^65^72^73^79^80 --selectId visit=1210 ccd=63^64^71^72^78^79 --selectId visit=1216 ccd=56^57^64^65^72^73 --selectId visit=1214 ccd=64^65^72^73^79^80 --selectId visit=23716 ccd=63^64^71^72^78^79 --selectId visit=1202 ccd=64^65^72^73^79^80
      

      and

      assembleCoadd.py --legacyCoadd /tigress/HSC/HSC/rerun/nate/ --output=/tigress/HSC/HSC/rerun/nate/old_clip --id tract=0 patch=2,2 filter=HSC-I --selectId visit=19658 ccd=64^65^72^73^79^80 --selectId visit=1248 ccd=56^64^72 --selectId visit=19696 ccd=65^66^73^74^80^81 --selectId visit=19684 ccd=64^65^72^73^79^80 --selectId visit=1238 ccd=64^65^72^73^79^80 --selectId visit=19710 ccd=56^64^72 --selectId visit=19680 ccd=56^64^72 --selectId visit=1230 ccd=64^65^72^73^79^80 --selectId visit=1236 ccd=63^64^71^72^78^79 --selectId visit=19694 ccd=64^65^72^73^79^80 --selectId visit=1232 ccd=64^65^72^73^79^80 --selectId visit=19698 ccd=64^65^72^73^79^80 --selectId visit=1228 ccd=64^65^72^73^79^80 --selectId visit=1246 ccd=63^64^71^72^78^79 --selectId visit=19682 ccd=63^64^71^72^78^79 --selectId visit=19708 ccd=64^65^72^73^79^80 --selectId visit=19662 ccd=64^65^72^73 --selectId visit=1240 ccd=64^65^72^73^79^80 --selectId visit=1244 ccd=56^57^64^65^72^73 --selectId visit=1242 ccd=56^57^64^65^72^73 --selectId visit=19660 ccd=64^65^72^73^79^80 --selectId visit=19712 ccd=56^57^64^65^72^73
      

        Attachments

          Issue Links

            Activity

            Hide
            lauren Lauren MacArthur added a comment -

            Re:

            The fix supresses all the instances that were thrown without the fix.


            What I meant was: Due to this loose threshold, did you "lose" many more sources in addition to those already failing and giving the error?

            Show
            lauren Lauren MacArthur added a comment - Re: The fix supresses all the instances that were thrown without the fix. What I meant was: Due to this loose threshold, did you "lose" many more sources in addition to those already failing and giving the error?
            Hide
            vpk24 Vishal Kasliwal [X] (Inactive) added a comment -

            Lauren MacArthur could you please take another look at this review. I've created https://jira.lsstcorp.org/browse/DM-5801 to handle the intyernal shape tests and magic number logic of SdssShape. I've made sure that we don't loose a significant number of sources (8/31793 lost) by setting the epsilon in the test to 1.0e-6.

            Show
            vpk24 Vishal Kasliwal [X] (Inactive) added a comment - Lauren MacArthur could you please take another look at this review. I've created https://jira.lsstcorp.org/browse/DM-5801 to handle the intyernal shape tests and magic number logic of SdssShape. I've made sure that we don't loose a significant number of sources (8/31793 lost) by setting the epsilon in the test to 1.0e-6.
            Hide
            lauren Lauren MacArthur added a comment -

            Looks good, but could you please also update your commit message (and note it was never "ensuring that the determinant is strictly greater than zero" but rather that "the determinant is not less than zero"). Make note of the use & selection of this "magic number" and the plans to amend the situation in the commit message and in the code (I'm not sure if this requires an official @todo, or if a simple comment will do).

            Show
            lauren Lauren MacArthur added a comment - Looks good, but could you please also update your commit message (and note it was never "ensuring that the determinant is strictly greater than zero" but rather that "the determinant is not less than zero"). Make note of the use & selection of this "magic number" and the plans to amend the situation in the commit message and in the code (I'm not sure if this requires an official @todo, or if a simple comment will do).
            Hide
            vpk24 Vishal Kasliwal [X] (Inactive) added a comment -

            Merged to master....

            Show
            vpk24 Vishal Kasliwal [X] (Inactive) added a comment - Merged to master....
            Hide
            swinbank John Swinbank added a comment -

            Brief mention added to release notes.

            Show
            swinbank John Swinbank added a comment - Brief mention added to release notes.

              People

              • Assignee:
                vpk24 Vishal Kasliwal [X] (Inactive)
                Reporter:
                nlust Nate Lust
                Reviewers:
                Lauren MacArthur
                Watchers:
                Jim Bosch, John Swinbank, Lauren MacArthur, Nate Lust, Vishal Kasliwal [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel