Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-3141

Reduce verbosity of astrometry

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: meas_astrom
    • Labels:
      None

      Description

      The astrometry.net solver that runs by default in meas_astrom 10.1 is very verbose. Here's an example running HSC data with an SDSS reference catalog:

      $ processCcd.py /tigress/HSC/HSC --output /tigress/pprice/lsst --id visit=904020 ccd=49 --clobber-config
      : Loading config overrride file '/home/pprice/LSST/obs/subaru/config/processCcd.py'
      WARNING: Unable to use psfex: No module named extensions.psfex.psfexPsfDeterminer
      hscAstrom is not setup; using LSST's meas_astrom instead
      Cannot import lsst.meas.multifit: disabling CModel measurements
      Cannot import lsst.meas.extensions.photometryKron: disabling Kron measurements
      Cannot enable shapeHSM ('MEAS_EXTENSIONS_SHAPEHSM_DIR'): disabling HSM shape measurements
      Cannot import lsst.meas.extensions.photometryKron: disabling Kron measurements
      Cannot enable shapeHSM ('MEAS_EXTENSIONS_SHAPEHSM_DIR'): disabling HSM shape measurements
      : Loading config overrride file '/home/pprice/LSST/obs/subaru/config/hsc/processCcd.py'
      : input=/tigress/HSC/HSC
      : calib=None
      : output=/tigress/pprice/lsst
      CameraMapper: Loading registry registry from /tigress/pprice/lsst/_parent/registry.sqlite3
      CameraMapper: Loading calibRegistry registry from /tigress/HSC/HSC/CALIB/calibRegistry.sqlite3
      processCcd: Processing {'taiObs': '2013-11-02', 'pointing': 671, 'visit': 904020, 'dateObs': '2013-11-02', 'filter': 'HSC-I', 'field': 'STRIPE82L', 'ccd': 49, 'expTime': 30.0}
      processCcd.isr: Performing ISR on sensor {'taiObs': '2013-11-02', 'pointing': 671, 'visit': 904020, 'dateObs': '2013-11-02', 'filter': 'HSC-I', 'field': 'STRIPE82L', 'ccd': 49, 'expTime': 30.0}
      processCcd.isr WARNING: Cannot write thumbnail image; hsc.fitsthumb could not be imported.
      afw.image.MaskedImage WARNING: Expected extension type not found: IMAGE
      processCcd.isr: Applying linearity corrections to Ccd 49
      processCcd.isr.crosstalk: Applying crosstalk correction
      afw.image.MaskedImage WARNING: Expected extension type not found: IMAGE
      : Empty WCS extension, using FITS header
      processCcd.isr: Set 0 BAD pixels to 647.04
      processCcd.isr WARNING: There were 6192 unmasked NaNs
      processCcd.isr WARNING: Cannot write thumbnail image; hsc.fitsthumb could not be imported.
      processCcd.isr: Flattened sky level: 647.130493 +/- 12.733898
      processCcd.isr: Measuring sky levels in 8x16 grids: 648.106765
      processCcd.isr: Sky flatness in 8x16 grids - pp: 0.024087 rms: 0.006057
      processCcd.calibrate: installInitialPsf fwhm=5.88235294312 pixels; size=15 pixels
      processCcd.calibrate.repair: Identified 80 cosmic rays.
      processCcd.calibrate.detection: Detected 303 positive sources to 5 sigma.
      processCcd.calibrate.detection: Resubtracting the background after object detection
      processCcd.calibrate.initialMeasurement: Measuring 303 sources (303 parents, 0 children) 
      processCcd.calibrate.astrometry: Applying distortion correction
      processCcd.calibrate.astrometry: Solving astrometry
      LoadReferenceObjects: read index files
      processCcd.calibrate.astrometry.solver: Number of selected sources for astrometry : 258
      processCcd.calibrate.astrometry.solver: Got astrometric solution from Astrometry.net
      LoadReferenceObjects: getting reference objects using center (1023.5, 2084.5) pix = Fk5Coord(320.3431396, 0.5002365, 2000.00) sky and radius 0.00194896 rad
      LoadReferenceObjects: search for objects at Fk5Coord(320.3431396, 0.5002365, 2000.00) with radius 0.111667372351 deg
      LoadReferenceObjects: found 495 objects
      LoadReferenceObjects: trimmed 257 out-of-bbox objects, leaving 238
      processCcd.calibrate.astrometry.solver: Fit WCS: use iter 2 because it had less linear scatter than the next iter: 0.307471 vs. 0.320229 pixels
      processCcd.calibrate.astrometry: 186 astrometric matches
      processCcd.calibrate.astrometry: Refitting WCS
      processCcd.calibrate.astrometry: Astrometric scatter: 0.047945 arcsec (with non-linear terms, 174 matches, 12 rejected)
      processCcd.calibrate.measurePsf: Measuring PSF
      /tigress/HSC/LSST/stack10_1/Linux64/anaconda/2.1.0-4-g35ca374/lib/python2.7/site-packages/numpy/core/_methods.py:59: RuntimeWarning: Mean of empty slice.
        warnings.warn("Mean of empty slice.", RuntimeWarning)
      /tigress/HSC/LSST/stack10_1/Linux64/anaconda/2.1.0-4-g35ca374/lib/python2.7/site-packages/numpy/core/_methods.py:71: RuntimeWarning: invalid value encountered in double_scalars
        ret = ret.dtype.type(ret / rcount)
      /home/pprice/LSST/meas/algorithms/python/lsst/meas/algorithms/objectSizeStarSelector.py:143: RuntimeWarning: invalid value encountered in less
        update = dist < minDist
      processCcd.calibrate.measurePsf: PSF star selector found 163 candidates
      processCcd.calibrate.measurePsf: PSF determination using 114/163 stars.
      processCcd.calibrate.repair: Identified 92 cosmic rays.
      processCcd.calibrate: Fit and subtracted background
      processCcd.calibrate.measurement: Measuring 303 sources (303 parents, 0 children) 
      processCcd.calibrate.astrometry: Applying distortion correction
      processCcd.calibrate.astrometry: Solving astrometry
      processCcd.calibrate.astrometry.solver: Number of selected sources for astrometry : 258
      Solver:
        Arcsec per pix range: 0.153025, 0.18516
        Image size: 2054 x 4186
        Quad size range: 205.4, 4662.78
        Objs: 0, 50
        Parity: 0, normal
        Use_radec? yes, (320.343, 0.500178), radius 1 deg
        Verify_pix: 1
        Code tol: 0.01
        Dist from quad bonus: yes
        Distractor ratio: 0.25
        Log tune-up threshold: inf
        Log bail threshold: -230.259
        Log stoplooking threshold: inf
        Maxquads 0
        Maxmatches 0
        Set CRPIX? no
        Tweak? no
        Indexes: 3
          /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_0.fits
          /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_1.fits
          /tigress/HSC/astrometry_net_data/sdss-dr9-fink-v5b/sdss-dr9-fink-v5b_and_263_2.fits
        Field: 258 stars
      Quad scale range: [641.674, 2208.56] pixels
      object 1 of 50: 0 quads tried, 0 matched.
      object 2 of 50: 0 quads tried, 0 matched.
      object 3 of 50: 0 quads tried, 0 matched.
      object 4 of 50: 0 quads tried, 0 matched.
      object 5 of 50: 0 quads tried, 0 matched.
      object 6 of 50: 0 quads tried, 0 matched.
      Got a new best match: logodds 787.099.
        log-odds ratio 787.099 (inf), 178 match, 1 conflict, 75 distractors, 220 index.
        RA,Dec = (320.343,0.500213), pixel scale 0.167612 arcsec/pix.
        Hit/miss:   Hit/miss: ++-+++++-++++++++++++--++-+--+++++-+-+++++++++-+++++++-+++++-+++++++++++++-++++++-++++++-+++++++-+++
      Pixel scale: 0.167612 arcsec/pix.
      Parity: pos.
      processCcd.calibrate.astrometry.solver: Got astrometric solution from Astrometry.net
      LoadReferenceObjects: getting reference objects using center (1023.5, 2084.5) pix = Fk5Coord(320.3431396, 0.5002365, 2000.00) sky and radius 0.00194896 rad
      LoadReferenceObjects: search for objects at Fk5Coord(320.3431396, 0.5002365, 2000.00) with radius 0.111667328272 deg
      LoadReferenceObjects: found 495 objects
      LoadReferenceObjects: trimmed 257 out-of-bbox objects, leaving 238
      processCcd.calibrate.astrometry.solver: Fit WCS: use iter 2 because it had less linear scatter than the next iter: 0.306732 vs. 0.320115 pixels
      processCcd.calibrate.astrometry: 186 astrometric matches
      processCcd.calibrate.astrometry: Refitting WCS
      processCcd.calibrate.astrometry: Astrometric scatter: 0.048271 arcsec (with non-linear terms, 174 matches, 12 rejected)
      processCcd.calibrate.photocal: Not applying color terms because config.applyColorTerms is False
      processCcd.calibrate.photocal: Magnitude zero point: 30.685281 +/- 0.058711 from 173 stars
      processCcd.calibrate: Photometric zero-point: 30.685281
      processCcd.detection: Detected 1194 positive sources to 5 sigma.
      processCcd.detection: Resubtracting the background after object detection
      processCcd.deblend: Deblending 1194 sources
      processCcd.deblend: Deblended: of 1194 sources, 143 were deblended, creating 358 children, total 1552 sources
      processCcd.measurement: Measuring 1552 sources (1194 parents, 358 children) 
      processCcd WARNING: Persisting background models
      processCcd: Matching icSource and Source catalogs to propagate flags.
      processCcd: Matching src to reference catalogue
      LoadReferenceObjects: getting reference objects using center (1023.5, 2087.5) pix = Fk5Coord(320.3429016, 0.5001781, 2000.00) sky and radius 0.00195667 rad
      LoadReferenceObjects: search for objects at Fk5Coord(320.3429016, 0.5001781, 2000.00) with radius 0.112109149864 deg
      LoadReferenceObjects: found 499 objects
      LoadReferenceObjects: trimmed 261 out-of-bbox objects, leaving 238
      processCcd.calibrate.astrometry.solver: Fit WCS: use iter 1 because it had less linear scatter than the next iter: 0.300624 vs. 0.300652 pixels
      

      The verbosity of the astrometry module is out of proportion with the rest of the modules, which makes it difficult to follow the processing.

      This is a pull request for fixes I have made.

        Attachments

          Container Issues

            Activity

              People

              • Assignee:
                price Paul Price
                Reporter:
                price Paul Price
                Reviewers:
                Russell Owen
                Watchers:
                Dominique Boutigny, Lauren MacArthur, Paul Price, Russell Owen, Simon Krughoff
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Summary Panel