# Implement per-object Galactic Extinction correction in color analysis QA plots

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
5
• Sprint:
DRP S18-3, DRP S18-4, DRP S18-5, DRP S18-6, DRP F18-1, DRP F18-2
• Team:
Data Release Production

#### Description

Implement a per-object Galactic Extinction correction for use with the color-analysis QA plots to replace the per-field placeholder included in DM-13154.  It looks like there is code in sims_photUtils (and dependencies) to do this, so this will be an attempt to get that working with the analysis scripts.

Note that this requires the A_filter/E(B-V) extinction coefficients for the HSC filters (awaiting a response from the HSC team, the placeholder noted above is just using SDSS filter values).

#### Activity

Hide
Lauren MacArthur added a comment - - edited

The per-object Galactic Extinction correction currently requires lsst.sims.catUtils to be setup as it uses its EBVbase class (in lsst.sims.catUtils.dust.EBV) to obtain the color excess, E(B-V), at a given RA, DEC. The call is thus put here in a try/except to fall back to the per-field correction (currently only hard-coded for the five tracts of HSC RC2 + RC datasets) if the sims repo is not setup until we can access the EBVbase class from an lsst_distrib installation (to be discussed with the sims team).

I made the sims code accessible to an lsst_distrib installation on lsst-dev with the following:

 $mkdir -p ~/LSST/sims/ups_db $ export EUPS_PATH=~/LSST/sims:$EUPS_PATH $ eups distrib install sims_catUtils -t sims

[where ~ evaluates to /home/lauren/ here]
The installed sims package is version 2.8.0.

And the package is setup as follows:

 $setup -k sims_catUtils -t sims  Hsin-Fang Chiang has confirmed that she can set this repo up herself and did a test run on the PDR1 data reprocessing (results can currently, but temporarily, be seen https://lsst-web.ncsa.illinois.edu/~hchiang2/lauren_DM-13519/ – this will be redone once this ticket is closed). Thus, anyone on lsst-dev could setup the repo in my /home/lauren/LSST/sims directory to make use of it having setup an lsst_distrib installation. As a check on the extinction corrections applied I confirmed that the mean value matches that used for the "per-field" correction (currently hardwired for the 5 RC2 + old RC datasets), and I created plots of the galactic extinction applied to each object for all bands, e.g. The following example of the wPerp Principal Color fit: Note that there are "fit" and "wired" results displayed. The latter is the current "canonical" value that is hardwired and the former is a live fit to the current dataset. The wired coefficients came from the tract shown as a "happy medium" between the three RC2 tracts. These may be adjusted again based on the PDR1 reprocessing (after running the full run through colorAnalysis with the per-object GE correction applied, see DM-14590.) The full set of results using the per-object GE correction can currently be perused here: https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22/ The fit and wired Principal Color axes will be added to the figures in DM-14826. Show Lauren MacArthur added a comment - - edited The per-object Galactic Extinction correction currently requires lsst.sims.catUtils to be setup as it uses its EBVbase class (in lsst.sims.catUtils.dust.EBV) to obtain the color excess, E(B-V), at a given RA, DEC. The call is thus put here in a try/except to fall back to the per-field correction (currently only hard-coded for the five tracts of HSC RC2 + RC datasets) if the sims repo is not setup until we can access the EBVbase class from an lsst_distrib installation (to be discussed with the sims team). I made the sims code accessible to an lsst_distrib installation on lsst-dev with the following:$ mkdir -p ~/LSST/sims/ups_db $export EUPS_PATH=~/LSST/sims:$EUPS_PATH $eups distrib install sims_catUtils -t sims [where ~ evaluates to /home/lauren/ here] The installed sims package is version 2.8.0. And the package is setup as follows:$ setup -k sims_catUtils -t sims Hsin-Fang Chiang has confirmed that she can set this repo up herself and did a test run on the PDR1 data reprocessing (results can currently, but temporarily, be seen https://lsst-web.ncsa.illinois.edu/~hchiang2/lauren_DM-13519/ – this will be redone once this ticket is closed).  Thus, anyone on lsst-dev could setup the repo in my /home/lauren/LSST/sims directory to make use of it having setup an lsst_distrib  installation. As a check on the extinction corrections applied I confirmed that the mean value matches that used for the "per-field" correction (currently hardwired for the 5 RC2 + old RC datasets), and I created plots of the galactic extinction applied to each object for all bands, e.g. The following example of the wPerp Principal Color fit: Note that there are "fit" and "wired" results displayed. The latter is the current "canonical" value that is hardwired and the former is a live fit to the current dataset. The wired coefficients came from the tract shown as a "happy medium" between the three RC2 tracts. These may be adjusted again based on the PDR1 reprocessing (after running the full run through colorAnalysis with the per-object GE correction applied, see DM-14590 .) The full set of results using the per-object GE correction can currently be perused here:  https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22/ The fit and wired Principal Color axes will be added to the figures in DM-14826 .
Hide
Lauren MacArthur added a comment -

Paul, would you be able to give this a look?  I'm pinging you first as I want you to at least look at the addition of the config files in obs_subaru to check that my implementation is suitable.  Let me know if you'd prefer to pass off the pipe_analysis part of the review.

Show
Lauren MacArthur added a comment - Paul, would you be able to give this a look?  I'm pinging you first as I want you to at least look at the addition of the config files in obs_subaru to check that my implementation is suitable.  Let me know if you'd prefer to pass off the pipe_analysis part of the review.
Hide
Paul Price added a comment -

Changes in obs_subaru look fine. If you don't mind, I'll duck reviewing pipe_analysis as I'm digging out from the vacation backlog and have multiple other reviews pending.

Show
Paul Price added a comment - Changes in obs_subaru look fine. If you don't mind, I'll duck reviewing pipe_analysis as I'm digging out from the vacation backlog and have multiple other reviews pending.
Hide
Lauren MacArthur added a comment -

Sophie Reed, would you mind having a look at the pipe_analysis changes here?  Paul Price has already approved the obs_subaru changes.

Show
Lauren MacArthur added a comment - Sophie Reed , would you mind having a look at the pipe_analysis  changes here?   Paul Price has already approved the obs_subaru  changes.
Hide
Lauren MacArthur added a comment -

I've addressed all your comments to date and pushed the updated branch.  A new run of colorAnalysis.py on the RC2 tracts is going now.  I moved the previous one to https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22old2 so this run will be in https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22 when it finishes.

Let me know if you see anything else that needs fixing.

Show
Lauren MacArthur added a comment - I've addressed all your comments to date and pushed the updated branch.  A new run of colorAnalysis.py on the RC2 tracts is going now.  I moved the previous one to  https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22old2  so this run will be in  https://lsst-web.ncsa.illinois.edu/~lauren/lauren/DM-13519/w_2018_22  when it finishes.   Let me know if you see anything else that needs fixing.
Hide
Sophie Reed added a comment -

Looks good to me

Show
Sophie Reed added a comment - Looks good to me
Hide
Lauren MacArthur added a comment -

Thanks both.  Merged to master.

Show
Lauren MacArthur added a comment - Thanks both.  Merged to master.

#### People

Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Paul Price, Sophie Reed
Watchers:
John Swinbank, Lauren MacArthur, Paul Price, Sophie Reed