# Create simple python script to create and insert fakes for the HiTS2015 AP processing.

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
8
• Sprint:
AP S20-4 (March)
• Team:
• Urgent?:
No

## Description

Create a simple script to insert fakes for AP using the HiTS2015 dataset. Likely this will be on an un-merged ticket of ap_verify.

This script will be fairly hacky as it will copy data around in the ap_verify_hists2015 dataset and edit the DecamMapper on the fly.

## Activity

Hide
Chris Morrison added a comment - - edited

Eric Bellm I've completed an initial run of fake injection into the full ap_verify_hits2015 dataset.

You can find the resulting Apdb here:

 /project/morriscb/src/ap_verify/fakesRepo/output/rerun/testFakes/fakesAssociation.db

The fakes that were injected are cataloged here:

 calexp: /project/morriscb/src/ap_verify/fakesRepo/calexpFakesTract0.csv coadd: /project/morriscb/src/ap_verify/fakesRepo/coaddFakesTract0.csv 

The calexp and coadd files share half of their sources, the rest are unique to the calexp or coadd injection. 180k PSF like sources were created over the tract in total. The shared objects are the last half of the calexp file and the first half of the coadd. The positions are uniformly random over the tract with the magnitudes being uniform random from 19-24 in g band magnitude. The calexp file has an additional 2% uniform scatter added to the magnitude so an object in the calexp should have a magnitude within 2% of the one in the coadd.

I haven't looking in the Apdb yet, but I am about to attempt to confirm that the sources were added to both the calexp and coadd.

Show
Hide
Chris Morrison added a comment -

Created a ap_verify/pipe script with configurable command line options to inject fakes into the the ap_verify_hits2015 or ap_verify_ci_hits2015 dataset.

To use the script: clone and setup the branches associated with this ticket. Clone and setup the ap_verify_hits2015 dataset. It is recommended to submit the job to the lsst-dev verification cluster on one node with 24 jobs requested in the script.

To use the CI dataset, one must currently edit the script. However, I could add this as an additional configurable if requested.

To be clear, this ticket will not be merged to master as it is a hacky work around to inject fakes into AP that is reliant on Gen2 butler. The real and final script will be created on another ticket once AP is fully Gen3 compatible.

Show
Chris Morrison added a comment - Created a ap_verify/pipe script with configurable command line options to inject fakes into the the ap_verify_hits2015 or ap_verify_ci_hits2015 dataset. To use the script: clone and setup the branches associated with this ticket. Clone and setup the ap_verify_hits2015 dataset. It is recommended to submit the job to the lsst-dev verification cluster on one node with 24 jobs requested in the script. To use the CI dataset, one must currently edit the script. However, I could add this as an additional configurable if requested. To be clear, this ticket will not be merged to master as it is a hacky work around to inject fakes into AP that is reliant on Gen2 butler. The real and final script will be created on another ticket once AP is fully Gen3 compatible.
Hide
Chris Morrison added a comment -

Currently with the ap_verify_hits2015 dataset, some patches have fakes inserted with NaN values. This is caused by the computeApertureFlux method of the PSF returning NaN for certain patches in the coadd. One specific example is dataId "tract=0 patch=10,9 filter=g". Fixing or investigating this is beyond the scope of this ticket.

Show
Chris Morrison added a comment - Currently with the ap_verify_hits2015 dataset, some patches have fakes inserted with NaN values. This is caused by the computeApertureFlux method of the PSF returning NaN for certain patches in the coadd. One specific example is dataId "tract=0 patch=10,9 filter=g". Fixing or investigating this is beyond the scope of this ticket.
Hide
Eric Bellm added a comment -

Thanks Chris Morrison! I created https://jira.lsstcorp.org/browse/DM-23914 for the followup Gen 3 version.

Show
Eric Bellm added a comment - Thanks Chris Morrison ! I created https://jira.lsstcorp.org/browse/DM-23914 for the followup Gen 3 version.

## People

• Assignee:
Chris Morrison
Reporter:
Chris Morrison
Reviewers:
Eric Bellm
Watchers:
Chris Morrison, Eric Bellm, John Swinbank