# Add sky coverage plots of visit summary quantities to analysis_tools

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
18
• Sprint:
DRP S22B
• Team:
Data Release Production
• Urgent?:
No

#### Description

As part of the DP0.2 V&V efforts, plots showing various "visit summary" quantities as a function of RA/Dec and focal plane coordinates were generated (see PREOPS-849). These are (small-ish) survey-level plots with a point per detector. The following are examples for the ~300 sq deg, 6 band, 19852 visit, 2804994 detector dataset that comprises the DP0.2 release (and probably represents the upper limit to which this granularity scales):

This ticket is to get these plots added to analysis_drp.

#### Activity

Hide
Lauren MacArthur added a comment - - edited

These can now make use of the ccdVisitTable datasets with the quantities added to those tables on DM-34630.

Show
Lauren MacArthur added a comment - - edited These can now make use of the ccdVisitTable datasets with the quantities added to those tables on DM-34630 .
Hide
Sophie Reed added a comment -

I added a few generic plot points in the validation team slack channel.

I think there are two many panels on each plot, you can't read the labels or see the details without scrolling around on your screen, at which point you might as well have multiple plots. Maybe they should be divided by band?

Show
Sophie Reed added a comment - I added a few generic plot points in the validation team slack channel. I think there are two many panels on each plot, you can't read the labels or see the details without scrolling around on your screen, at which point you might as well have multiple plots. Maybe they should be divided by band?
Hide
Lauren MacArthur added a comment - - edited

Here are some examples based on your (Sophie Reed) suggestions on slack (namely to use the colorblind-friendly colormap and binning...I had to factor out that functionality from the skyPlot action and put it in plotUtils to allow reuse).  I'm still including all the panels as I want to ask a wider audience as to which groupings would be best and then we can lock those into pipelines (it's all configurable, so making multiple versions with specific band/parameter subsets is easy enough).

Show
Lauren MacArthur added a comment - - edited Here are some examples based on your ( Sophie Reed ) suggestions on slack (namely to use the colorblind-friendly colormap and binning...I had to factor out that functionality from the skyPlot  action and put it in plotUtils  to allow reuse).  I'm still including all the panels as I want to ask a wider audience as to which groupings would be best and then we can lock those into pipelines (it's all configurable, so making multiple versions with specific band/parameter subsets is easy enough).
Hide
Lauren MacArthur added a comment -

I have tested this thoroughly on several different datasets (including HSC RC2, imsim/DC2, and LATISS collections) and altering config defaults using the attached notebook (note: there are a few things to edit if you want to run yourself, namely the details specifying the specific collection you want to look at and the output directory if you want to persist pngs of the plots). Various examples are attached.

I am leaving adding an entry into out regularly run pipelines for another ticket as it will require a discussion on how we want these plots to be partitioned/named. As such, I also did not want to officially register any dataset names into our repos until that discussion has occurred, so I tested running the attached pipeline coveragePlots.yaml (again varying many of the config parameters) on a local ci_hsc_gen3 repo with the following command:

 $pipetask --long-log run -d "instrument='HSC' AND skymap='discrete/ci_hsc'" -b /sdf/data/rubin/user/laurenma/LSST/ci_hsc_gen3/DATA --input HSC/runs/ci_hsc --output u/laurenma/DM-34625 -p /sdf/group/rubin/user/laurenma/tickets/DM-34625/coveragePlots.yaml#raDecCoveragePlots,focalPlaneCoveragePlots  The resulting plots look like (note there are very few visits in ci_hsc_gen3!): Show Lauren MacArthur added a comment - I have tested this thoroughly on several different datasets (including HSC RC2, imsim/DC2, and LATISS collections) and altering config defaults using the attached notebook (note: there are a few things to edit if you want to run yourself, namely the details specifying the specific collection you want to look at and the output directory if you want to persist pngs of the plots). Various examples are attached. I am leaving adding an entry into out regularly run pipelines for another ticket as it will require a discussion on how we want these plots to be partitioned/named. As such, I also did not want to officially register any dataset names into our repos until that discussion has occurred, so I tested running the attached pipeline coveragePlots.yaml (again varying many of the config parameters) on a local ci_hsc_gen3 repo with the following command:$ pipetask - - long - log run - d "instrument='HSC' AND skymap='discrete/ci_hsc'" - b / sdf / data / rubin / user / laurenma / LSST / ci_hsc_gen3 / DATA - - input HSC / runs / ci_hsc - - output u / laurenma / DM - 34625 - p / sdf / group / rubin / user / laurenma / tickets / DM - 34625 / coveragePlots.yaml #raDecCoveragePlots,focalPlaneCoveragePlots The resulting plots look like (note there are very few visits in ci_hsc_gen3 !):
Hide
Lauren MacArthur added a comment - - edited

Would you mind giving this a look when you get a chance?

Jenkins is: running green.

Show
Lauren MacArthur added a comment - - edited Would you mind giving this a look when you get a chance? Jenkins is:  running green .
Hide
Lauren MacArthur added a comment -

Thanks for the review, Clare.  I addressed all your comments in the PR and reran Jenkins. Merged to main and done!

Show
Lauren MacArthur added a comment - Thanks for the review, Clare.  I addressed all your comments in the PR and reran Jenkins . Merged to main and done!

#### People

Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Clare Saunders
Watchers:
Clare Saunders, Lauren MacArthur, Sophie Reed, Yusra AlSayyad