Details
-
Type:
Story
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: pipe_tasks
-
Labels:
-
Story Points:3
-
Epic Link:
-
Sprint:DRP S22B
-
Team:Data Release Production
-
Urgent?:No
Description
The per-collection consolidated ccdVisitTable dataset is required for the plots to be made in DM-34625. However, they are currently missing a number of the columns most useful for QA. The current column listing is:
['visitId', 'physical_filter', 'band', 'ra', 'decl', 'zenithDistance', 'zeroPoint', 'psfSigma', 'skyBg', 'skyNoise', 'detector', 'seeing', 'skyRotation', 'expMidpt', 'expTime', 'obsStart', 'darkTime', 'xSize', 'ySize', 'llcra', 'llcdec', 'ulcra', 'ulcdec', 'urcra', 'urcdec', 'lrcra', 'lrcdec'] |
To this list I would like to add the following columns which come directly from the visitSummary tables (they were added after the ccdVisitTable dataset was defined, hence they are not yet included):
['astromOffsetMean', 'astromOffsetStd', 'nPsfStar', 'psfStarDeltaE1Median', 'psfStarDeltaE2Median', 'psfStarDeltaE1Scatter', 'psfStarDeltaE2Scatter', 'psfStarDeltaSizeMedian', 'psfStarDeltaSizeScatter', 'psfStarScaledDeltaSizeScatter'] |
Attachments
Issue Links
Activity
| Status | To Do [ 10001 ] | In Progress [ 3 ] |
Would you mind giving this a look? One thing I'm not sure about is if there are any schemas that need updating for this change.
Jenkins + ci_hsc + ci_imsim is running green.
| Reviewers | Yusra AlSayyad [ yusra ] | |
| Status | In Progress [ 3 ] | In Review [ 10004 ] |
This fell behind main while awaiting review, so I rebased and ran another jenkins.
And another rebase & green Jenkins after the great merge yesterday.
Green light to add to the existing CcdVisit Table.
The CcdVisit schema need updates in:
| Status | In Review [ 10004 ] | Reviewed [ 10101 ] |
Awesome, thanks Yusra. One question: while adding the entries into the schema, I see some entries there include the fields:
mysql:datatype:
|
fits:tunit:
|
while others do not. It is not clear to me where the distinction lies (the existing entries for the CcdVisit Table do not include them, but I'm not sure if this was an oversight). Can you enlighten me as to when they are indeed required?
As per our discussion, it is not clear at present if those fields are indeed required and, if they are, they can all be updated in bulk on a subsequent ticket.
Final Jenkins is happy.
| Resolution | Done [ 10000 ] | |
| Status | Reviewed [ 10101 ] | Done [ 10002 ] |
| Story Points | 3 |
| Remote Link | This issue links to "Page (Confluence)" [ 34432 ] |
I ran this through ci_hsc_gen3 and looked at the results. The additional columns are indeed now included in the ccdVisitTable (with the correct values as compared with those in the visitSummary tables).
In case it's important, the size difference between the ccdVisitTable table created on main vs. this ticket branch is: 25K vs. 33K, so a factor of 1.32 larger.