# Implement RFC-534: Update naming of base_Blendedness fields

XMLWordPrintable

## Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
3
• Sprint:
DRP F18-5
• Team:
Data Release Production

## Description

The implementation of this RFC includes:

1. strip "_instFlux" from the base_Blendedness_raw_instFlux and base_Blendedness_abs_instFlux names
2. move "_instFlux" to the end of the name in the fields with actual flux units, thus
3. base_Blendedness_raw_instFlux_child --> base_Blendedness_raw_child_instFlux
4. base_Blendedness_raw_instFlux_parent --> base_Blendedness_raw_parent_instFlux
5. base_Blendedness_abs_instFlux_child --> base_Blendedness_abs_child_instFlux
6. base_Blendedness_abs_instFlux_parent --> base_Blendedness_abs_parent_instFlux
7. an update to the doc strings to make a clear distinction/description between "raw" and "abs"

## Activity

Hide
Lauren MacArthur added a comment -

As per discussion on DM-16068, this will also include the following change for the field:

 name="deblend_psfFlux", doc="If deblended-as-psf, the PSF flux" 

deblend_psfFlux --> deblend_psf_instFlux

Show
Lauren MacArthur added a comment - As per discussion on DM-16068 , this will also include the following change for the field: name = "deblend_psfFlux" , doc = "If deblended-as-psf, the PSF flux" deblend_psfFlux --> deblend_psf_instFlux
Hide
Lauren MacArthur added a comment - - edited

I'm thinking I should also change these:

 name="deblend_psfCenter_x", doc="If deblended-as-psf, the PSF centroid", units="pixel" name="deblend_psfCenter_y", doc="If deblended-as-psf, the PSF centroid", units="pixel" 

to

 name="deblend_PsfCentroid_x" name="deblend_PsfCentroid_y" 

to match, e.g.

 name="base_SdssCentroid_x"

and that the flux name above should actually be

 name="deblend_PsfFlux_instFlux"

to match, e.g.

 name="base_PsfFlux_instFlux"

Jim Bosch do you agree?

Show
Lauren MacArthur added a comment - - edited I'm thinking I should also change these: name = "deblend_psfCenter_x" , doc = "If deblended-as-psf, the PSF centroid" , units = "pixel" name = "deblend_psfCenter_y" , doc = "If deblended-as-psf, the PSF centroid" , units = "pixel" to name = "deblend_PsfCentroid_x" name = "deblend_PsfCentroid_y" to match, e.g. name = "base_SdssCentroid_x" and that the flux name above should actually be name = "deblend_PsfFlux_instFlux" to match, e.g. name = "base_PsfFlux_instFlux" Jim Bosch do you agree?
Hide
Jim Bosch added a comment -

The convention we use for measurement plugin fields isn't as clearly applicable here, as the deblender isn't a measurement plugin.  In particular, while plugin names are supposed to be distinct and achieve this by embedding the class name in the field names (because they can all be run together at once), different deblenders are mutually exclusive in the same catalog (at least the way things are structured now), so it's more desirable that their fields have the same names.  I think it's probably best to minimize changes to those except when necessary to make sure:

• instrumental fluxes end in _instFlux
• centroids end in _x, _y
• shapes end in _xx, _yy, _xy

In other words, as long as the last bit of the suffix is ok, I think the names can mostly stay as-is.

Show
Jim Bosch added a comment - The convention we use for measurement plugin fields isn't as clearly applicable here, as the deblender isn't a measurement plugin.  In particular, while plugin names are supposed to be distinct and achieve this by embedding the class name in the field names (because they can all be run together at once), different deblenders are mutually exclusive in the same catalog (at least the way things are structured now), so it's more desirable that their fields have the same names.  I think it's probably best to minimize changes to those except when necessary to make sure: instrumental fluxes end in _instFlux centroids end in _x, _y shapes end in _xx, _yy, _xy In other words, as long as the last bit of the suffix is ok, I think the names can mostly stay as-is.
Hide
Lauren MacArthur added a comment -

Awesome, thanks...I will stick to what was on tap before my previous comment!

Show
Lauren MacArthur added a comment - Awesome, thanks...I will stick to what was on tap before my previous comment!
Hide
Lauren MacArthur added a comment - - edited

Jim, could you give this a quick look to make sure I've made the changes as proposed. I ran multiBandDriver.py (pointing at the outputs from the w_2018_41 processing at /datasets/hsc/repo/rerun/RC/w_2018_41/DM-16011/ on lsst-dev as input: output directory is datasets/hsc/repo/rerun/private/lauren/DM-16070multi/. The new fields in the deepCoadd_meas schema look like this:

  (Field['D'](name="base_Blendedness_raw", doc="Measure of how much instFlux is affected by neighbors: (1 - child_instFlux/parent_instFlux). Operates on the "raw" pixel values."), Key(offset=120, nElements=1)),  (Field['D'](name="base_Blendedness_raw_child_instFlux", doc="instFlux of the child, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="count"), Key(offset=128, nElements=1)),  (Field['D'](name="base_Blendedness_raw_parent_instFlux", doc="instFlux of the parent, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="count"), Key(offset=136, nElements=1)),  (Field['D'](name="base_Blendedness_abs", doc="Measure of how much instFlux is affected by neighbors: (1 - child_instFlux/parent_instFlux). Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details."), Key(offset=144, nElements=1)),  (Field['D'](name="base_Blendedness_abs_child_instFlux", doc="instFlux of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="count"), Key(offset=152, nElements=1)),  (Field['D'](name="base_Blendedness_abs_parent_instFlux", doc="instFlux of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="count"), Key(offset=160, nElements=1)),  (Field['D'](name="base_Blendedness_raw_child_xx", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=168, nElements=1)),  (Field['D'](name="base_Blendedness_raw_child_yy", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=176, nElements=1)),  (Field['D'](name="base_Blendedness_raw_child_xy", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=184, nElements=1)),  (Field['D'](name="base_Blendedness_raw_parent_xx", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=192, nElements=1)),  (Field['D'](name="base_Blendedness_raw_parent_yy", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=200, nElements=1)),  (Field['D'](name="base_Blendedness_raw_parent_xy", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the "raw" pixel values.", units="pixel^2"), Key(offset=208, nElements=1)),  (Field['D'](name="base_Blendedness_abs_child_xx", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=216, nElements=1)),  (Field['D'](name="base_Blendedness_abs_child_yy", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=224, nElements=1)),  (Field['D'](name="base_Blendedness_abs_child_xy", doc="Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=232, nElements=1)),  (Field['D'](name="base_Blendedness_abs_parent_xx", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=240, nElements=1)),  (Field['D'](name="base_Blendedness_abs_parent_yy", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=248, nElements=1)),  (Field['D'](name="base_Blendedness_abs_parent_xy", doc="Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a "de-noised" value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details.", units="pixel^2"), Key(offset=256, nElements=1)) 

I ran a successful lsst_distrib lsst_ci ci_hsc Jenkins over the weekend, but made a few updates today (and added COPYRIGHT & LICENSE files), so another one is running (it has PASSED).

I will also write a brief community post outlining the changes once this gets merged.

Show
Lauren MacArthur added a comment - - edited Jim, could you give this a quick look to make sure I've made the changes as proposed. I ran multiBandDriver.py (pointing at the outputs from the w_2018_41 processing at /datasets/hsc/repo/rerun/RC/w_2018_41/ DM-16011 / on lsst-dev as input: output directory is datasets/hsc/repo/rerun/private/lauren/ DM-16070 multi/. The new fields in the deepCoadd_meas schema look like this: (Field[ 'D' ](name = "base_Blendedness_raw" , doc = "Measure of how much instFlux is affected by neighbors: (1 - child_instFlux/parent_instFlux). Operates on the " raw " pixel values." ), Key<D>(offset = 120 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_child_instFlux" , doc = "instFlux of the child, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "count" ), Key<D>(offset = 128 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_parent_instFlux" , doc = "instFlux of the parent, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "count" ), Key<D>(offset = 136 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs" , doc = "Measure of how much instFlux is affected by neighbors: (1 - child_instFlux/parent_instFlux). Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." ), Key<D>(offset = 144 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_child_instFlux" , doc = "instFlux of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "count" ), Key<D>(offset = 152 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_parent_instFlux" , doc = "instFlux of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "count" ), Key<D>(offset = 160 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_child_xx" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 168 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_child_yy" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 176 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_child_xy" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 184 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_parent_xx" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 192 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_parent_yy" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 200 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_raw_parent_xy" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the " raw " pixel values." , units = "pixel^2" ), Key<D>(offset = 208 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_child_xx" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 216 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_child_yy" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 224 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_child_xy" , doc = "Shape of the child, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 232 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_parent_xx" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 240 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_parent_yy" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 248 , nElements = 1 )), (Field[ 'D' ](name = "base_Blendedness_abs_parent_xy" , doc = "Shape of the parent, measured with a Gaussian weight matched to the child. Operates on the absolute value of the pixels to try to obtain a " de - noised " value. See section 4.9.11 of Bosch et al. 2018, PASJ, 70, S5 for details." , units = "pixel^2" ), Key<D>(offset = 256 , nElements = 1 )) I ran a successful lsst_distrib lsst_ci ci_hsc Jenkins over the weekend, but made a few updates today (and added COPYRIGHT & LICENSE files), so another one is running (it has PASSED ). PRs: https://github.com/lsst/meas_base/pull/132 https://github.com/lsst/meas_deblender/pull/62 I will also write a brief community post outlining the changes once this gets merged.
Hide
Jim Bosch added a comment -

Made a few suggestions about docstrings on the PRs.  Feel free to merge after you think you've addressed them; no need to send it back to me.

Show
Jim Bosch added a comment - Made a few suggestions about docstrings on the PRs.  Feel free to merge after you think you've addressed them; no need to send it back to me.
Hide
Lauren MacArthur added a comment - - edited

Thanks for getting to this so quickly.  I made all your suggested changes and am rerunning Jenkins just to be safe.

Show
Lauren MacArthur added a comment - - edited Thanks for getting to this so quickly.  I made all your suggested changes and am rerunning Jenkins just to be safe.

## People

• Assignee:
Lauren MacArthur
Reporter:
Lauren MacArthur
Reviewers:
Jim Bosch
Watchers:
Jim Bosch, John Parejko, Lauren MacArthur, Yusra AlSayyad