Fix Version/s: None
Sprint:SUIT Sprint 2018-09, SUIT Sprint 2018-10, SUIT Sprint 2018-11, SUIT Sprint 2018-12, SUIT Sprint 2019-01, SUIT Sprint 2019-02, SUIT Sprint 2019-03, SUIT Sprint 2019-04
Team:Science User Interface
This ticket is to gather information for numerical data display for LSST data in SUIT.
Current Firefly displays 6 digits after decimal point by default. This may not be best for some data. We need to have a plan either specify the precision for each column in table data, or have a guideline for precision for different type of data, i.e ra, dec, magnitude, flux, error, ...
- relates to
DM-14743 Firefly table-download bug - corrupted numbers in exponential notation
DM-15274 file upload bug: the precision of numbers changed
DM-18489 Empty float from VOTable shows as "0.0000000000e+00" in the table
- links to
We discussed this ticket a bit in the FireflyCCB meeting today. We are going to talk about this in the FireflyCCB-D advisory group now, possibly under a new ticket in IPAC Jira (which we'll post here).
When deciding how to be user friendly, we should understand, that whatever algorithm we choose to limit the precision of the displayed numbers, there will be a scenario (a set of numbers), when it fails. Hence if we are limiting the precision, we should allow to change it. And we should allow users to see the original data from the data provider. Hence, preserving the original precision in the absence of other information is the first step toward being user friendly.
We should avoid displaying wrong data or misrepresenting precision.
Loi has suggested using Java’s toString(). There is no loss of data and no trailing zeros. The disadvantage is that the formatting of a column might be inconsistent: since the trailing zeros are omitted, the numbers in a column might appear with the different number of decimal places. Also, choosing scientific or decimal format depends on the data value, see https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#toString(double)
Using Java’s toString() works very well when the input data came as strings. With binary data (like binary FITS table), we’d display the original number. For example, if 0.1 is represented by 0.0999999999999659, we’d display 0.0999999999999659. This is not user friendly, but as long as we are not displaying wrong data — and we are displaying exactly what we have received — the rounding issues can be addressed later by allowing users to adjust the format.
Gregory Dubois-Felsmann could you post here the CCB ticket. I can't find it. Thanks.
DM-15826 recognizes the VOTable precision for values and Firefly will respect that attribute. Specifically the following was in that ticket:
Note: values of attributes precision and width are concatenated like 'wwEn' or 'wwFn' and stored as 'precision' in DataType, where 'ww' stands for the value from attribute width and 'En' or 'Fn' stands for value from attribute precision. If the original precision value contains numeric digit only like '2', it is converted to be 'F2'.
Currently Firefly uses %9g for FITS tables, guessing with minimum of 8 decimal digits - for VO Tables. If the first value is in scientific format with 1 digit precision, all values will be formatted like this.
To Illustrate the problem, consider the following result table:
Displayed in Firefly, the table will look like this:
I see at least 3 issues here: