We discussed this ticket a bit in the FireflyCCB meeting today.  We are going to talk about this in the FireflyCCB-D advisory group now, possibly under a new ticket in IPAC Jira (which we'll post here).

When deciding how to be user friendly, we should understand, that whatever algorithm we choose to limit the precision of the displayed numbers, there will be a scenario (a set of numbers), when it fails. Hence if we are limiting the precision, we should allow to change it. And we should allow users to see the original data from the data provider. Hence, preserving the original precision in the absence of other information is the first step toward being user friendly.

We should avoid displaying wrong data or misrepresenting precision.

Loi has suggested using Java’s toString(). There is no loss of data and no trailing zeros. The disadvantage is that the formatting of a column might be inconsistent: since the trailing zeros are omitted, the numbers in a column might appear with the different number of decimal places. Also, choosing scientific or decimal format depends on the data value, see https://docs.oracle.com/javase/7/docs/api/java/lang/Double.html#toString(double)

Using Java’s toString() works very well when the input data came as strings. With binary data (like binary FITS table), we’d display the original number. For example, if 0.1 is represented by 0.0999999999999659, we’d display 0.0999999999999659. This is not user friendly, but as long as we are not displaying wrong data — and we are displaying exactly what we have received — the rounding issues can be addressed later by allowing users to adjust the format.

Gregory Dubois-Felsmann could you post here the CCB ticket. I can't find it. Thanks.

DM-15826 recognizes the VOTable precision for values and Firefly will respect that attribute. Specifically the following was in that ticket:

Note: values of attributes precision and width are concatenated like 'wwEn' or 'wwFn'  and stored as 'precision' in DataType, where 'ww' stands for the value from attribute width and  'En' or 'Fn' stands for value from  attribute precision. If the original precision value contains numeric digit only like '2', it is converted to be 'F2'.