Details

Type: Story

Status: Done

Resolution: Done

Fix Version/s: None

Component/s: SUIT

Labels:

Story Points:10

Epic Link:

Sprint:SUIT Sprint 20176, SUIT Sprint 20177

Team:Science User Interface
Description
We need to migrate Firefly density plot (Heatmap with the data calculated on server side) to multitrace chart architecture. This includes options and table connections.
This ticket is dealing with the options for heatmap. It will not change the serverside algorithm. Currently I am planning to include the options for number of bins in X and Y direction and add an option for Plotly color scale. (If needed we can also use the discretized color scale as we did before.)
07/05/2017 TG
Per Xiuqin’s request, I am adding current server side implementation details for the binning algorithm, which is supposed to split XY space into rectangles and record how many points are falling into each bin. (xUnit and yUnit below are the dimensions of each bin.)
Input:
table,

xColumnOrExpr and yColumnOrExpr – columns or column expressions for x and y

maxPoints – maximum number of bins

xyRatio (default 1, aspect ratio of the plot – if we want bins to be more squarelike)

xMin, xMax, yMin, yMax (if NaN, use min or max value of the corresponding column/column expression)


Output:
1. “decimated" table with the following columns:

x, y, rowidx, weight, decimateKey

where

x, y – x and Y values of a “representative” point – a random point from the bin

rowidx – row index of the “representative” point in the original table

weight – total number of points in the bin

decimateKey – a string "<xIndex>:<yIndex>”, which identifies the bin

2. decimate key information, which contains xMin, xMax, xUnit, and yUnit

this information allows to calculate center of the bin from x and y values above

Algorithm:
1. Calculate min and max of x and y columns OR accept them as parameters.

2. Calculate xUnit and yUnit:

nXbins = (int)sqrt(maxPoints*xyRatio)

nYbins = (int)sqrt(maxPoints/xyRatio)


xUnit = (xMax  xMin)/nXs

yUnit = (yMax  yMin)/nYs;

// increase cell size a bit to include max values into grid

xUnit += xUnit/1000.0/nXs;

yUnit += yUnit/1000.0/nYs;


3. Create a map from bin key (decimateKey) to sample (“representative”) point.

for each row

get x and y values (xval and yval) for this row

if either x or y and NaN or the point is not within limits, continue to the next row

calculate decimateKey:

xIndex = (int)((xvalxMin)/xUnit)

yIndex = (int)((yvalyMin)/yUnit)

decimateKey = xIndex+”:”+yIndex

if decimateKey is already present in the map:

increase the number of represented rows weight for this bin by one

randomly decide if the current row should be chosen as a “representative”:

if (Math.random < 1.0/weight):

make current row representative

else

add current point to the map with weight 1


4. Create an output table from the map created in step 3

The algorithm is implemented in doDecimation method of QueryUtil.java