Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10832

Heatmap (density) plot in multi-trace chart

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: SUIT
    • Labels:
    • Story Points:
      10
    • Sprint:
      SUIT Sprint 2017-6, SUIT Sprint 2017-7
    • Team:
      Science User Interface

      Description

      We need to migrate Firefly density plot (Heatmap with the data calculated on server side) to multi-trace chart architecture. This includes options and table connections.

      This ticket is dealing with the options for heatmap. It will not change the server-side algorithm. Currently I am planning to include the options for number of bins in X and Y direction and add an option for Plotly color scale. (If needed we can also use the discretized color scale as we did before.)

      07/05/2017 TG

      Per Xiuqin’s request, I am adding current server side implementation details for the binning algorithm, which is supposed to split XY space into rectangles and record how many points are falling into each bin. (xUnit and yUnit below are the dimensions of each bin.)

      Input:

        table,
        xColumnOrExpr and yColumnOrExpr – columns or column expressions for x and y
        maxPoints – maximum number of bins
        xyRatio (default 1, aspect ratio of the plot – if we want bins to be more square-like)
        xMin, xMax, yMin, yMax (if NaN, use min or max value of the corresponding column/column expression)
      
      

      Output:

      1. “decimated" table with the following columns:
              x, y, rowidx, weight, decimateKey  
      where
              x, y – x and Y values of a “representative” point – a random point from the bin
              rowidx – row index of the “representative” point in the original table
              weight – total number of points in the bin
              decimateKey – a string "<xIndex>:<yIndex>”, which identifies the bin
      2. decimate key information, which contains xMin, xMax, xUnit, and yUnit
                  this information allows to calculate center of the bin from x and y values above
      

      Algorithm:

      1. Calculate min and max of x and y columns OR accept them as parameters.
      2. Calculate xUnit and yUnit:
          nXbins = (int)sqrt(maxPoints*xyRatio)
          nYbins = (int)sqrt(maxPoints/xyRatio)
       
          xUnit = (xMax - xMin)/nXs
          yUnit = (yMax - yMin)/nYs;
          // increase cell size a bit to include max values into grid
          xUnit += xUnit/1000.0/nXs;
          yUnit += yUnit/1000.0/nYs;
       
      3. Create a map from bin key (decimateKey) to sample (“representative”) point.
          for each row
             get x and y values (xval and yval) for this row
             if either x or y and NaN or the point is not within limits, continue to the next row
             calculate decimateKey:
                xIndex = (int)((xval-xMin)/xUnit)
                yIndex = (int)((yval-yMin)/yUnit)
                decimateKey = xIndex+”:”+yIndex
            if decimateKey is already present in the map:
                increase the number of represented rows  weight for this bin by one           
                randomly decide if the current row should be chosen as a “representative”:
                   if (Math.random < 1.0/weight):
                      make current row representative
            else 
               add current point to the map with weight 1
       
      4. Create an output table from the map created in step 3
      

      The algorithm is implemented in doDecimation method of QueryUtil.java

        Attachments

          Issue Links

            Activity

            There are no comments yet on this issue.

              People

              Assignee:
              tatianag Tatiana Goldina
              Reporter:
              tatianag Tatiana Goldina
              Reviewers:
              Loi Ly
              Watchers:
              Emmanuel Joliet, Loi Ly, Tatiana Goldina, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: