Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10949

Add support for inherently cyclic columns to Firefly



    • Type: Improvement
    • Status: To Do
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Firefly, SUIT
    • Labels:


      Firefly should provide for column metadata that marks a column as representing an inherently cyclic variable such as an angle or a fractional phase, and support the graphical display of such data over more than one cycle of the variable without requiring the duplication of rows in the associated data table.

      Background: Yi Mei [X] and I were reviewing the behavior of the light-curve viewing application and the selection of light curve data in the tri-view. It was a requirement in the design of the interactive period-finding screen that the phase-folded light curve be displayed over two cycles (i.e., from phase 0 to phase 2). This is well-justified scientifically, as it is helpful to be able to ensure that interesting features of a light curve are not inadvertently (and scientifically meaninglessly) cut in half at the edges of a display from phase 0 to phase 1.

      When the phase-folded data table is returned to the tri-view, it is returned with a new "phase" column added, and with every row duplicated (with 1 added to the phase of each duplicate), in order to allow the unmodified tri-view to display plots over phase 0-2. However, this produces highly undesirable effects if the data are then displayed or sorted on any column other than phase, with duplicates visible both in the data table and in the image cutout pane. The workaround also has the unexpected effect that highlighting a point in the 0-1 range of phase does not simultaneously highlight the corresponding point - and row - in the 1-2 range.

      This is in most cases merely annoying (and, undoubtedly, confusing to users who do not understand the workaround we are using and its rationale). However, if the tools are now used to make a histogram, every point will be double-counted and the resulting histogram will have improper statistical properties; this could have damaging scientific consequences.

      (In a separate ticket I am recommending that we no longer return duplicates upon completion of work with the phase-folding tool.)

      As a long-term solution to this problem, I would like to suggest that Firefly be enhanced to understand the existence of columns representing cyclic variables. These will commonly have ranges 0-360, 0-2pi, 0-1, or offset variants of the above. When displaying x-y plots containing cyclic variables, if the axis range includes values outside the actual range of the variable, duplicates of the points will be shown, ideally as many times as needed to fill the display range.

      Selection of a row in a table will highlight all duplicates in any display. Selection of a point in a display, regardless of whether it is in the primary range of a cyclic variable or not, will select the primary and all duplicates, as well as the corresponding (primary-value) row in the table.

      Duplicates will not be shown in tables. Associated image displays will show duplicate images only when the image sort order is determined by a cyclic variable - i.e., they'll behave just like x-y plots.

      Duplicates will not be counted in histograms accumulated over non-cyclic axes. Histograms accumulated over cyclic axes will "really" only have bins over the primary range of the variable, with appropriate duplicate bins shown (an optional feature) if the x-axis extends beyond the primary range of the variable.

      Column filters on cyclic variables should operate only on the primary values (the ones actually in the table).

      Data downloads will not contain duplicates.

      This work is important but not time-critical.



          No builds found.
          gpdf Gregory Dubois-Felsmann created issue -
          xiuqin Xiuqin Wu [X] (Inactive) made changes -
          Field Original Value New Value
          Epic Link DM-10570 [ 32500 ]
          gpdf Gregory Dubois-Felsmann added a comment -

          https://jira.ipac.caltech.edu/browse/FIREFLY-408 created to capture this idea on the IPAC side.

          gpdf Gregory Dubois-Felsmann added a comment - https://jira.ipac.caltech.edu/browse/FIREFLY-408 created to capture this idea on the IPAC side.


            Unassigned Unassigned
            gpdf Gregory Dubois-Felsmann
            David Ciardi [X] (Inactive), Gregory Dubois-Felsmann, Trey Roby, Yi Mei [X] (Inactive)
            0 Vote for this issue
            4 Start watching this issue




                No builds found.