Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-18510

Basic ObsCore support

    XMLWordPrintable

    Details

    • Type: Story
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: Firefly
    • Labels:
    • Story Points:
      30
    • Epic Link:
    • Sprint:
      SUIT Sprint 2019-03, SUIT Sprint 2019-04
    • Team:
      Science User Interface

      Description

      As part of the developing strategy for designing clearer interfaces into the Science Platform:

      Firefly should recognize tabular data in ObsCore format and expose appropriate behavior when it is received in any form carrying sufficient metadata for it to be recognized.

      The result should not be strongly dependent on whether the ObsCore data are received via a "manual" table upload via the Firefly API, the opening of a file (e.g., from a workspace) in this format, or via the normal Firefly search processor API.

      This ticket covers only recognizing the table and providing the behavior, not the provision of any search screens or other interfaces that assist the user in finding/retrieving ObsCore-formatted data. However, it is worth noting here that implementing this ticket is a key predecessor to providing good support in Firefly for SIAv2 and ObsTAP searches, in particular.

      Priority 1: the most essential parts of this are:

      1. Recognizing the "s_ra" and "s_dec" columns (UCDs "pos.eq.ra" and "pos.eq.dec") and displaying all of the corresponding points, if not nulled, in a context image (especially a HiPS image). A "blank" context image should be supported eventually, but this is not required at first.
      2. Recognizing the "s_region" column in an ObsCore data table and successfully displaying it, if not nulled, as a geometrical region in a context image (especially a HiPS image). It should be API-selectable whether all regions are always displayed, with the highlighted row in the table having its region highlighted in the context image, or whether only the highlighted row has its region displayed at all.
      3. Recognizing the "access_url" column, displaying it as a link, and enabling the link to be used for a download of the associated data file.
      4. Recognizing "dataproduct_type=image" rows and loading and displaying the associated images if the "access_format" value suggests that Firefly should understand the image data format (FITS, HiPS, or a browser-displayable image such as JPEG, PNG, SVG).
      5. There must be API support for supplying the context image to be used. (It may be useful to design the system so that a context image could be inferred from data in the table itself, e.g., the "facility_name", "instrument_name", and "obs_collection". That is not a Priority 1 requirement, though.)
      6. This ticket does not mandate providing any search-form support for SIAv2 or ObsTAP. However, if a valid URL for a method=GET synchronous search on such a service, returning valid ObsCore data, is provided to Firefly in the existing "Upload table from URL" screen it must be recognized as ObsCore and displayed appropriately. This is a key feature for testing as well as for demonstrations of the capability.

       

      This ticket captures only the "priority one" items from DM-15783.

        Attachments

          Issue Links

          is child task of

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. DM-15783 Recognize and exploit ObsCore-structured data in Firefly

          • To Do
          links to

          Web Link IPAC Firefly-42

          Web Link IPAC Firefly-43

          Web Link IPAC Firefly-44

          Web Link pull request for FIREFLY-42

          Web Link pull request for FIREFLY-44

          Web Link Pull request fr FIREFLY-43

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            gpdf Gregory Dubois-Felsmann added a comment - - edited

            Some clarifications based on today's group meeting discussion:

            1. The observation centers (normally from the s_ra and s_dec columns) should be displayed in a separate drawing layer from the observation regions (normally from the s_region column), with separate checkboxes in the layer dialog. This means there are two layers deriving from the same table.
            2. The layer for the observation centers should behave exactly like a normal catalog layer.
            3. The layer dialog checkbox for the "s_region" layer controls whether regions are displayed at all or not. Radio buttons in the regions layer controller "highlighted", "selected", and "all" determine which regions are drawn. When "selected" is chosen, the highlighted region must always be drawn, even if it is not selected.
            4. I'd like the names of the columns being used for the ObsCore overlays to be displayed in the layer dialog.
            5. The observation-regions layer must have click-to-highlight behavior: clicking on the boundary of a region must highlight it and the corresponding table row. (I.e., the "can highlight" property should be true.)
            6. The observation-regions layer should not (for now) interact with the area-selection tools (rectangle and ellipse). (I.e., the "can select" property should be false.) That is, for an ObsCore table, the area-selection tools will act only based on the points in the observation-centers layer (i.e., on s_ra and s_dec). This means that if the observation-centers layer's controller checkbox is off, area-selection tools will not act on that ObsCore table at all. (We may at a later date return to this question.)
            7. Nevertheless, note that if an area selection is performed on an ObsCore table, acting on the observation centers, the result of the selection will be visible in the corresponding regions changing to the "selected" color.
            8. If an unsupported or invalid s_region value is found in an ObsCore table, this status should be shown to the user by displaying a "red exclamation mark circle" icon in the observation-regions layer in the layer dialog. Clicking on that icon should bring up a further dialog with error text. The error text should state the number of rows with unsupported or invalid region values, and show as an example the text of one of the bad values. Please ensure that the entire error text is copyable! At this time we are not requiring that the error UI distinguish between unsupported and invalid values (since there is no firm standard for the values in the first place).
            9. If it is not difficult, please include the standard Firefly ?-in-circle help icon in the error dialog; we will use this to link to an onlinehelp target that explains what s_region syntax we currently support.
            10. It is only necessary to implement "POLYGON" and "CIRCLE" support in order to close this initial ticket. "BOX" is controversial in the IVOA at the moment and I would suggest not addressing this yet. "POSITION" seems unlikely to show up in an s_region. We'll make additional tickets for any future extensions to the set of region texts to be supported. This situation may be clearer after the May meeting.

            As a separate ticket I will ask that we develop a system for displaying a "summary error status" badge on the toolbar button for the layer dialog. I'll make this an IPAC FIREFLY-* ticket.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - - edited Some clarifications based on today's group meeting discussion: The observation centers (normally from the s_ra and s_dec columns) should be displayed in a separate drawing layer from the observation regions (normally from the s_region column), with separate checkboxes in the layer dialog. This means there are two layers deriving from the same table. The layer for the observation centers should behave exactly like a normal catalog layer. The layer dialog checkbox for the "s_region" layer controls whether regions are displayed at all or not. Radio buttons in the regions layer controller "highlighted", "selected", and "all" determine which regions are drawn. When "selected" is chosen, the highlighted region must always be drawn, even if it is not selected. I'd like the names of the columns being used for the ObsCore overlays to be displayed in the layer dialog. The observation-regions layer must have click-to-highlight behavior: clicking on the boundary of a region must highlight it and the corresponding table row. (I.e., the "can highlight" property should be true.) The observation-regions layer should not (for now) interact with the area-selection tools (rectangle and ellipse). (I.e., the "can select" property should be false.) That is, for an ObsCore table, the area-selection tools will act only based on the points in the observation-centers layer (i.e., on s_ra and s_dec ). This means that if the observation-centers layer's controller checkbox is off, area-selection tools will not act on that ObsCore table at all . (We may at a later date return to this question.) Nevertheless, note that if an area selection is performed on an ObsCore table, acting on the observation centers, the result of the selection will be visible in the corresponding regions changing to the "selected" color. If an unsupported or invalid s_region value is found in an ObsCore table, this status should be shown to the user by displaying a "red exclamation mark circle" icon in the observation-regions layer in the layer dialog. Clicking on that icon should bring up a further dialog with error text. The error text should state the number of rows with unsupported or invalid region values, and show as an example the text of one of the bad values. Please ensure that the entire error text is copyable! At this time we are not requiring that the error UI distinguish between unsupported and invalid values (since there is no firm standard for the values in the first place). If it is not difficult , please include the standard Firefly ?-in-circle help icon in the error dialog; we will use this to link to an onlinehelp target that explains what s_region syntax we currently support. It is only necessary to implement "POLYGON" and "CIRCLE" support in order to close this initial ticket. "BOX" is controversial in the IVOA at the moment and I would suggest not addressing this yet. "POSITION" seems unlikely to show up in an s_region . We'll make additional tickets for any future extensions to the set of region texts to be supported. This situation may be clearer after the May meeting. As a separate ticket I will ask that we develop a system for displaying a "summary error status" badge on the toolbar button for the layer dialog. I'll make this an IPAC FIREFLY-* ticket.
            Hide
            Permalink
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            https://github.com/Caltech-IPAC/firefly/pull/796 for IPAC Firefly ticket Firefly-43

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - https://github.com/Caltech-IPAC/firefly/pull/796  for IPAC Firefly ticket Firefly-43

              People

              Assignee:
              cwang Cindy Wang [X] (Inactive)
              Reporter:
              xiuqin Xiuqin Wu [X] (Inactive)
              Watchers:
              Gregory Dubois-Felsmann, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.