Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16809

Science discussion of "preferred/popular data" for TAP_schema presentation

    XMLWordPrintable

    Details

    • Story Points:
      2
    • Sprint:
      SUIT Sprint 2019-01, SUIT Sprint 2019-02, SUIT Sprint 2019-03, SUIT Sprint 2019-04, SUIT Sprint 2019-05, SUIT Sprint 2019-06
    • Team:
      DM Science

      Description

      Lead a discussion and propose ways to provide the information for "preferred/popular data" from project point view, how application should respond to the given information and present UI accordingly.  

        Attachments

          Issue Links

            Activity

            Hide
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            The team spent more than one hour to discuss the pros and cons of having an application controlled configuration file to override the schema-index and table_index to reflect the sequence of importance, making the more important schemas and tables appear first in the list. In the end, we decided it is really the data service provider's responsibility to give those index numbers. Although it could make Firefly looks bad displaying those lists without obvious importance sequence, but Firefly does not have the know-how and it could become a maintenance nightmare. 

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - The team spent more than one hour to discuss the pros and cons of having an application controlled configuration file to override the schema-index and table_index to reflect the sequence of importance, making the more important schemas and tables appear first in the list. In the end, we decided it is really the data service provider's responsibility to give those index numbers. Although it could make Firefly looks bad displaying those lists without obvious importance sequence, but Firefly does not have the know-how and it could become a maintenance nightmare. 
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            I agree with everything Xiuqin Wu [X] said above as far as the overrides are concerned. After all that discussion, it seemed like the realistic use cases for overrides were fewer and fewer.

            However, I think there is value in proposing data-publisher-side metadata that would be useful to us, and in particular, it might be valuable to have something similar to "principal" (a column attribute) but at the table, or even schema, level. I would propose to discuss this with other data providers at the IVOA meeting in May (and we can discuss this within IRSA before that as well).

            Reverting to "In Progress".

            Show
            gpdf Gregory Dubois-Felsmann added a comment - I agree with everything Xiuqin Wu [X] said above as far as the overrides are concerned. After all that discussion, it seemed like the realistic use cases for overrides were fewer and fewer. However, I think there is value in proposing data-publisher-side metadata that would be useful to us, and in particular, it might be valuable to have something similar to "principal" (a column attribute) but at the table, or even schema, level. I would propose to discuss this with other data providers at the IVOA meeting in May (and we can discuss this within IRSA before that as well). Reverting to "In Progress".
            Hide
            gpdf Gregory Dubois-Felsmann added a comment -

            Closing this ticket with an explanation and some context for how we think about this now.

            The original discussion within the Firefly/SUIT team was about whether Firefly should include a layer that would permit us to remap an arbitrary external TAP service, even one we don't control, to change certain key TAP_SCHEMA parameters.  We discussed the possibility of remapping schema_index and table_index, in particular, with the aim of being able to reorder the presentation of a poorly-curated external service to improve its usability.

            Within the Rubin Observatory context, the significance of this was that we were wondering about a scenario in which we (the Portal/SUIT team) were handed a TAP service on a take-it-or-leave-it basis by upstream components of the project, with a low-quality curation of TAP_SCHEMA.

            This is what led to Xiuqin's comment from March 2019 above.  We decided that a) within the Rubin Observatory project this could be avoided simply by working together in a more integrated way (which, in the end, is essentially how we are doing things now), b) these sorts of overrides to independent external services would be essentially impossible to maintain.


            What remains of this idea is that within the Rubin project I believe that we still need the ability to reorder the schemas in TAP_SCHEMA - i.e., to change their schema_index values, at a level "higher" (or, let's say, "after") the schema contents are defined.

            It's simply none of a schema's business to know or claim its relative position in the presentation order of the TAP service as a whole.  Separation of concerns leads to the notion that there must at least be a way to reassign schema_index values by a central authority without requiring the maintenance headache of edits to each source of the rest of the TAP_SCHEMA information.

            Show
            gpdf Gregory Dubois-Felsmann added a comment - Closing this ticket with an explanation and some context for how we think about this now. The original discussion within the Firefly/SUIT team was about whether Firefly should include a layer that would permit us to remap an arbitrary external TAP service, even one we don't control, to change certain key TAP_SCHEMA parameters.  We discussed the possibility of remapping schema_index and table_index , in particular, with the aim of being able to reorder the presentation of a poorly-curated external service to improve its usability. Within the Rubin Observatory context, the significance of this was that we were wondering about a scenario in which we (the Portal/SUIT team) were handed a TAP service on a take-it-or-leave-it basis by upstream components of the project, with a low-quality curation of TAP_SCHEMA. This is what led to Xiuqin's comment from March 2019 above.  We decided that a) within the Rubin Observatory project this could be avoided simply by working together in a more integrated way (which, in the end, is essentially how we are doing things now), b) these sorts of overrides to independent external services would be essentially impossible to maintain. What remains of this idea is that within the Rubin project I believe that we still need the ability to reorder the schemas in TAP_SCHEMA - i.e., to change their schema_index values, at a level "higher" (or, let's say, "after") the schema contents are defined. It's simply none of a schema's business to know or claim its relative position in the presentation order of the TAP service as a whole.  Separation of concerns leads to the notion that there must at least be a way to reassign schema_index values by a central authority without requiring the maintenance headache of edits to each source of the rest of the TAP_SCHEMA information.

              People

              Assignee:
              gpdf Gregory Dubois-Felsmann
              Reporter:
              xiuqin Xiuqin Wu [X] (Inactive)
              Watchers:
              Gregory Dubois-Felsmann, Vandana Desai, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.