Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-16048

Save tables in other formats develop infrastructure and implement csv and tsv

    XMLWordPrintable

    Details

      Description

      So far, Firefly saves the table data in IPAC table format, adding extra meta data about the table as key=value in the file header before data.  When DM-15826 is implemented, Firefly will have more meta data about table supplied by VOTable, and keeping them in the saved file would be desirable. 

      Develop the infrastructure and and implement csv and tsv.

       

      Related tickets proposes to give user options to save the table in VOTable form and maybe FITS table format too. 

        Attachments

          Issue Links

            Activity

            Hide
            desai Vandana Desai added a comment - - edited

            I was told to test this feature, even though the ticket says "Done". Here are my comments. I realize many of these need to be spun off into separate tickets.

            1.  In the "Save table" dialog, the help link doesn't go anywhere useful.  (DM-17110)
             
            2. In the "Save table" dialog, the second line says "Save as" I think to be analogous with "File format" that appears just above, "Save as" should be changed to "File name". (DM-17110)
             
            3. The default filename for a cone search has a dash, underscore, parenthesis, colon, space, and double quotes. 
            Example: WISE-allwise_p3as_psd (Cone_100'').csv
             
            This is too many special characters. 
             
            I suggest we remove all of the special characters except for dashes and underscores. Replace parentheses with dashes. Replace double-quotes with 'asec'. Remove spaces. Replace decimal points with 'p' for 'point'.
             
            Examples:
            WISE-allwise_p3as_psd-Cone100asec.csv
            WISE-allwise_p3as_psd-Ellipse60_0_0p26_asec.csvWISE-allwise_p3as_psd-Box100asec.csv
            WISE-allwise_p3as_psd-Polygon.csv
            WISE-allwise_p3as_psd-MultiObject.csv
            IRAS-irasgal-AllSky.csv
             (DM-17110)
            4. For elliptical searches, the filename comes out with the string 'Eliptical'. This should be spelled 'Elliptical'. Actually, to be analogous with 'Cone', we should use 'Ellipse'. Also, the ellipse searches only provide the semi-major Axis, not the position angle or axial ratio. We need all, separated by underscores. (DM-17110)
             
            5. When you upload a table, there is no string to indicate what type of search was done. I suggest appending 'MultiObject'. (DM-17110)
             
            6. We don't have one-to-one match in the Firefly Catalog Search! (IRSA-2542)
             
            7. In the "Load Catalog Dialog", the chosen filename is severely truncated in the display, even though there is tons of room. Also, I see "Custom catalog in IPAC table format" in that dialog, even though I seem to be able to load csv and tsv files. Finally, when I click on the help, it doesn't go to the right place.  (DM-17110)
             
            8. TSV and CSV can't be read into Gator for the Multi-Object Search. (IRSA-2541)
             

            Show
            desai Vandana Desai added a comment - - edited I was told to test this feature, even though the ticket says "Done". Here are my comments. I realize many of these need to be spun off into separate tickets. 1.  In the "Save table" dialog, the help link doesn't go anywhere useful.  ( DM-17110 )   2. In the "Save table" dialog, the second line says "Save as" I think to be analogous with "File format" that appears just above, "Save as" should be changed to "File name". ( DM-17110 )   3. The default filename for a cone search has a dash, underscore, parenthesis, colon, space, and double quotes.  Example: WISE-allwise_p3as_psd (Cone_100'').csv   This is too many special characters.    I suggest we remove all of the special characters except for dashes and underscores. Replace parentheses with dashes. Replace double-quotes with 'asec'. Remove spaces. Replace decimal points with 'p' for 'point'.   Examples: WISE-allwise_p3as_psd-Cone100asec.csv WISE-allwise_p3as_psd-Ellipse60_0_0p26_asec.csvWISE-allwise_p3as_psd-Box100asec.csv WISE-allwise_p3as_psd-Polygon.csv WISE-allwise_p3as_psd-MultiObject.csv IRAS-irasgal-AllSky.csv  ( DM-17110 ) 4. For elliptical searches, the filename comes out with the string 'Eliptical'. This should be spelled 'Elliptical'. Actually, to be analogous with 'Cone', we should use 'Ellipse'. Also, the ellipse searches only provide the semi-major Axis, not the position angle or axial ratio. We need all, separated by underscores. ( DM-17110 )   5. When you upload a table, there is no string to indicate what type of search was done. I suggest appending 'MultiObject'. ( DM-17110 )   6. We don't have one-to-one match in the Firefly Catalog Search! (IRSA-2542)   7. In the "Load Catalog Dialog", the chosen filename is severely truncated in the display, even though there is tons of room. Also, I see "Custom catalog in IPAC table format" in that dialog, even though I seem to be able to load csv and tsv files. Finally, when I click on the help, it doesn't go to the right place.  ( DM-17110 )   8. TSV and CSV can't be read into Gator for the Multi-Object Search. (IRSA-2541)  
            Hide
            ejoliet Emmanuel Joliet added a comment -

            Great, thanks for testing!!

            I won't comment on the labels, that's easy fix once agreed.

            I'm concerned about item 8. We should make a ticket for that.

            8. TSV and CSV can't be read into Gator for the Multi-Object Search.

            For item 3., this is an old story and there is many inconsistencies everywhere and because the core functionality is used in IFE applications, we should fix those issues as a whole instead of fixing it in this particular case. It happens also when saving images, on downloads, etc.

             

            Show
            ejoliet Emmanuel Joliet added a comment - Great, thanks for testing!! I won't comment on the labels, that's easy fix once agreed. I'm concerned about item 8. We should make a ticket for that. 8. TSV and CSV can't be read into Gator for the Multi-Object Search. For item 3., this is an old story and there is many inconsistencies everywhere and because the core functionality is used in IFE applications, we should fix those issues as a whole instead of fixing it in this particular case. It happens also when saving images, on downloads, etc.  
            Hide
            xiuqin Xiuqin Wu [X] (Inactive) added a comment -

            Thank you, Vandana Desai.

            I will make more tickets out of your feedback and suggestions.

            Show
            xiuqin Xiuqin Wu [X] (Inactive) added a comment - Thank you, Vandana Desai . I will make more tickets out of your feedback and suggestions.
            Hide
            roby Trey Roby added a comment -

            Good feedback Vandana Desai, thank you.

            We should put the more straight forward issues in to DM-17110 and DM-17111 since we plan to do those are part of the whole file saving task.

            Show
            roby Trey Roby added a comment - Good feedback Vandana Desai , thank you. We should put the more straight forward issues in to DM-17110 and DM-17111 since we plan to do those are part of the whole file saving task.
            Hide
            roby Trey Roby added a comment -

            As far as TSV and CVS, we save a title line,  which I think this is the right thing to do.  It is probable that gator does not recognize a title line.

             

             

            Show
            roby Trey Roby added a comment - As far as TSV and CVS, we save a title line,  which I think this is the right thing to do.  It is probable that gator does not recognize a title line.    
            Hide
            ejoliet Emmanuel Joliet added a comment -

            Fixed in version dev and upcoming release-2019.1

            Show
            ejoliet Emmanuel Joliet added a comment - Fixed in version dev and upcoming release-2019.1

              People

              Assignee:
              cwang Cindy Wang [X] (Inactive)
              Reporter:
              xiuqin Xiuqin Wu [X] (Inactive)
              Reviewers:
              Loi Ly, Trey Roby
              Watchers:
              Emmanuel Joliet, Gregory Dubois-Felsmann, Loi Ly, Trey Roby, Vandana Desai, Xiuqin Wu [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Jenkins

                  No builds found.