Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36003

Error using Astropy to read catalogs output from Portal as VOTable-FITS

    XMLWordPrintable

    Details

    • Team:
      Portal
    • Urgent?:
      No

      Description

      After executing an ADQL query in the Portal Aspect, then exporting the results as "VOTable - FITS", I get an error when trying to read the file with Astropy's Table module (see traceback in comments).

       

      I have attached the offending file to this ticket.

      test_LVV-T1232_dp02_catalog.fits.vot

        Attachments

          Activity

          Hide
          jcarlin Jeffrey Carlin added a comment -

          Traceback:

          In [*2*]: dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')

          ---------------------------------------------------------------------------

          KeyError                                  Traceback (most recent call last)

          Input In [2], in <cell line: 1>()

          ----> 1 dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/table/connect.py:62, in TableRead._call_(self, *args, **kwargs)

               59 units = kwargs.pop('units', None)

               60 descriptions = kwargs.pop('descriptions', None)

          ---> 62 out = self.registry.read(cls, *args, **kwargs)

               64 # For some readers (e.g., ascii.ecsv), the returned `out` class is not

               65 # guaranteed to be the same as the desired output `cls`.  If so,

               66 # try coercing to desired class without copying (io.registry.read

               67 # would normally do a copy).  The normal case here is swapping

               68 # Table <=> QTable.

               69 if cls is not out._class_:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/registry/core.py:212, in UnifiedInputRegistry.read(self, cls, format, cache, *args, **kwargs)

              208     format = self._get_valid_format(

              209         'read', cls, path, fileobj, args, kwargs)

              211 reader = self.get_reader(format, cls)

          --> 212 data = reader(*args, **kwargs)

              214 if not isinstance(data, cls):

              215     # User has read with a subclass where only the parent class is

              216     # registered.  This returns the parent class, so try coercing

              217     # to desired subclass.

              218     try:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/connect.py:87, in read_table_votable(input, table_id, use_names_over_ids, verify, **kwargs)

               50 """

               51 Read a Table object from an VO table file

               52

             (...)

               84     :func:`astropy.io.votable.table.parse`.

               85 """

               86 if not isinstance(input, (VOTableFile, VOTable)):

          ---> 87     input = parse(input, table_id=table_id, verify=verify, **kwargs)

               89 # Parse all table objects

               90 table_id_mapping = dict()

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/utils/decorators.py:546, in deprecated_renamed_argument.<locals>.decorator.<locals>.wrapper(*args, **kwargs)

              543             msg += f'\n        Use *{alternative}* instead.'

              544         warnings.warn(msg, warning_type, stacklevel=2)

          --> 546 return function(*args, **kwargs)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/table.py:160, in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)

              154     config['filename'] = source

              156 with iterparser.get_xml_iterator(

              157         source,

              158         _debug_python_based_parser=_debug_python_based_parser) as iterator:

              159     return tree.VOTableFile(

          --> 160         config=config, pos=(1, 1)).parse(iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3615, in VOTableFile.parse(self, iterator, config)

             3613 for start, tag, data, pos in iterator:

             3614     if start:

          -> 3615         tag_mapping.get(tag, self._add_unknown_tag)(

             3616             iterator, tag, data, config, pos)

             3617     elif tag == 'DESCRIPTION':

             3618         if self.description is not None:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3494, in VOTableFile._add_resource(self, iterator, tag, data, config, pos)

             3492 resource = Resource(config=config, pos=pos, **data)

             3493 self.resources.append(resource)

          -> 3494 resource.parse(self, iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3292, in Resource.parse(self, votable, iterator, config)

             3290 for start, tag, data, pos in iterator:

             3291     if start:

          -> 3292         tag_mapping.get(tag, self._add_unknown_tag)(

             3293             iterator, tag, data, config, pos)

             3294     elif tag == 'DESCRIPTION':

             3295         if self.description is not None:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3238, in Resource._add_table(self, iterator, tag, data, config, pos)

             3236 table = Table(self._votable, config=config, pos=pos, **data)

             3237 self.tables.append(table)

          -> 3238 table.parse(iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2538, in Table.parse(self, iterator, config)

             2536     except ValueError:

             2537         vo_raise(E17, (), config, pos)

          -> 2538     self.array = self._parse_fits(

             2539         iterator, extnum, config)

             2540     break

             2541 else:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2805, in Table._parse_fits(self, iterator, extnum, config)

             2800 if start:

             2801     warn_unknown_attrs(

             2802         'STREAM', data.keys(), config, pos,

             2803         ['type', 'href', 'actuate', 'encoding', 'expires',

             2804          'rights'])

          -> 2805     href = data['href']

             2806     encoding = data.get('encoding', None)

             2807 else:

           

          KeyError: 'href'

          Show
          jcarlin Jeffrey Carlin added a comment - Traceback: In [*2*] : dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable') --------------------------------------------------------------------------- KeyError                                  Traceback (most recent call last) Input In [2] , in <cell line: 1>() ----> 1 dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/table/connect.py:62, in TableRead._ call _(self, *args, **kwargs)      59 units = kwargs.pop('units', None )      60 descriptions = kwargs.pop('descriptions', None ) ---> 62 out = self.registry.read(cls, *args, **kwargs)      64 # For some readers (e.g., ascii.ecsv), the returned `out` class is not      65 # guaranteed to be the same as the desired output `cls`.  If so,      66 # try coercing to desired class without copying (io.registry.read      67 # would normally do a copy).  The normal case here is swapping      68 # Table <=> QTable.      69 if cls is not out._ class _:   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/registry/core.py:212, in UnifiedInputRegistry.read(self, cls, format, cache, *args, **kwargs)     208     format = self._get_valid_format(     209         'read', cls, path, fileobj, args, kwargs)     211 reader = self.get_reader(format, cls) --> 212 data = reader(*args, **kwargs)     214 if not isinstance(data, cls):     215     # User has read with a subclass where only the parent class is     216     # registered.  This returns the parent class, so try coercing     217     # to desired subclass.     218     try :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/connect.py:87, in read_table_votable(input, table_id, use_names_over_ids, verify, **kwargs)      50 """      51 Read a Table object from an VO table file      52    (...)      84     :func:`astropy.io.votable.table.parse`.      85 """      86 if not isinstance(input, (VOTableFile, VOTable)): ---> 87     input = parse(input, table_id=table_id, verify=verify, **kwargs)      89 # Parse all table objects      90 table_id_mapping = dict()   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/utils/decorators.py:546, in deprecated_renamed_argument.<locals>.decorator.<locals>.wrapper(*args, **kwargs)     543             msg += f' \n         Use *{ alternative }* instead.'     544         warnings.warn(msg, warning_type, stacklevel=2) --> 546 return function(*args, **kwargs)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/table.py:160, in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)     154     config ['filename'] = source     156 with iterparser.get_xml_iterator(     157         source,     158         _debug_python_based_parser=_debug_python_based_parser) as iterator:     159     return tree.VOTableFile( --> 160         config=config, pos=(1, 1)).parse(iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3615, in VOTableFile.parse(self, iterator, config)    3613 for start, tag, data, pos in iterator:    3614     if start: -> 3615         tag_mapping.get(tag, self._add_unknown_tag)(    3616             iterator, tag, data, config, pos)    3617     elif tag == 'DESCRIPTION':    3618         if self.description is not None :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3494, in VOTableFile._add_resource(self, iterator, tag, data, config, pos)    3492 resource = Resource(config=config, pos=pos, **data)    3493 self.resources.append(resource) -> 3494 resource.parse(self, iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3292, in Resource.parse(self, votable, iterator, config)    3290 for start, tag, data, pos in iterator:    3291     if start: -> 3292         tag_mapping.get(tag, self._add_unknown_tag)(    3293             iterator, tag, data, config, pos)    3294     elif tag == 'DESCRIPTION':    3295         if self.description is not None :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3238, in Resource._add_table(self, iterator, tag, data, config, pos)    3236 table = Table(self._votable, config=config, pos=pos, **data)    3237 self.tables.append(table) -> 3238 table.parse(iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2538, in Table.parse(self, iterator, config)    2536     except ValueError :    2537         vo_raise(E17, (), config, pos) -> 2538     self.array = self._parse_fits(    2539         iterator, extnum, config)    2540     break    2541 else :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2805, in Table._parse_fits(self, iterator, extnum, config)    2800 if start:    2801     warn_unknown_attrs(    2802         'STREAM', data.keys(), config, pos,    2803         ['type', 'href', 'actuate', 'encoding', 'expires',    2804           'rights']) -> 2805     href = data ['href']    2806     encoding = data.get('encoding', None )    2807 else :   KeyError: 'href'
          Hide
          gpdf Gregory Dubois-Felsmann added a comment -

          For context: Jeffrey Carlin also reported that the same file was openable in TOPCAT without difficulty.

          Show
          gpdf Gregory Dubois-Felsmann added a comment - For context: Jeffrey Carlin also reported that the same file was openable in TOPCAT without difficulty.
          Hide
          gpdf Gregory Dubois-Felsmann added a comment -

          The task in this ticket is to look at it enough to determine whether this is likely to be an Astropy bug, or if there's something wrong with the Portal/Firefly VOTable-with-FITS-payload persistence.

          Show
          gpdf Gregory Dubois-Felsmann added a comment - The task in this ticket is to look at it enough to determine whether this is likely to be an Astropy bug, or if there's something wrong with the Portal/Firefly VOTable-with-FITS-payload persistence.

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            jcarlin Jeffrey Carlin
            Watchers:
            Gregory Dubois-Felsmann, Jeffrey Carlin, Loi Ly
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:

                Jenkins

                No builds found.