Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36003

Error using Astropy to read catalogs output from Portal as VOTable-FITS

    XMLWordPrintable

Details

    • Portal
    • No

    Description

      After executing an ADQL query in the Portal Aspect, then exporting the results as "VOTable - FITS", I get an error when trying to read the file with Astropy's Table module (see traceback in comments).

       

      I have attached the offending file to this ticket.

      test_LVV-T1232_dp02_catalog.fits.vot

      Attachments

        Activity

          Traceback:

          In [*2*]: dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')

          ---------------------------------------------------------------------------

          KeyError                                  Traceback (most recent call last)

          Input In [2], in <cell line: 1>()

          ----> 1 dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/table/connect.py:62, in TableRead._call_(self, *args, **kwargs)

               59 units = kwargs.pop('units', None)

               60 descriptions = kwargs.pop('descriptions', None)

          ---> 62 out = self.registry.read(cls, *args, **kwargs)

               64 # For some readers (e.g., ascii.ecsv), the returned `out` class is not

               65 # guaranteed to be the same as the desired output `cls`.  If so,

               66 # try coercing to desired class without copying (io.registry.read

               67 # would normally do a copy).  The normal case here is swapping

               68 # Table <=> QTable.

               69 if cls is not out._class_:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/registry/core.py:212, in UnifiedInputRegistry.read(self, cls, format, cache, *args, **kwargs)

              208     format = self._get_valid_format(

              209         'read', cls, path, fileobj, args, kwargs)

              211 reader = self.get_reader(format, cls)

          --> 212 data = reader(*args, **kwargs)

              214 if not isinstance(data, cls):

              215     # User has read with a subclass where only the parent class is

              216     # registered.  This returns the parent class, so try coercing

              217     # to desired subclass.

              218     try:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/connect.py:87, in read_table_votable(input, table_id, use_names_over_ids, verify, **kwargs)

               50 """

               51 Read a Table object from an VO table file

               52

             (...)

               84     :func:`astropy.io.votable.table.parse`.

               85 """

               86 if not isinstance(input, (VOTableFile, VOTable)):

          ---> 87     input = parse(input, table_id=table_id, verify=verify, **kwargs)

               89 # Parse all table objects

               90 table_id_mapping = dict()

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/utils/decorators.py:546, in deprecated_renamed_argument.<locals>.decorator.<locals>.wrapper(*args, **kwargs)

              543             msg += f'\n        Use *{alternative}* instead.'

              544         warnings.warn(msg, warning_type, stacklevel=2)

          --> 546 return function(*args, **kwargs)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/table.py:160, in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)

              154     config['filename'] = source

              156 with iterparser.get_xml_iterator(

              157         source,

              158         _debug_python_based_parser=_debug_python_based_parser) as iterator:

              159     return tree.VOTableFile(

          --> 160         config=config, pos=(1, 1)).parse(iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3615, in VOTableFile.parse(self, iterator, config)

             3613 for start, tag, data, pos in iterator:

             3614     if start:

          -> 3615         tag_mapping.get(tag, self._add_unknown_tag)(

             3616             iterator, tag, data, config, pos)

             3617     elif tag == 'DESCRIPTION':

             3618         if self.description is not None:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3494, in VOTableFile._add_resource(self, iterator, tag, data, config, pos)

             3492 resource = Resource(config=config, pos=pos, **data)

             3493 self.resources.append(resource)

          -> 3494 resource.parse(self, iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3292, in Resource.parse(self, votable, iterator, config)

             3290 for start, tag, data, pos in iterator:

             3291     if start:

          -> 3292         tag_mapping.get(tag, self._add_unknown_tag)(

             3293             iterator, tag, data, config, pos)

             3294     elif tag == 'DESCRIPTION':

             3295         if self.description is not None:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3238, in Resource._add_table(self, iterator, tag, data, config, pos)

             3236 table = Table(self._votable, config=config, pos=pos, **data)

             3237 self.tables.append(table)

          -> 3238 table.parse(iterator, config)

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2538, in Table.parse(self, iterator, config)

             2536     except ValueError:

             2537         vo_raise(E17, (), config, pos)

          -> 2538     self.array = self._parse_fits(

             2539         iterator, extnum, config)

             2540     break

             2541 else:

           

          File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2805, in Table._parse_fits(self, iterator, extnum, config)

             2800 if start:

             2801     warn_unknown_attrs(

             2802         'STREAM', data.keys(), config, pos,

             2803         ['type', 'href', 'actuate', 'encoding', 'expires',

             2804          'rights'])

          -> 2805     href = data['href']

             2806     encoding = data.get('encoding', None)

             2807 else:

           

          KeyError: 'href'

          jcarlin Jeffrey Carlin added a comment - Traceback: In [*2*] : dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable') --------------------------------------------------------------------------- KeyError                                  Traceback (most recent call last) Input In [2] , in <cell line: 1>() ----> 1 dat_fits = Table.read('test_LVV-T1232_dp02_catalog.fits.vot', format='votable')   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/table/connect.py:62, in TableRead._ call _(self, *args, **kwargs)      59 units = kwargs.pop('units', None )      60 descriptions = kwargs.pop('descriptions', None ) ---> 62 out = self.registry.read(cls, *args, **kwargs)      64 # For some readers (e.g., ascii.ecsv), the returned `out` class is not      65 # guaranteed to be the same as the desired output `cls`.  If so,      66 # try coercing to desired class without copying (io.registry.read      67 # would normally do a copy).  The normal case here is swapping      68 # Table <=> QTable.      69 if cls is not out._ class _:   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/registry/core.py:212, in UnifiedInputRegistry.read(self, cls, format, cache, *args, **kwargs)     208     format = self._get_valid_format(     209         'read', cls, path, fileobj, args, kwargs)     211 reader = self.get_reader(format, cls) --> 212 data = reader(*args, **kwargs)     214 if not isinstance(data, cls):     215     # User has read with a subclass where only the parent class is     216     # registered.  This returns the parent class, so try coercing     217     # to desired subclass.     218     try :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/connect.py:87, in read_table_votable(input, table_id, use_names_over_ids, verify, **kwargs)      50 """      51 Read a Table object from an VO table file      52    (...)      84     :func:`astropy.io.votable.table.parse`.      85 """      86 if not isinstance(input, (VOTableFile, VOTable)): ---> 87     input = parse(input, table_id=table_id, verify=verify, **kwargs)      89 # Parse all table objects      90 table_id_mapping = dict()   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/utils/decorators.py:546, in deprecated_renamed_argument.<locals>.decorator.<locals>.wrapper(*args, **kwargs)     543             msg += f' \n         Use *{ alternative }* instead.'     544         warnings.warn(msg, warning_type, stacklevel=2) --> 546 return function(*args, **kwargs)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/table.py:160, in parse(source, columns, invalid, verify, chunk_size, table_number, table_id, filename, unit_format, datatype_mapping, _debug_python_based_parser)     154     config ['filename'] = source     156 with iterparser.get_xml_iterator(     157         source,     158         _debug_python_based_parser=_debug_python_based_parser) as iterator:     159     return tree.VOTableFile( --> 160         config=config, pos=(1, 1)).parse(iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3615, in VOTableFile.parse(self, iterator, config)    3613 for start, tag, data, pos in iterator:    3614     if start: -> 3615         tag_mapping.get(tag, self._add_unknown_tag)(    3616             iterator, tag, data, config, pos)    3617     elif tag == 'DESCRIPTION':    3618         if self.description is not None :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3494, in VOTableFile._add_resource(self, iterator, tag, data, config, pos)    3492 resource = Resource(config=config, pos=pos, **data)    3493 self.resources.append(resource) -> 3494 resource.parse(self, iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3292, in Resource.parse(self, votable, iterator, config)    3290 for start, tag, data, pos in iterator:    3291     if start: -> 3292         tag_mapping.get(tag, self._add_unknown_tag)(    3293             iterator, tag, data, config, pos)    3294     elif tag == 'DESCRIPTION':    3295         if self.description is not None :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:3238, in Resource._add_table(self, iterator, tag, data, config, pos)    3236 table = Table(self._votable, config=config, pos=pos, **data)    3237 self.tables.append(table) -> 3238 table.parse(iterator, config)   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2538, in Table.parse(self, iterator, config)    2536     except ValueError :    2537         vo_raise(E17, (), config, pos) -> 2538     self.array = self._parse_fits(    2539         iterator, extnum, config)    2540     break    2541 else :   File /opt/anaconda3/envs/py310/lib/python3.10/site-packages/astropy/io/votable/tree.py:2805, in Table._parse_fits(self, iterator, extnum, config)    2800 if start:    2801     warn_unknown_attrs(    2802         'STREAM', data.keys(), config, pos,    2803         ['type', 'href', 'actuate', 'encoding', 'expires',    2804           'rights']) -> 2805     href = data ['href']    2806     encoding = data.get('encoding', None )    2807 else :   KeyError: 'href'

          For context: jcarlin also reported that the same file was openable in TOPCAT without difficulty.

          gpdf Gregory Dubois-Felsmann added a comment - For context: jcarlin also reported that the same file was openable in TOPCAT without difficulty.

          The task in this ticket is to look at it enough to determine whether this is likely to be an Astropy bug, or if there's something wrong with the Portal/Firefly VOTable-with-FITS-payload persistence.

          gpdf Gregory Dubois-Felsmann added a comment - The task in this ticket is to look at it enough to determine whether this is likely to be an Astropy bug, or if there's something wrong with the Portal/Firefly VOTable-with-FITS-payload persistence.

          People

            Unassigned Unassigned
            jcarlin Jeffrey Carlin
            Gregory Dubois-Felsmann, Jeffrey Carlin, Loi Ly
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Jenkins

                No builds found.