[DM-25347] Integer out of range when trying to convert RC2 rerun using postgresql - Jira

Details

Type: Story
Status: Done
Resolution: Done
Fix Version/s: None
Component/s: daf_butler
Labels:
- gen3-middleware

Story Points:
1
Epic Link:
f20a-drp-gen3
Team:
Data Release Production
Urgent?:
No

Description

While running gen3-hsc-rc2/bootstrap.py on recent RC2 rerun (w_2020_22) using PG, it failed with the following.

INFO  2020-06-09T14:03:29.926-0500 convertRepo - Ingesting 241 deepCoadd_ref datasets into run RC2/w_2020_22/DM-25176/remainder.

INFO  2020-06-09T14:03:31.115-0500 convertRepo - Ingesting 241 objectTable datasets into run RC2/w_2020_22/DM-25176/remainder.

INFO  2020-06-09T14:03:32.539-0500 convertRepo - Ingesting 3 objectTable_tract datasets into run RC2/w_2020_22/DM-25176/remainder.

Traceback (most recent call last):

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1228, in _execute_context

    cursor, statement, parameters, context

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 857, in do_executemany

    cursor.executemany(statement, parameters)

psycopg2.errors.NumericValueOutOfRange: integer out of range

The above exception was the direct cause of the following exception:Traceback (most recent call last):

  File "./gen3-hsc-rc2/bootstrap.py", line 269, in <module>

    main()

  File "./gen3-hsc-rc2/bootstrap.py", line 265, in main

    continue_=options.continue_, reruns=reruns)

  File "./gen3-hsc-rc2/bootstrap.py", line 216, in run

    visits=makeVisitList(tracts, filters)

  File "/software/lsstsw/stack_20200515/stack/miniconda3-4.7.12-46b24e8/Linux64/obs_base/19.0.0-72-g37abf38+2/python/lsst/obs/base/gen2to3/convertRepo.py", line 572, in run

    converter.ingest()

  File "/software/lsstsw/stack_20200515/stack/miniconda3-4.7.12-46b24e8/Linux64/obs_base/19.0.0-72-g37abf38+2/python/lsst/obs/base/gen2to3/repoConverter.py", line 495, in ingest

    self.task.butler3.ingest(*datasetsForType, transfer=self.task.config.transfer, run=run)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/core/utils.py", line 249, in inner

    return func(self, *args, **kwargs)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/_butler.py", line 1214, in ingest

    self.datastore.ingest(*datasets, transfer=transfer)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/core/datastore.py", line 559, in ingest

    self._finishIngest(prepData, transfer=transfer)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/core/utils.py", line 249, in inner

    return func(self, *args, **kwargs)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/datastores/fileLikeDatastore.py", line 736, in _finishIngest

    self._register_datasets(refsAndInfos)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/datastores/genericDatastore.py", line 155, in _register_datasets

    self.addStoredItemInfo(expandedRefs, expandedItemInfos)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/datastores/fileLikeDatastore.py", line 327, in addStoredItemInfo

    self._table.insert(*records)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/registry/opaque.py", line 67, in insert

    self._db.insert(self._table, *data)

  File "/scratch/mgower/rc2_convert_w19/daf_butler/python/lsst/daf/butler/registry/interfaces/_database.py", line 955, in insert

    self._connection.execute(table.insert(), *rows)

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 984, in execute

    return meth(self, multiparams, params)

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection

    return connection._execute_clauseelement(self, multiparams, params)

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1103, in _execute_clauseelement

    distilled_params,

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1288, in _execute_context

    e, statement, parameters, cursor, context

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1482, in _handle_dbapi_exception

    sqlalchemy_exception, with_traceback=exc_info[2], from_=e

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 178, in raise_

    raise exception

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1228, in _execute_context

    cursor, statement, parameters, context

  File "/software/lsstsw/stack_20200515/python/miniconda3-4.7.12/envs/lsst-scipipe/lib/python3.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 857, in do_executemany

    cursor.executemany(statement, parameters)

sqlalchemy.exc.DataError: (psycopg2.errors.NumericValueOutOfRange) integer out of range[SQL: INSERT INTO mgower_4.posix_datastore_records (dataset_id, path, formatter, storage_class, component, checksum, file_size) VALUES (%(dataset_id)s, %(path)s, %(formatter)s, %(storage_class)s, %(component)s, %(checksum)s, %(file_size)s)]

[parameters: ({'dataset_id': 605010, 'path': 'RC2/w_2020_22/DM-25176/remainder/objectTable_tract/9615/objectTable_tract_9615_hsc_rings_v1_RC2_w_2020_22_DM-25176_remainder.parq', 'formatter': 'lsst.daf.butler.formatters.parquetFormatter.ParquetFormatter', 'storage_class': 'DataFrame', 'component': '__NULL_STRING__', 'checksum': None, 'file_size': 2591970489}, {'dataset_id': 605011, 'path': 'RC2/w_2020_22/DM-25176/remainder/objectTable_tract/9813/objectTable_tract_9813_hsc_rings_v1_RC2_w_2020_22_DM-25176_remainder.parq', 'formatter': 'lsst.daf.butler.formatters.parquetFormatter.ParquetFormatter', 'storage_class': 'DataFrame', 'component': '__NULL_STRING__', 'checksum': None, 'file_size': 2911408427}, {'dataset_id': 605012, 'path': 'RC2/w_2020_22/DM-25176/remainder/objectTable_tract/9697/objectTable_tract_9697_hsc_rings_v1_RC2_w_2020_22_DM-25176_remainder.parq', 'formatter': 'lsst.daf.butler.formatters.parquetFormatter.ParquetFormatter', 'storage_class': 'DataFrame', 'component': '__NULL_STRING__', 'checksum': None, 'file_size': 2236160139})]

(Background on this error at: http://sqlalche.me/e/9h9h)

Attachments

Issue Links

blocks

DM-25297 Convert RC2 Gen2 run to Gen3 as test of process and code

Done

is duplicated by

DM-24914 Check integer column types throughout Gen3 database schema

Invalid

Activity

Hide

Permalink

Jim Bosch added a comment - 09/Jun/20 9:59 PM

Changes are small, and will hopefully unblock Michelle Gower on RC2 vs. PostgreSQL (and help Dominique Boutigny next time he starts over). Jenkins is running, and I expect it to confirm my belief that it's just daf_butler that's affected: https://github.com/lsst/daf_butler/pull/308 .

Show

Jim Bosch added a comment - 09/Jun/20 9:59 PM Changes are small, and will hopefully unblock Michelle Gower on RC2 vs. PostgreSQL (and help Dominique Boutigny next time he starts over). Jenkins is running, and I expect it to confirm my belief that it's just daf_butler that's affected: https://github.com/lsst/daf_butler/pull/308 .

Data Management

Integer out of range when trying to convert RC2 rerun using postgresql

Details

Description

Attachments

Issue Links

Activity

People

Dates

Jenkins