# Add sqlalchemy engine timeout setting to Ppdb config

XMLWordPrintable

## Details

• Type: Improvement
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
• Story Points:
2
• Team:

## Description

Mutli-processing runs of ap_pipe using the sqlite setting for Ppdb are running into the issue of timeouts when storing, assessing the database. This is a similar issue to what to was discovered in the previous sqlite, db backend. This ticket will add a timeout config to Ppdb that will allow this setting to be increased for both sqlite and other db options.

Additionally, Andy Salnikov stated that PpdbProtoVisits was not meant to be used in production, only his testing. As such I will remove this call from being used in ap_association.

## Activity

Hide
Chris Morrison added a comment -

Meredith Rawls, could you please try running with this ticket using ap_pipe and your slurm setup? To use the new feature you will only have to add ppdb.connection_timeout=60 to your config.

Show
Chris Morrison added a comment - Meredith Rawls , could you please try running with this ticket using ap_pipe and your slurm setup? To use the new feature you will only have to add ppdb.connection_timeout=60 to your config.   Jenkins run:  https://ci.lsst.codes/blue/organizations/jenkins/stack-os-matrix/detail/stack-os-matrix/29099/pipeline
Hide
Meredith Rawls added a comment -

Thanks for catching this error before I noticed! I've kicked off a new ap_pipe job using the branches for this ticket and your new config setting, but there's a bit of a slurm backlog at the moment.

Show
Meredith Rawls added a comment - Thanks for catching this error before I noticed! I've kicked off a new ap_pipe job using the branches for this ticket and your new config setting, but there's a bit of a slurm backlog at the moment.
Hide
Andy Salnikov added a comment -

Looks, good, couple of minor comments for dax_ppdb.

Show
Andy Salnikov added a comment - Looks, good, couple of minor comments for dax_ppdb .
Hide
Chris Morrison added a comment -

Per Andy Salnikov's comment on the dax_ppdb pull request.

A bug was found in inf was not properly mapped into the Ppdb based on runs by Meredith Rawls of the HiTS2015 dataset. To safely account for these values, the isnan() mapping was changed to not isfinite(). Since this was a small change and necessary for Meredith Rawls to rerun the HiTS dataset, I included the change in this ticket.

Show
Chris Morrison added a comment - Per Andy Salnikov 's comment on the dax_ppdb pull request. A bug was found in inf was not properly mapped into the Ppdb based on runs by Meredith Rawls of the HiTS2015 dataset. To safely account for these values, the isnan() mapping was changed to not isfinite(). Since this was a small change and necessary for Meredith Rawls to rerun the HiTS dataset, I included the change in this ticket.

## People

• Assignee:
Chris Morrison
Reporter:
Chris Morrison
Reviewers:
Andy Salnikov
Watchers:
Andy Salnikov, Chris Morrison, Eric Bellm, John Swinbank, Meredith Rawls