# Fix doxygen errors in pipe_tasks

XMLWordPrintable

#### Details

• Type: Story
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
4
• Sprint:
AP S22-5 (April)
• Team:
• Urgent?:
No

#### Description

Examining the build logs for pipe_tasks demonstrated that we are still using doxygen to build the docs and there are some unexplained slow downs during that process. In particular between CharacterizeImageTask and MakeCoaddTempExpTask there is a 2 minute pause in logging on a Jenkins run:

It's possibly that fixing the errors will fix the slow down. Ideally we would convert to sphinx but that is beyond the scope of this ticket.

#### Activity

Hide
Krzysztof Findeisen added a comment -

I've started work on this, but I can't reproduce the slowdown locally. scons doc runs noisily but quickly. I do see a big slowdown right before

 passed running global pytest...

which in the linked Jenkins log seems to have no delay.

Show
Krzysztof Findeisen added a comment - I've started work on this, but I can't reproduce the slowdown locally. scons doc runs noisily but quickly. I do see a big slowdown right before passed running global pytest... which in the linked Jenkins log seems to have no delay.
Hide
Tim Jenness added a comment -

Running locally with scons my log says:

 scons: Nothing to be done for examples'. buildConfig(["doc/doxygen.conf"], ["doc/doxygen.conf.in"]) running tests/nopytest_test_coadds.py... doxygen /Users/timj/work/lsstsw/build/pipe_tasks/doc/doxygen.conf error: Tag file '/Users/timj/work/lsstsw/stack/lsst-scipipe-3.0.0/Darwin/utils/g617c0b0dc2+9633a190c8/doc/utils.tag' does not exist or is not a file. Skipping it... error: Tag file '/Users/timj/work/lsstsw/stack/lsst-scipipe-3.0.0/Darwin/obs_base/g7a69c27ea0+bead29cdf2/doc/obs_base.tag' does not exist or is not a file. Skipping it... error: Tag file '/Users/timj/work/lsstsw/stack/lsst-scipipe-3.0.0/Darwin/daf_butler/g8243fdef33+f83c363f3e/doc/daf_butler.tag' does not exist or is not a file. Skipping it... /Users/timj/work/lsstsw/build/pipe_tasks/python/lsst/pipe/tasks/transformMeasurement.py:173: warning: multiple use of section label 'pipe_tasks_transform_Contents' while adding section, (first occurrence: /Users/timj/work/lsstsw/build/pipe_tasks/python/lsst/pipe/tasks/transformMeasurement.py, line 61) /Users/timj/work/lsstsw/build/pipe_tasks/python/lsst/pipe/tasks/calibrate.py:339: warning: Found recursive @copybrief or @copydoc relation for argument 'CalibrateTask'.

and then there is a pause right at the end where it says "passed". That gap is caused by the nopytest_test_coadds.py running. We block the pytest startup until the standalone tests are run. This is all consistent with the doxygen and nopytest test running in parallel. The confusing thing about the Jenkins output is that the pause is in the middle of the oxygen output, not at the end of it. Is it possible that eupspkg capturing the standard out from scons is causing the confusing timestamps and the gap is from the test but the doxygen log entries are buffered? I think Eli Rykoff saw the same problem.

Show
Hide
Eli Rykoff added a comment -

Yes, I agree that this pause is due to nopytest_test_coadds.py and not from doxygen. That said, the log timestamps are confusing and Tim's hypothesis (or something related to different log buffers) might explain that part.

Show
Eli Rykoff added a comment - Yes, I agree that this pause is due to nopytest_test_coadds.py and not from doxygen . That said, the log timestamps are confusing and Tim's hypothesis (or something related to different log buffers) might explain that part.
Hide
Krzysztof Findeisen added a comment -

Thanks for agreeing to review this, Kian-Tat Lim!

Show
Krzysztof Findeisen added a comment - Thanks for agreeing to review this, Kian-Tat Lim !
Hide
Kian-Tat Lim added a comment -

It looked like there was one still-correct line that was maybe inadvertently removed.  Otherwise, looks fine.

Show
Kian-Tat Lim added a comment - It looked like there was one still-correct line that was maybe inadvertently removed.  Otherwise, looks fine.

#### People

Assignee:
Krzysztof Findeisen
Reporter:
Tim Jenness
Reviewers:
Kian-Tat Lim
Watchers:
Eli Rykoff, Ian Sullivan, Kian-Tat Lim, Krzysztof Findeisen, Tim Jenness