I did not check multi-process option for some time and now it breaks:
Traceback (most recent call last):
File "/project/salnikov/DM-15686/pipe_supertask/bin/stac", line 25, in <module>
File "/project/salnikov/DM-15686/pipe_supertask/python/lsst/pipe/supertask/cmdLineFwk.py", line 226, in parseAndRun
return self.runPipeline(qgraph, butler, args)
File "/project/salnikov/DM-15686/pipe_supertask/python/lsst/pipe/supertask/cmdLineFwk.py", line 372, in runPipeline
File "/project/salnikov/DM-15686/pipe_supertask/python/lsst/pipe/supertask/cmdLineFwk.py", line 96, in __call__
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/multiprocessing/pool.py", line 644, in get
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/multiprocessing/pool.py", line 424, in _handle_tasks
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/multiprocessing/connection.py", line 206, in send
File "/software/lsstsw/stack3_20171023/python/miniconda3-4.3.21/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
TypeError: can't pickle lsst.log.log.log.Log objects
Need to fix it ASAP
Re-implement task execution in laptop activator
Ready for review, most changes are trivial (replacing lsst.log with logging) but there is small piece of code that fixes pickle support which I hope is done right now. Unit tests were added for that.
Jenkins build passes, no big rush with that, can wait.
Tim Jenness, thanks for flake8 fix. I have rebased branch on new master, now passes both Travis and Jenkins. Do you plan to review pipe_supertask too or do you trust me on that? If latter then you can just mark the ticket as reviewed, I'll merge both packages.
Ok. I had a quick look at pipe_supertask. Looks fine, although in daf_butler I have not been stripping lsst from the name when using the logging package. I hadn't thought about it.
Thanks, I'll merge things then. Developer guide says we are supposed to strip "lsst.": https://developer.lsst.io/stack/logging.html#logger-names
Merged and done