Details
-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Fix Version/s: None
-
Component/s: None
-
Labels:None
-
Epic Link:
-
Sprint:DB_S17_2
-
Team:Data Access and Database
Description
Investigate why the job could not be found and take appropriate action.
[2017-02-06T19:14:21.430-0600] [LWP:20631] DEBUG qdisp.QueryRequest (core/modules/qdisp/QueryRequest.cc:194) - QI=1943:33; ProcessResponseData with buflen=118 (more)
|
[2017-02-06T19:14:21.561-0600] [LWP:20631] DEBUG rproc.InfileMerger (core/modules/rproc/InfileMerger.cc:155) - Executing InfileMerger::merge(QI=1943:33; largeResult=0 sizes=45, 118, rowCount=0, row_size=0, errCode=0 hasErMsg=1)
|
[2017-02-06T19:14:21.726-0600] [LWP:20631] DEBUG qdisp.QueryRequest (core/modules/qdisp/QueryRequest.cc:235) - QI=1943:33; ProcessResponse data flush failed
|
[2017-02-06T19:14:21.745-0600] [LWP:20631] DEBUG qdisp.QueryRequest (core/modules/qdisp/QueryRequest.cc:306) - QI=1943:33; QueryRequest::_errorFinish() shouldCancel=0
|
[2017-02-06T19:14:25.279-0600] [LWP:20705] DEBUG qdisp.Executive (core/modules/qdisp/Executive.cc:377) - Executive reaped requester for QI=1943:33;
|
[2017-02-06T19:14:26.692-0600] [LWP:20631] DEBUG qdisp.QueryRequest (core/modules/qdisp/QueryRequest.cc:324) - QI=1943:33; QueryRequest::_errorFinish ok
|
[2017-02-06T19:14:26.713-0600] [LWP:20631] DEBUG qdisp.Executive (core/modules/qdisp/Executive.cc:215) - Executive::markCompleted QI=1943:33; 0
|
[2017-02-06T19:14:26.724-0600] [LWP:20631] DEBUG qdisp.Executive (core/modules/qdisp/Executive.cc:225) - Executive::markCompleted failed to find tracked QI=1943:33; size=1077
|
[2017-02-06T19:14:26.748-0600] [LWP:20631] ERROR lsst.qserv (core/modules/global/Bug.cc:49) - Bug::Bug:Executive::markCompleted failed to find tracked QI=1943:33; size=1077
|
what(): Executive::markCompleted failed to find tracked QI=1943:33; size=1077
|
Attachments
Activity
The job retry logic appears sound as long as _reapRequesters is removed. Jobs should only be removed from the _incompleteJobs map by calls to markCompleted. markCompleted is only called if the job succeed or all retries failed.
John Gates
added a comment - The job retry logic appears sound as long as _reapRequesters is removed. Jobs should only be removed from the _incompleteJobs map by calls to markCompleted. markCompleted is only called if the job succeed or all retries failed. I've reviewed the code, and the changes make a sense to me.
The error( indicated by InfileMerger::merge(..., hasErMsg=1) ) caused _reapRequester to run, which removed the entry for QI=1943:33; from the _incompleteJobs map. When markCompleted could not locate the job in _incompleteJobs map, it threw a Bug exception.
It looks like _reapRequester may be unnecessary, but there could be problems with the job retry logic.