# Mismatching dataId in logger output

XMLWordPrintable

## Details

• Type: Bug
• Status: Done
• Resolution: Done
• Fix Version/s: None
• Component/s:
• Labels:
None
• Story Points:
1
• Sprint:
DB_S17_6
• Team:
Data Access and Database

## Description

When running processCcd.py on multiple visits, self-contradictory outputs are generated by the logger. For example:

 INFO 2017-05-25T00:07:04.004 processCcd (DataId(initialdata={'visit': 242618039}, tag=set([])))(processCcd.py:181)- Processing DataId(initialdata={'visit': 242619360}, tag=set([])) 

and

 FATAL 2017-05-25T00:07:04.004 processCcd (DataId(initialdata={'visit': 242713680}, tag=set([])))(cmdLineTask.py:351)- Failed on dataId=DataId(initialdata={'visit': 242715702}, tag=set([])): Unable to match sources 

The problem above is that the visit numbers don't match, but there should be no way for this to ever be the case, as far as I can see. FWIW, the second number is the true visit number, at least in my case.

This is only noticeable when --longlog is used, to allow disentangling of parallel processed data, but I don't know whether that means the logger is also wrong when sequentially running multiple visits on a single core (I suspect it's OK in that instance, but I'm not certain).

This bug is bad for tracking down which visits are failing and why.

## Activity

Hide
Merlin Fisher-Levine added a comment -

Other than the one comment I made this looks fine. I haven't tested the fix as that requires rebuilding a whole stack, but assuming it fixes the problem then this is good to merge

Show
Merlin Fisher-Levine added a comment - Other than the one comment I made this looks fine. I haven't tested the fix as that requires rebuilding a whole stack, but assuming it fixes the problem then this is good to merge
Hide
Fritz Mueller added a comment -

You could have had five points of vacation out of that one, Andy! (Thanks, Andy)

Show
Fritz Mueller added a comment - You could have had five points of vacation out of that one, Andy! (Thanks, Andy)
Hide
Andy Salnikov added a comment -

Thanks for review! Merged and done with a minimum amount of points while also helping poor HEP guys at CERN

Show
Andy Salnikov added a comment - Thanks for review! Merged and done with a minimum amount of points while also helping poor HEP guys at CERN
Hide
Andy Salnikov added a comment - - edited

One note for Merlin Fisher-Levine - testing does not require rebuilding whole stack, patches are trivial and binary compatible (in case of log). You can just clone log and pipe_base (latter is optional), build them locally, do setup -k -r ... and it will work. I tested it that way to make sure that it actually fixes output.

Show
Andy Salnikov added a comment - - edited One note for Merlin Fisher-Levine - testing does not require rebuilding whole stack, patches are trivial and binary compatible (in case of log ). You can just clone log and pipe_base (latter is optional), build them locally, do setup -k -r ... and it will work. I tested it that way to make sure that it actually fixes output.
Hide
Merlin Fisher-Levine added a comment -

Oh, that's good to know, thanks. I never touch the C++ layer, so thought that anything like that automatically required a rebuild from there on up (down?). I will do that soon then, just so that I can have the bug-fix (though presumably this will go into the next weekly, so I might not bother given that it's already merged...)

Show
Merlin Fisher-Levine added a comment - Oh, that's good to know, thanks. I never touch the C++ layer, so thought that anything like that automatically required a rebuild from there on up (down?). I will do that soon then, just so that I can have the bug-fix (though presumably this will go into the next weekly, so I might not bother given that it's already merged...)

## People

• Assignee:
Andy Salnikov
Reporter:
Merlin Fisher-Levine
Reviewers:
Merlin Fisher-Levine
Watchers:
Andy Salnikov, Fritz Mueller, Hsin-Fang Chiang, John Swinbank, Kian-Tat Lim, Merlin Fisher-Levine, Paul Price, Robert Lupton