Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-10623

Mismatching dataId in logger output

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Fix Version/s: None
    • Component/s: log, pipe_base, pipe_tasks
    • Labels:
      None
    • Templates:
    • Story Points:
      1
    • Sprint:
      DB_S17_6
    • Team:
      Data Access and Database

      Description

      When running processCcd.py on multiple visits, self-contradictory outputs are generated by the logger. For example:

      INFO  2017-05-25T00:07:04.004 processCcd (DataId(initialdata={'visit': 242618039}, tag=set([])))(processCcd.py:181)- Processing DataId(initialdata={'visit': 242619360}, tag=set([]))
      

      and

      FATAL 2017-05-25T00:07:04.004 processCcd (DataId(initialdata={'visit': 242713680}, tag=set([])))(cmdLineTask.py:351)- Failed on dataId=DataId(initialdata={'visit': 242715702}, tag=set([])): Unable to match sources
      

      The problem above is that the visit numbers don't match, but there should be no way for this to ever be the case, as far as I can see. FWIW, the second number is the true visit number, at least in my case.

      This is only noticeable when --longlog is used, to allow disentangling of parallel processed data, but I don't know whether that means the logger is also wrong when sequentially running multiple visits on a single core (I suspect it's OK in that instance, but I'm not certain).

      This bug is bad for tracking down which visits are failing and why.

        Attachments

          Container Issues

            Issue Links

              Activity

                People

                • Assignee:
                  salnikov Andy Salnikov
                  Reporter:
                  mfisherlevine Merlin Fisher-Levine
                  Reviewers:
                  Merlin Fisher-Levine
                  Watchers:
                  Andy Salnikov, Fritz Mueller, Hsin-Fang Chiang, John Swinbank, Kian-Tat Lim, Merlin Fisher-Levine, Paul Price, Robert Lupton
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  8 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Summary Panel