Uploaded image for project: 'Data Management'
  1. Data Management
  2. DM-36360

Make QuantumGraph-building diagnostics more prominent

    XMLWordPrintable

    Details

      Description

      The warnings printed prior to the exception traceback when a QuantumGraph is empty are often missed by users reporting problems.  Make them more prominent by upgrading the log level to "fatal" and squashing the traceback.

       

        Attachments

          Activity

          Hide
          jbosch Jim Bosch added a comment -

          New output looks like this (obtained manually by attempting a bad QG against the ci_hsc repo; the layering of the CLI stuff in ctrl_mpexec makes it hard to get the big picture from the unit tests):

          lsst.pipe.base.graphBuilder CRITICAL: Initial data ID query returned no rows, so QuantumGraph will be empty.
          lsst.pipe.base.graphBuilder CRITICAL: To reproduce this query for debugging purposes, run Registry.queryDataIds with these arguments:
          lsst.pipe.base.graphBuilder CRITICAL:   dimensions=['band', 'instrument', 'skymap', 'detector', 'physical_filter', 'tract', 'exposure', 'patch', 'visit'],
          lsst.pipe.base.graphBuilder CRITICAL:   dataId={'instrument': 'HSC'},
          lsst.pipe.base.graphBuilder CRITICAL:   where="skymap='discrete/ci_hsc' AND tract=1 AND patch=69",
          lsst.pipe.base.graphBuilder CRITICAL:   datasets=['raw', 'skyMap', 'jointcalPhotoCalibCatalog', 'jointcalSkyWcsCatalog'],
          lsst.pipe.base.graphBuilder CRITICAL:   collections=['HSC/raw/all', 'HSC/external', 'refcats', 'HSC/calib', 'skymaps', 'HSC/masks'],
          Error: QuantumGraph was empty; diagnostic logging above may have details. 

          What's new here:

          • First line that provides context to the actual diagnostics that follow.
          • log level is CRITICAL, rather than WARNING
          • Final line used to be a RuntimeError with a long traceback, obscuring the important logs.

          In particular, the middle is unchanged; this an example of a case where the diagnostics can't actually tell you the problem, so they just give you a way to minimally reproduce it.

          Show
          jbosch Jim Bosch added a comment - New output looks like this (obtained manually by attempting a bad QG against the ci_hsc repo; the layering of the CLI stuff in ctrl_mpexec makes it hard to get the big picture from the unit tests): lsst.pipe.base.graphBuilder CRITICAL: Initial data ID query returned no rows, so QuantumGraph will be empty. lsst.pipe.base.graphBuilder CRITICAL: To reproduce this query for debugging purposes, run Registry.queryDataIds with these arguments: lsst.pipe.base.graphBuilder CRITICAL: dimensions=[ 'band' , 'instrument' , 'skymap' , 'detector' , 'physical_filter' , 'tract' , 'exposure' , 'patch' , 'visit' ], lsst.pipe.base.graphBuilder CRITICAL: dataId={ 'instrument' : 'HSC' }, lsst.pipe.base.graphBuilder CRITICAL: where= "skymap='discrete/ci_hsc' AND tract=1 AND patch=69" , lsst.pipe.base.graphBuilder CRITICAL: datasets=[ 'raw' , 'skyMap' , 'jointcalPhotoCalibCatalog' , 'jointcalSkyWcsCatalog' ], lsst.pipe.base.graphBuilder CRITICAL: collections=[ 'HSC/raw/all' , 'HSC/external' , 'refcats' , 'HSC/calib' , 'skymaps' , 'HSC/masks' ], Error: QuantumGraph was empty; diagnostic logging above may have details. What's new here: First line that provides context to the actual diagnostics that follow. log level is CRITICAL, rather than WARNING Final line used to be a RuntimeError with a long traceback, obscuring the important logs. In particular, the middle is unchanged; this an example of a case where the diagnostics can't actually tell you the problem, so they just give you a way to minimally reproduce it.
          Hide
          jbosch Jim Bosch added a comment -

          John Parejko, this is something you requested a while back.  Changes are split across pipe_base and ctrl_mpexec, and an example of the new output is in a comment above.

          In both packages there's an extra unrelated commit to deal with some typing issues that popped up due to a new MyPy appearing on pypi since the last merge to main.

           

          Show
          jbosch Jim Bosch added a comment - John Parejko , this is something you requested a while back.  Changes are split across  pipe_base and ctrl_mpexec , and an example of the new output is in a comment above. In both packages there's an extra unrelated commit to deal with some typing issues that popped up due to a new MyPy appearing on pypi since the last merge to main.  
          Hide
          Parejkoj John Parejko added a comment -

          I don't remember asking for this, but I like it!

          Show
          Parejkoj John Parejko added a comment - I don't remember asking for this, but I like it!

            People

            Assignee:
            jbosch Jim Bosch
            Reporter:
            jbosch Jim Bosch
            Reviewers:
            John Parejko
            Watchers:
            Jim Bosch, John Parejko, Tim Jenness
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Jenkins

                No builds found.