pytest caplog is incompatible with logging.config.dictConfig()

[thebuzzstop]

Overview

• I have been going around in circles on this problem for several days now and I am no closer to a solution than when I started.
• I have reviewed all of the pytest caplog documentation and github and stackoverflow entries dealing with the pytest caplog fixture and I have narrowed my problem down to the use of logging.config.dictConfig()
• I have tried multiple configurations, with and without propagate=True, and they all result in the same problem ... logging is not captured when using dictConfig().
• None of the pytest caplog examples that I have found use anything other than the root logger and they do not use the logging.config.dictConfig()
• All this leads me to believe that pytest logging is incompatible with config.dictConfig()

Observations

• pytest caplog functions as expected if I simply use the root logger with no call to config.dictConfig().
• pytest caplog does not capture logging when I use config.dictConfig() to configure loggers for multiple loggers.

Request

• I would really like to either have pytest honor config.dictConfig() settings ... or ...
• have some kind of easily implemented caplog test fixture wrapper that I could use in my pytest test cases.
• If this is a known issue then I think the documentation should be updated to reflect the limitation.

Sample code

• My sample test code can be found in my pytest_caplog github repository.
• The repository is public
• It's not a comprehensive test in that it doesn't do any meaningful asserts
• But it does demonstrate the issue that the caplog test fixture is not capturing any logging output when using logging.config.dictConfig()

Versions

• Ubuntu 20.04.5 LTS
• pytest 7.1.3

pip list

alabaster 0.7.12
attrs 22.1.0
Babel 2.10.3
certifi 2022.9.24
charset-normalizer 2.1.1
docutils 0.19
idna 3.4
imagesize 1.4.1
importlib-metadata 5.0.0
iniconfig 1.1.1
Jinja2 3.1.2
MarkupSafe 2.1.1
mock 4.0.3
packaging 21.3
pip 22.2.2
pluggy 1.0.0
py 1.11.0
Pygments 2.13.0
pyparsing 3.0.9
pytest 7.1.3
pytz 2022.4
requests 2.28.1
setuptools 60.2.0
snowballstemmer 2.2.0
Sphinx 5.2.3
sphinxcontrib-applehelp 1.0.2
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.5
tomli 2.0.1
urllib3 1.26.12
wheel 0.37.1
zipp 3.9.0

• a detailed description of the bug or problem you are having
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

[erik-hasse]

I thought I was going crazy, so I'm really glad I'm not the only one experiencing this! FWIW, this is related to a change in Python 3.9. I can run the exact same tests with the exact same requirements.txt on 3.8 and it passes. But in 3.9 and up it fails.

I'm surprised this isn't a more common issue.

[erik-hasse]

Solved it! The issue is that we had a logger called "root" in the config dictionary. Looking at your example you do too. In 3.8 and earlier, calling logger.getLogger("root") returned an otherwise ordinary logger named "root". But in 3.9 this change was introduced:

The logging.getLogger() API now returns the root logger when passed the name 'root', whereas previously it returned a non-root logger named 'root'. This could affect cases where user code explicitly wants a non-root logger named 'root', or instantiates a logger using logging.getLogger(name) in some top-level module called 'root.py'.

So with this change, caplog is adding its handler to the root logger, but then when you call dictConfig during the test it gets the root logger and replaces it with the contents of your configuration, overwriting pytest's caplog.

For us there was no reason that we needed a logger named "root" in the config, so we renamed it and everything passes. But I suspect that there's a way you could configure the true root logger and also have it work with pytest using some combination of the root, incremental, and disable_existing_loggers options.

[nicoddemus]

Awesome finding @erik-hasse, thanks!

I will be closing this then.

If somebody wants to contribute a note to the docs, it would be greatly appreciated. 🙇

[nicoddemus]

I will change this into a discussion, so we can mark @erik-hasse as "answer" for others with the same problem to find it more easily. 👍