-
Notifications
You must be signed in to change notification settings - Fork 2
Description
The problem
It turns out that logging with multiprocessing is non-trivial if you're using processes rather than threads.
Definition of "done"
- Test that our logs come through, even when using multiprocessing
- Test that the above is true on windows too
- Ideally, also ensure that tests don't hang when using forking too (Test using multiprocessing only hangs when ran with test suite, works alone pytest-dev/pytest#11174 suggests that it's probably a problem in our code)
Additional context
Looking at the loguru tests, e.g. https://github.com/znichollscr/loguru/blob/61963240dece1979961ec43840e06a1f396479c4/tests/test_multiprocessing.py#L449 and https://github.com/znichollscr/loguru/blob/61963240dece1979961ec43840e06a1f396479c4/tests/test_multiprocessing.py#L476, it seems like the only way to support this on windows is to use spawn multiprocessing (windows doesn't support forking). I think that also means that we would need to support passing the logger in as an (optional) argument all over our API. That would be ok, but is definitely not ideal.
If we can see a different solution, that would be great. It might also be fine to just not support proper parallel logging on windows (this tool will mainly be used on linux systems anyway).