AttributeError: type object 'Path' has no attribute '_flavour' - error seen when using Click 8

[adamtheturtle]

Describe the bug

In Click 8, a new feature was introduced, enabling users to use pathlib.Path as
a type for arguments.

This indirectly uses the _flavour attribute of Path.
This errors when I use PyFakeFS.

I get the following error:

Traceback (most recent call last):
  File "/Users/adam/Desktop/click_example/example.py", line 27, in <module>
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 1134, in __call__
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 1058, in main
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 927, in make_context
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 1376, in parse_args
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 2352, in handle_parse_result
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 2308, in process_value
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/core.py", line 2295, in type_cast_value
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/types.py", line 75, in __call__
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/types.py", line 891, in convert
  File "/Users/adam/.virtualenvs/click-example/lib/python3.9/site-packages/click/types.py", line 826, in coerce_path_result
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1073, in __new__
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 698, in _from_parts
  File "/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 691, in _parse_args
AttributeError: type object 'Path' has no attribute '_flavour'

I believe that this may be related to https://bugs.python.org/issue24132.

How To Reproduce

Realistic use case

# Requires
# pip install pyfakefs==4.4.0 click==8.0.0

import click
import pathlib
from pyfakefs.fake_filesystem_unittest import Patcher

@click.command()
@click.argument('foo', type=click.Path(path_type=pathlib.Path))
def hello(foo):
    pass

_patcher = Patcher()
_patcher.setUp()
hello(['/'])
_patcher.tearDown()

Narrow use case

import pathlib

import click
from pyfakefs.fake_filesystem_unittest import Patcher

patcher = Patcher()
click_path = click.Path(path_type=pathlib.Path)
patcher.setUp()
click_path.convert(value='/', param=None, ctx=None)
patcher.tearDown()

Your environment

Python 3.9.5 (default, May  3 2021, 19:12:05) 
[Clang 12.0.5 (clang-1205.0.22.9)]
pyfakefs 4.4.0

[mrbean-bremen]

Thanks for the report! I had a first look, and I'm not sure yet what to do in this case.

As pathlib.Path is read in the decorator at load time, it will not be patched, as it is already cached at patch time. Usually, reloading the module in question helps to fix that, but in this case, it doesn't - the click module still holds the unpatched type somewhere. click does a bit of magic here by declaring it's own Path class to be of type pathlib.Path, which may be a problem for pyfakefs.

I will have a closer look, but cannot promise anything yet.

[adamtheturtle]

@mrbean-bremen - Thanks so much for the quick response! I really appreciate the project and your work on it. Workarounds / anything like that are welcome if any come to mind.

[mrbean-bremen]

Sorry, didn't get anywhere - I need to have a look at this someday with a fresh eye.

[adamtheturtle]

No need to be sorry!

[mrbean-bremen]

Ok, now with a fresh brain I can see that I obviously did a mistake last time I checked this. Sorry it took so long, looks like I needed the weekend to clear my head...

The problem is, as stated above, that the decorator is evaluated at load time, before the patcher had a chance to patch it. The only way to work around this is to reload the module during the test, so that the decorator will be re-evaluated. This assumes that you have the sut and the test in different modules. Here is a version that worked for me:

pyfakefs_tests/pathlib_click.py

import pathlib
import click

@click.command()
@click.argument('foo', type=click.Path(path_type=pathlib.Path))
def hello(foo):
    print(foo)

pyfakefs_tests/test_pathlib_click.py

from pyfakefs.fake_filesystem_unittest import Patcher

from pyfakefs_tests import pathlib_click

with Patcher(modules_to_reload=[pathlib_click]):
    pathlib_click.hello(['/'])

Note that in this case it is essential not to do from pyfakefs_tests.pathlib_clock import hello, as in this case you would have the unpatched hello object cached in the test itself. You could import it locally in the test, though:

from pyfakefs.fake_filesystem_unittest import Patcher
from pyfakefs_tests import pathlib_click

with Patcher(modules_to_reload=[pathlib_click]):
    from pathlib_click import hello
    hello(['/'])

Note that these problems are not specific to pyfakefs, but a general problem with patching using unittest or pytest.

[adamtheturtle]

@mrbean-bremen - No need to apologise, and thank you for your time and thought. I'm happy to work around this and I appreciate your work. I'll leave the choice of whether to close this to you.

[mrbean-bremen]

If the repo maintainer enables discussions, I will convert this into a Q/A discussion issue, as it may help others.