`pytest-cov` emits hundreds of megabytes of `AstArcAnalyzer`

[imnotjames]

Struggling to create a minimal implementation of this, but it seems that when we're using pytest-cov, mocking out os, and then have an internal pytest error we have hundreds of megabytes of AST data printed out because os.getenv("COVERAGE_TRACK_ARCS") gets accidentally set to a truthy value.

Working on a minimal reproduction..

[imnotjames]

Where the debug flag gets set:

https://github.com/nedbat/coveragepy/blob/9a36d3350791a17da1459f25dd338da62e303048/coverage/parser.py#L679

Where the debug logs get emitted:

https://github.com/nedbat/coveragepy/blob/9a36d3350791a17da1459f25dd338da62e303048/coverage/parser.py#L756-L758

[imnotjames]

Okay - found a bit more info as to what's happening.. sorta.

Using pytest-mock, we mock out os.

However, our tests failed via an internal error, a StopIteration in a mock for some reason which was not handled by pytest correctly. This leads to the test bailing out immediately without cleaning up any fixtures. Things like pytest-mock don't run and leave the mock in place on os.

I'm guessing that because this runs as a pytest plugin that it runs in the same global scope as tests so when coverage.py is called we end up leaving the mock in place.

Yikes, that's a messy problem to try to unravel. Part of this feels more like a problem in pytest-mock? But I'm not sure if there's a way to avoid weird state issues like this from causing coverage.py to run in such a misconfigured way..

[RonnyPfannschmidt]

I'm working on a fix for stopiteration

A workaround is to use pytest before modernized hookwrappers to debug

When modernizing hookwrappers in pluggy we missed a edge case

[nedbat]

Try adding this in coverage/parser.py, after the imports:

from coverage.misc import isolate_module

os = isolate_module(os)

[nedbat]

I'm adding those lines to coverage.py....

[nedbat]

This is now released as part of coverage 7.6.5.

[imnotjames]

Awesome! Reading a bit between the lines, I think this is not really a problem with pytest-cov but instead the modernized hookwrappers leaving an incomplete state. The coverage.py change also helps avoid a pretty big bunch of debug logging while that failure state occurs.

A thought I had on this -- I don't know if there's a way for pytest-cov to ALWAYS disable debug mode in coverage.py, avoiding the env vars directly.. but that would probably be nice & would also help prevent this case?

I haven't had the time to look into much of this myself; I didn't expect y'all to be so quick to dig into it! You all rock.

[imnotjames]

I'll note - I was able to work around this by using coverage run -m pytest and it no longer exhibited the issues described above. I'm guessing because all the coverage.py imports are kicked off earlier in that case so weird module mocks don't impact anything.