Please change default tests coverage argument to --cov or allow users to configure it

[xymy]

Currently this extension will add --cov=. by default when tests with coverage. According to pytest-cov's docs:

If you use the --cov=something option (with a value) then coverage’s source option will also get overridden. If you have multiple sources it might be easier to set those in .coveragerc and always use --cov (without a value) instead of having a long command line with --cov=pkg1 --cov=pkg2 --cov=pkg3 ....

so this will override my coverage config file and lead to include unwanted files in my coverage report.

To solve it, this extension's docs guide users to configure python.testing.pytestArgs:

When the pytest argument --cov exists in python.testing.pytestArgs, the Python extension will make no additional edits to coverage args, to allow your customizations to take effect. If there is no --cov argument found, the extension will add --cov=. to the pytest args prior to run to enable coverage at the workspace root.

In my opinion, this is not a good behavior. Beacause python.testing.pytestArgs doesn't distinguish tests and tests with coverage. Since vscode give us separate buttons for them, it's better to give separate pytest arguments too.

I think --cov maybe a better default behavior, and just guide users to configure coverage for file discovery.

A new option likes python.testing.pytestCovArgs is a good idea to allow more customize coverage behavior.

[skilkis]

@eleanorjboyd I would appreciate this feature as well. The coverage feature of VS Code is awesome, and I believe adding python.testing.pytestCovArgs or the like as suggested by @xymy would increase its value.

At the moment, for anyone looking for a workaround one can add an explicit "omit" to their config such as in pyproject.toml:

[tool.coverage.run]
source = ["src/"]
# Prevent VS-Code's built-in coverage runner from listing
# unwanted sources in coverage results
omit = ['tests\**\*.py', 'examples\**\*.py', 'docs\**\*.py']

[eleanorjboyd]

Hello! Thanks so much for the detailed info and report. I understand your reasoning and I think both make sense (the default is bad and there needs to be more configurability). My worry with switching the default from "--cov=.to--cov` means the default experience wont work out-of-the-box which is a downside for beginners with coverage. We had been thinking about a more extensive overhaul for custom testing arguments. Give that a like to bump it but right now it is stuck in proposal and trying to find time to work on it. Let me consider a interim solution and Ill get back to you