My test coverage shows different result local vs github

[SebastianStehle]

Hi,

I have integrated vitest with code coverage into my pipeline: https://github.com/mydraft-cc/ui/actions/runs/6603816462/job/17937329518

And I get very different results in github (linux) vs local (windows).

See the following file: ordering.ts:

In Github:

  ordering.ts      |   36.36 |    28.57 |    12.5 |   33.33 | 21,26-52

Local:

 ordering.ts               |   95.45 |    85.71 |     100 |   95.23 | 52

What I tried:

1. Delete node_modules locally
2. Delete package-lock.json locally.
3. Check all dependencies.

The node versions are almost identical:

Local: v18.17.0
Github: v18.18.2

[AriPerkkio]

Does it differ locally if you use CI=true vitest run?

Would be great if you could reduce this into a minimal reproduction. Looking into a complete project is a lot.

[SebastianStehle]

Same results with this environment variable. I have no clue what the root issue could be. I was having trouble to run anything that would import react. But I have eliminated all the calls to that in my tests and other files with similar imports work fine as well.

[AriPerkkio]

It might be worth to see if Windows CI would have different coverage too. It would narrow down the root cause to OS at least.

Does the coverage differ locally if you run it on Linux or Mac?

[SebastianStehle]

I don't have a mac available, but I could run it under Linux. On github the coverage seems to be the same under all operation systems.

[SebastianStehle]

The difference happens between the first and the next runs. On the first run it does produce a wrong coverage:

See here: https://github.com/mydraft-cc/ui/actions/runs/6613019183

[moritz-baecker-integra]

In my case it was a configuration problem:

• I generated vitest coverage
• I generated cypress component coverage
• Use SonarQube to combine reports

What i did not know was that cypress and vitest export the lcov.info file to:
coverage/lcov.info

the .nyc_output/lcov.info file from cypress / nyc is only a temporary file and not the final lcov report.

I solved this by cofiguring cypress with a .nycrc file:
{ "nyc": { "reporter": [ "lcov", "text" ], "report-dir": "coverage/cypress" } }

and vitest with its configuration parameter:
coverage: {reportsDirectory: 'coverage/vitest'}