Need to filter properties from object diffs

[sailingKieler]

Hi friends of vitest,

I'm working a lot on and with Langium and Vitest has been huge benefit there.

I just ran into a problem that I would like to compare two objects for equality like

  expect(A).toBe(B);

A and B be a part of a rather large spider web of objects, und in case of a mismatch the creation of the diff visualization makes the test execution crash after some time of polluting the memory.

I saw that you have a config/customization hook for sorting the properties of objects, but no opportunity for filtering them. Ignoring a few known properties being prone to causing problems would help me a lot.

vitest/packages/utils/src/diff/index.ts

Lines 186 to 190 in 4e60333

	return {
	...formatOptions,
	compareKeys,
	printBasicPrototype,
	}

Ideally I would love if I could contribute a custom plugin to the list of pretty-format plugins:

vitest/packages/utils/src/diff/index.ts

Lines 37 to 57 in 4e60333

	const {
	AsymmetricMatcher,
	DOMCollection,
	DOMElement,
	Immutable,
	ReactElement,
	ReactTestComponent,
	} = prettyFormatPlugins
	const PLUGINS = [
	ReactTestComponent,
	ReactElement,
	DOMElement,
	DOMCollection,
	Immutable,
	AsymmetricMatcher,
	prettyFormatPlugins.Error,
	]
	const FORMAT_OPTIONS = {
	plugins: PLUGINS,
	}

Realizing that getFormatOptions(...) simply ignores my custom plugin contribution almost made me biting my table 😖

vitest/packages/utils/src/diff/index.ts

Lines 180 to 191 in 4e60333

	function getFormatOptions(
	formatOptions: PrettyFormatOptions,
	options?: DiffOptions,
	): PrettyFormatOptions {
	const { compareKeys, printBasicPrototype } = normalizeDiffOptions(options)
	return {
	...formatOptions,
	compareKeys,
	printBasicPrototype,
	}
	}

What do think about opening something in that corner?

[hi-ogawa]

A and B be a part of a rather large spider web of objects, und in case of a mismatch the creation of the diff visualization makes the test execution crash after some time of polluting the memory.

Do you have reproduction? I've also seen oom crash when equality fails #7009
I was guessing this is a bug of printer and cyclic references, or even if it's genuinely too big, it shouldn't crash and need a way to bail out printing early somehow.

Then if this turns out to be not fixable, perhaps we can consider adding user-land workaround.

[sailingKieler]

We've public examples of Langium-based projects (see also our playground https://langium.org/showcase/), I'll see whether I can provide you something. I also saw your cyclic ref detection and wondered why it didn't cut it (literally), but I didn't go deeper there.

[hi-ogawa]

FYI, I'm making a PR to help some cases #7133

Not sure about your case, but it might also help avoiding a large object diff by implementing toJSON as it's prioritized in some cases.

[bgoscinski]

I think I have similar usecase. I'm testing a Vue application and I had a test that kept OOMing. I reduced the repro to this. The test prints HUGE diff because it traverses the whole reactivity graph where something simpler (such as RefImpl { value: 0 }) would completely suffice. When I increased the CHAIN_LENGTH constant to 100 I gave up waiting for the test result after 4 minutes when the memory consumption reached about 4GB.

I don't expect vitest to come out of the box with pretty printer that somehow knows how to print vue Refs but I'd like to teach it how to do it - some API similar to expect.addSnapshotSerializer would be perfect 🤔

[hi-ogawa]

Currently users can only customize via toJSON method, but I think we can explore other API to customize diff too. Please feel free to raise an issue / feature request.

At the same time, OOM or crash in any case should be avoided, so fixing such worst case is also important though (e.g. automatically truncate by detecting during large diff printing).

---

Maybe, this is the way to patch toJSON currently https://stackblitz.com/edit/vitest-dev-vitest-rh6fqqwt?file=test%2Fsuite.test.js

  computed(() => 0).constructor.prototype.toJSON = function () {
    return { type: this.constructor.name, value: this.value };
  };

[hi-ogawa]

Also there might be several issues with a circular object. Either toEqual/toBe assertion itself is buggy (or too expensive) or diff generation on error is too large. Normally both happens for the same case, but the former is a clear bug and should be fixed #7429.

[bgoscinski]

toJSON seems to work if I compare it to another object but when I expect something of a different type such as number it prints the whole object again: here I'll raise an issue

[bgoscinski]

done #7478