Run real ESLint in test, baseline output #824
Conversation
|
|
||
| ==== types/no-const-enum/index.d.ts ==== | ||
|
|
||
| enum F {} |
There was a problem hiding this comment.
I snapshotted even the "good" files; the runner doesn't do a great job of removing unused snapshots (at least in this special "file snapshot" addon).
There was a problem hiding this comment.
Unused baselines are now caught via a test, but I think leaving these to make sure we encode the expected state is probably valuable.
| @@ -40,6 +40,9 @@ | |||
| "@types/strip-json-comments", | |||
| "strip-json-comments" | |||
| ] | |||
| }, | |||
| "patchedDependencies": { | |||
| "jest-file-snapshot@0.5.0": "patches/jest-file-snapshot@0.5.0.patch" | |||
There was a problem hiding this comment.
This is due to satya164/jest-file-snapshot#23 (which I only found after I had patched it locally myself).
There was a problem hiding this comment.
It looks like https://github.com/igor-dv/jest-specific-snapshot is more popular, but it does a slightly different thing; it makes a jest-style snapshot file. That's good if you want that format, but in our case I just want some random text files and integration into -u, so jest-file-snapshot works fine.
|
cc @bradzacher, this is beautiful 🥲 We've been interested in this in the typescript-eslint side too. typescript-eslint/typescript-eslint#6499 -> typescript-eslint/typescript-eslint#6994 |
|
Yeah it's hard because doing this in the general case is much harder than a per-repo case. People have certain expectations about automation and on top of that the rule tester is framework-agnostic. One-file-per-case is a really really nice way to do testing of zero-option rules like we've got here - but when you need to test options it quickly balloons out into an unmanageable mess of copy-pasted files that are hard to grep through to find duplicates and such. I still want to solve this from the framework side so that we can provide such tooling to everyone - we just need to do some thinking and design work. |
In the case of this test setup, the fixtures folder is a valid eslint root such that we can just make more directories and modify things via eslintrcs in each. It also means I can open an editor to the side in the directory and see live what's going on (which was handy while fixing the off by ones...) |
|
Yeah sorry I meant as an example this rule's tests. The Now finding duplicates you have to do cross-file searches as opposed to same-file searches (there's a lint rule which reports on duplicate test cases). Changing duplicate cases becomes a multi-file find-and-replace as opposed to a multi-cursor edit. That's not even our biggest rule test suite either! That's what I meant by hard to manage - when you have to manage options and re-testing the same code under different variations things can be a little tricky as the space scales exponentially! I'm not saying this style is bad at all btw - I like it a lot! We use this style as well for our ast fixtures. |
This moves all of our eslint tests into the "fixtures" folder. In the test, it runs eslint on each file, baselining the output in a human-readable format with error ranges and so on (like the TS error deltas).
This exposed some incorrect error ranges (and probably more; I need to skim the changes again).