8000 fix(rule-tester): use cwd option to set base path for tests with file name by reduckted · Pull Request #10201 · typescript-eslint/typescript-eslint · GitHub
[go: up one dir, main page]

Skip to content

fix(rule-tester): use cwd option to set base path for tests with file name #10201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 28, 2024

Conversation

reduckted
Copy link
Contributor
@reduckted reduckted commented Oct 23, 2024

PR Checklist

Overview

If the filename specified in a test is absolute or steps up at least one directory, then the Linter's cwd option is used to specify the base path.

When creating a FlatConfigArray, ESLint uses the linter's cwd option to specify the base path:
https://github.com/eslint/eslint/blob/725962731538eaa38d5d78b9e82ce3fccc9762d0/lib/linter/linter.js#L1524

The cwd option is specified when creating the Linter object:
https://github.com/eslint/eslint/blob/3a4eaf921543b1cd5d1df4ea9dec02fab396af2a/lib/linter/linter.js#L1304

That means the RuleTester may need to create multiple Linter objects. I've created a Map of Linter objects keyed by their base path.

For absolute paths and paths that step up a directory, the base path is calculated by resolving the file name relative to either the tsconfigRootDir specified in the parser options, or process.cwd(). The root of that path is used as the base path. This is equivalent to what #10174 was fixing. For any other paths, the filename does not affect the base path, meaning the behavior prior to #10174 is used.

Warning

This bug only exists after ESLint v9.5 (see #10191 (comment)).

I have tested these changes locally against ESLint v9.13.0, but haven't updated the version in this PR. I'm not sure what the desired solution is in regards to testing against multiple ESLint versions.

@typescript-eslint
Copy link
Contributor

Thanks for the PR, @reduckted!

typescript-eslint is a 100% community driven project, and we are incredibly grateful that you are contributing to that community.

The core maintainers work on this in their personal time, so please understand that it may not be possible for them to review your work immediately.

Thanks again!


🙏 Please, if you or your company is finding typescript-eslint valuable, help us sustain the project by sponsoring it transparently on https://opencollective.com/typescript-eslint.

Copy link
netlify bot commented Oct 23, 2024

Deploy Preview for typescript-eslint ready!

Name Link
🔨 Latest commit 1e47a9e
🔍 Latest deploy log https://app.netlify.com/sites/typescript-eslint/deploys/671f7880d489ee0008e9f9db
😎 Deploy Preview https://deploy-preview-10201--typescript-eslint.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
1 paths audited
Performance: 99 (🟢 up 5 from production)
Accessibility: 100 (no change from production)
Best Practices: 92 (no change from production)
SEO: 98 (no change from production)
PWA: 80 (no change from production)
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link
nx-cloud bot commented Oct 23, 2024

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 1e47a9e. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution


✅ Successfully ran 2 targets

Sent with 💌 from NxCloud.

@reduckted reduckted changed the title fix(rule-tester) use cwd option to set base path for tests with file name fix(rule-tester): use cwd option to set base path for tests with file name Oct 23, 2024
Copy link
codecov bot commented Oct 23, 2024

Codecov Report

Attention: Patch coverage is 95.83333% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.50%. Comparing base (79c27a8) to head (1e47a9e).
Report is 12 commits behind head on main.

Files with missing lines Patch % Lines
packages/rule-tester/src/RuleTester.ts 95.83% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10201      +/-   ##
==========================================
+ Coverage   86.18%   86.50%   +0.32%     
==========================================
  Files         430      430              
  Lines       15029    15088      +59     
  Branches     4360     4380      +20     
==========================================
+ Hits        12952    13052     +100     
+ Misses       1725     1679      -46     
- Partials      352      357       +5     
Flag Coverage Δ
unittest 86.50% <95.83%> (+0.32%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/rule-tester/src/RuleTester.ts 73.67% <95.83%> (+9.96%) ⬆️

... and 6 files with indirect coverage changes

@reduckted 8000
Copy link
Contributor Author

Is the code coverage sufficient to get this merged? I feel like what it's reporting is wrong because it says that this statement is uncovered:

       basePath = path.parse(
         path.resolve(basePath ?? process.cwd(), filename),
       ).root;

But that statement is definitely being hit because the tests would fail if it wasn't. 😕

@jeremybanka
Copy link
jeremybanka commented Oct 24, 2024

Is the code coverage sufficient to get this merged? I feel like what it's reporting is wrong because it says that this statement is uncovered:

       basePath = path.parse(
         path.resolve(basePath ?? process.cwd(), filename),
       ).root;

But that statement is definitely being hit because the tests would fail if it wasn't. 😕

Hey thank you for taking a look at this. You're right, this line should be covered. But it's not being run at all somehow. It seems to me that the reason that it's not is this one line in filename.test.ts:

import { RuleTester } from '@typescript-eslint/rule-tester';

This appears to resolve to a built asset in the dist/ folder and should be replaced with

import { RuleTester } from '../src/RuleTester';

That's what RuleTester.test.ts does anyway.

See if this fixes the missing coverage issue!

@reduckted
Copy link
Contributor Author

This appears to resolve to a built asset in the dist/ folder and should be replaced with

Ah, brilliant! That's exactly what the problem was. Thanks!

@lotmek
Copy link
Contributor
lotmek commented Oct 26, 2024

Thank you @reduckted for this amazing fix !

I'm the one who introduced that regression with my changes and I sincerely apologize for any inconvenience it caused... 😔

@kirkwaiblinger
Copy link
Member

Warning

This bug only exists after ESLint v9.5 (see #10191 (comment)).

I have tested these changes locally against ESLint v9.13.0, but haven't updated the version in this PR. I'm not sure what the desired solution is in regards to testing against multiple ESLint versions.

I think for now, let's just go with "I tested it on my machine and it worked" (infamous last words) rather than upgrade eslint for this PR. Related, #1752.

kirkwaiblinger
kirkwaiblinger previously approved these changes Oct 26, 2024
Copy link
Member
@kirkwaiblinger kirkwaiblinger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for sending this in!!

@kirkwaiblinger kirkwaiblinger added the 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge label Oct 26, 2024
@kirkwaiblinger kirkwaiblinger requested a review from a team October 26, 2024 21:51
@github-actions github-actions bot removed the 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge label Oct 26, 2024
@kirkwaiblinger kirkwaiblinger added the 1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge label Oct 26, 2024
Copy link
Member
@bradzacher bradzacher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly LGTM -- let's just get the test coverage up for that one extra undefined branch

errors: [
{
messageId: 'foo',
suggestions: [{ messageId: 'createError', output: '//' }],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a syntax error, right? Cos a line comment is valid. We need something invalid like ( that will truly crash out the parser.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bradzacher bradzacher merged commit bda3444 into typescript-eslint:main Oct 28, 2024
62 checks passed
@bradzacher
Copy link
Member

Thanks for your work here!

@jtbandes
Copy link
Contributor

Awesome! Any chance of a patch release for this? :)

@Zamiell
Copy link
Contributor
Zamiell commented Oct 28, 2024

@jtbandes Yes, there will be a release, read this page, which explains: https://typescript-eslint.io/users/releases/

@bradzacher
Copy link
Member

I meant to get to this before the release but baby stuff got in the way.
I've done an out-of-band release to get this out.

https://github.com/typescript-eslint/typescript-eslint/releases/tag/v8.12.1

@reduckted reduckted deleted the bugfix/10191 branch October 29, 2024 08:58
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 6, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
1 approval >=1 team member has approved this PR; we're now leaving it open for more reviews before we merge
Projects
None yet
< 46F3 div class="js-discussion-sidebar-item" data-target="create-branch.sidebarContainer">
Development

Successfully merging this pull request may close these issues.

Bug: RuleTester: cannot read properties of undefined (reading 'parse' )
7 participants
0