8000 [2.4.0][Watch] Watching causes crash in eslint fix · Issue #1084 · typescript-eslint/typescript-eslint · GitHub
[go: up one dir, main page]
Skip to content

[2.4.0][Watch] Watching causes crash in eslint fix #1084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
kyle7zhang opened this issue Oct 15, 2019 · 6 comments · Fixed by #1106
Closed

[2.4.0][Watch] Watching causes crash in eslint fix #1084

kyle7zhang opened this issue Oct 15, 2019 · 6 comments · Fixed by #1106
Labels
bug Something isn't working package: typescript-estree Issues related to @typescript-eslint/typescript-estree

Comments

@kyle7zhang
Copy link
kyle7zhang commented Oct 15, 2019 •
edited
Loading

Error option line:

https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/typescript-estree/src/tsconfig-parser.ts#L106

What code were you trying to parse?

// Any code

tsconfig.json:

{
    "compileOnSave": true,
    "compilerOptions": {
        "target": "esnext",
        "module": "es6",
        "strict": true,
        "noImplicitAny": false,
        "experimentalDecorators": true,
        "emitDecoratorMetadata": true,
        "allowSyntheticDefaultImports": true,
        "charset": "utf8",
        "allowJs": false,
        "pretty": true,
        "lib": ["es6", "dom"],
        "noEmitOnError": false,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "allowUnreachableCode": false,
        "allowUnusedLabels": false,
        "strictPropertyInitialization": false,
        "noFallthroughCasesInSwitch": true,
        "inlineSourceMap": true,
        "declaration": true,
        "outDir": "./lib",
        "skipLibCheck": true
    },
    "include": [
        "src",
        "types"
    ],
    "exclude": [
        "node_modules",
        "**/__tests__/*"
    ]
}

eslintrc:

module.exports = {
    root: true,
    parserOptions: {
        project: './tsconfig.json',
        // noWatch: true, // if add this option, then eslint --fix xxx.ts succeed
    },
    parser: '@typescript-eslint/parser',
    env: {
        es6: true,
        node: true,
        browser: true,
        jest: true,
    },
    extends: [
        'airbnb-base/legacy',
        'plugin:@typescript-eslint/recommended',
    ],
...

What did you expect to happen?

Can run eslint --fix without memory leak.

What actually happened?

Run command eslint --fix XXX.ts, this command still runs and the memory keeps increasing.

  eslint:cli Fix mode enabled - applying fixes +1s

<--- Last few GCs --->

[90902:0x108000000]    64687 ms: Scavenge 1317.6 (1433.7) -> 1311.2 (1438.7) MB, 7.5 / 0.0 ms  (average mu = 0.206, current mu = 0.058) allocation failure 
[90902:0x108000000]    64728 ms: Scavenge 1322.2 (1438.7) -> 1315.9 (1444.7) MB, 9.4 / 0.0 ms  (average mu = 0.206, current mu = 0.058) allocation failure 
[90902:0x108000000]    65315 ms: Mark-sweep 1326.9 (1444.7) -> 1310.7 (1443.7) MB, 559.0 / 2.0 ms  (average mu = 0.213, current mu = 0.222) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x28b71165be3d]
Security context: 0x3cd6c629e6e9 <JSObject>
    1: new Readable [0x3cd62a766339] [_stream_readable.js:~145] [pc=0x28b7128f65ef](this=0x3cd65eff6241 <Readable map = 0x3cd6d357dc49>,options=0x3cd65eff6219 <Object map = 0x3cd6d357db99>)
    2: ConstructFrame [pc: 0x28b71160d02f]
    3: StubFrame [pc: 0x28b71195bd3d]
    4: new ReaddirpStream(aka ReaddirpStream) [0x3cd6e0816fb9] [/Users/icese7en/Developer...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x10003c597 node::Abort() [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 2: 0x10003c7a1 node::OnFatalError(char const*, char const*) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 3: 0x1001ad575 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 4: 0x100579242 v8::internal::Heap::FatalProcessOutOfMemory(char const*) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 5: 0x10057bd15 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 6: 0x100577bbf v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 7: 0x100575d94 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 8: 0x10058262c v8::internal::Heap::AllocateRawWithLigthRetry(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
 9: 0x1005826af v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
10: 0x100551ff4 v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
11: 0x1007da044 v8::internal::Runtime_AllocateInNewSpace(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/icese7en/.nvm/versions/node/v10.15.3/bin/node]
12: 0x28b71165be3d 
[1]    90902 abort      node ./node_modules/.bin/eslint --fix ./src/messageApi/index.ts --debug

Then add noWatch, the problem solved.

Versions

package version
@typescript-eslint/typescript-estree 2.4.0
TypeScript 3.6.4
node 10.15.3
npm 6.12.0
@kyle7zhang kyle7zhang added package: typescript-estree Issues related to @typescript-eslint/typescript-estree triage Waiting for team members to take a look labels Oct 15, 2019
@JounQin
Copy link
Contributor
JounQin commented Oct 15, 2019

Maybe related: Error: ENOSPC: System limit for number of file watchers reached

https://travis-ci.com/1stG/configs/builds/131991828#L596

@JounQin
Copy link
Contributor
JounQin commented Oct 15, 2019

node_modules should be ignored by default, or an option is required at least.

@kyle7zhang
Copy link
Author

node_modules should be ignored by default, or an option is required at least.

Actually, files in other directories have also been watched.

Running directory: /Users/A/git/hello/world
But other directories files: /Users/A/git/sample/XXX has also been watched.

@bradzacher bradzacher added bug Something isn't working and removed triage Waiting for team members to take a look labels Oct 15, 2019
@bradzacher
Copy link
Member

you can also use the environment variable PARSER_NO_WATCH=true to disable file watching whilst we look into this.

node_modules should be ignored by default, or an option is required at least.

Yes and no.

We watch all folders that typescript asks us to. And typescript asks for everything it thinks it needs to watch to keep its internal type model correct.

TS will not ask us to watch much, if any of node_modules.
It will however, ask us to watch folders such as node_modules/@types and its descendants, because it needs to know about changes to those folders so it can update its internal model.
In the same vein, it knows which other packages contain type definitions it cares for, so it will ask us to watch those package folders as well (i.e. our package typescript-estree has type definitions, so typescript will want to watch that folder).

If you're using a tool like lerna to symlink between packages within your project, then typescript will follow the symlinks in node_modules to their actual folder (I assume it's because there are bugs that can happen from watching a symlink instead of the actual folder), so you can end up with some weird looking watched directories.

I'll have a look into pruning the watchers more.

Maybe related: Error: ENOSPC: System limit for number of file watchers reached

See: #1080

@bradzacher bradzacher changed the title noWatch cause eslint fix failed [2.4.0][Watch] Watching causes crash in eslint fix Oct 17, 2019
@bradzacher bradzacher mentioned this issue Oct 19, 2019
Merged
@bradzacher
Copy link
Member

I've just landed #1106, and it will be deployed to the canary tag imminently.
This change reworks the invalidation approach so that it doesn't need file watchers at all.

I'd appreciate if you could take a moment to test the canary tag on your codebases, and let me know if anything isn't working as expected for your real-world codebase.

@kyle7zhang
Copy link
Author

I've just landed #1106, and it will be deployed to the canary tag imminently.
This change reworks the invalidation approach so that it doesn't need file watchers at all.

I'd appreciate if you could take a moment to test the canary tag on your codebases, and let me know if anything isn't working as expected for your real-world codebase.

Problem solved, thanks

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
bug Something isn't working package: typescript-estree Issues related to @typescript-eslint/typescript-estree
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat(typescript-estree): support long running lint without watch typescript-eslint/typescript-eslint
3 participants
@kyle7zhang @bradzacher @JounQin
0