8000 [2.4.0][Watch][Windows] ESLint doesn't exit after completing a cli lint · Issue #1079 · typescript-eslint/typescript-eslint · GitHub
[go: up one dir, main page]

Skip to content

[2.4.0][Watch][Windows] ESLint doesn't exit after completing a cli lint #1079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
doberkofler opened this issue Oct 14, 2019 · 36 comments · Fixed by #1106
Closed

[2.4.0][Watch][Windows] ESLint doesn't exit after completing a cli lint #1079

doberkofler opened this issue Oct 14, 2019 · 36 comments · Fixed by #1106
Labels
bug Something isn't working package: typescript-estree Issues related to @typescript-eslint/typescript-estree

Comments

@doberkofler
Copy link
Contributor
doberkofler commented Oct 14, 2019

Simply upgrading eslint-pluginand parser from 2.3.3 to 2.4.0 causes eslint to hang at the end of the processing.

npx eslint --cache --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"

I've enabled basic debugging DEBUG=eslint:cli-engine and the last message is eslint:cli-engine Linting complete in: 110453ms +219ms and then the eslint command does not seem to return (within several minutes) and I can only kill the process.
I have tried this without any change to the sources to be parsed or the configuration and simply upgrading to 2.4.0 makes eslint hang and after moving back to 2.3.3 it does no longer hang.

(I'm aware that I should upload a reproducible test case but given the nature of this problem this seems not possible right now)

Versions

package version
@typescript-eslint/eslint-plugin 2.4.0
@typescript-eslint/parser 2.4.0
TypeScript 3.6.4
ESLint 6.5.1
node 12.12.0
npm 6.12.0
@doberkofler doberkofler added package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin triage Waiting for team members to take a look labels Oct 14, 2019
@bradzacher bradzacher added awaiting response Issues waiting for a reply from the OP or another party and removed triage Waiting for team members to take a look labels Oct 14, 2019
@bradzacher
Copy link
Member

I am unable to repro this in our repo.
I tried running:

  • eslint . --ext .js,.ts
  • eslint . --ext .js,.ts --cache
  • eslint . --ext .js,.ts --cache --report-unused-disable-directives
  • eslint . --ext .js,.ts --report-unused-disable-directives

Questions:

  • What OS are you running?
  • Does your codebase use symlinks or similar extensively?

as a temp workaround you can turn on the environment variable
PARSER_NO_WATCH=true npx eslint --cache --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"

@bradzacher
Copy link
Member

Could you try adding "extendedDiagnostics": true to your tsconfig(s), running your command again, and inspecting the output of your run?
If you're able to, can you post the output of that here? I understand if you can't because it's a private project, but all the information possible will make solving this easier.

@doberkofler
Copy link
Contributor Author

What OS are you running?
Windows 7 x64 (I just tried on MacOS and there it seems to work also after the update to 2.4.0)

Does your codebase use symlinks or similar extensively?
None

I also just tried to upgrade to node 12.12.0 but the problem persists.

as a temp workaround you can turn on the environment variable
PARSER_NO_WATCH=true npx eslint --cache --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"
Using

set PARSER_NO_WATCH=true
npx eslint --cache --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"

does seem to resolve the problem. Could you elaborate on what this settings does or rather prevents from doing?

@doberkofler
Copy link
Contributor Author

Should I use "extendedDiagnostics": true with or without the PARSER_NO_WATCH=true environment variable?

@doberkofler
Copy link
Contributor Author

This is the output of eslint when running with "extendedDiagnostics": true but without PARSER_NO_WATCH=true: eslint.zip

@bradzacher
Copy link
Member
bradzacher commented Oct 14, 2019

Windows 7 x64

gah. windows. the one platform we don't have any coverage on.
This is more than likely the culprit.
cc @JamesHenry - is it possible to add windows to the CI?

I'll have to dust off my windows laptop and do some investigation.

Could you elaborate on what this settings does or rather prevents from doing?

When you run eslint, typescript will tell us which folders it wants us to watch.
In <2.4.0, we just no-oped that CB, because we thought we didn't care about setting up watchers because eslint would provide us with the most up-to-date file contents.
However this had a drawback with the 2.0.0 change we added to parse a stricter set of files - it meant that when you were in a long-running lint session (i.e. via an IDE), then typescript wouldn't know that a new file should be included in the parse, and we would thus throw errors on valid files.

2.4.0 adds support for weak file watching - it attaches watchers to the folders typescript cares about, as well as the tsconfig file(s). These watchers are set to be non-persistent - i.e. they are lightweight and will not hold the process open.
But it looks like there's something wrong on Windows specifically.
chokidar was supposed to handle the platform differences for me automatically, but I might have missed something in the docs.

The environment variable does what it says on the tin - it turns off file watching within the parser (or more accurately typescript-estree). I put it in because on some large, complex setups (like those with symlinks and such), file watching can have a pretty significant overhead, so you might want to turn it off for a single run.

function watch(
path: string,
options: chokidar.WatchOptions,
extra: Extra,
): Watcher {
// an escape hatch to disable the file watchers as they can take a bit to initialise in some cases
// this also supports an env variable so it's easy to switch on/off from the CLI
if (process.env.PARSER_NO_WATCH === 'true' || extra.noWatch === true) {
return {
close: (): void => {},
on: (): void => {},
};
}
return chokidar.watch(path, {
ignoreInitial: true,
persistent: false,
useFsEvents: false,
...options,
});
}

The problem we have is that eslint has no specific concept of a long running session, because it's intended to be pretty stateless.
This is why the watchers are setup always, because eslint has no mechanism to tell us that it's being used in a long running use case.

Should I use "extendedDiagnostics": true with or without the PARSER_NO_WATCH=true environment variable?

Without, please.

This is the output of eslint when running with "extendedDiagnostics": true but without PARSER_NO_WATCH=true

Thank you! I'll investigate this as soon as I can.

@doberkofler
Copy link
Contributor Author

Thank you for the in-depth explanation all the good work!
Sorry that I quite often (I know) report problem on the Windows platform but this is simply because our CI always runs on multiple MacOS AND Windows versions.

@JamesHenry
Copy link
Member

Should be pretty straightforward to add a Windows agent to the Azure Pipelines build, hopefully we can figure out a minimal repro to turn into a test

@bradzacher bradzacher added bug Something isn't working and removed awaiting response Issues waiting for a reply from the OP or another party labels Oct 14, 2019
@bradzacher bradzacher changed the title eslint hangs after upgrading from 2.3.3 to 2.4.0 [windows] eslint hangs with 2.4.0 file watching Oct 14, 2019
@AviVahl

This comment has been minimized.

@bradzacher

This comment has been minimized.

@bradzacher bradzacher pinned this issue Oct 15, 2019
@bradzacher bradzacher changed the title [windows] eslint hangs with 2.4.0 file watching [2.4.0][Watch][Windows] ESLint doesn't exit after completing a cli lint Oct 15, 2019
@bradzacher bradzacher added package: typescript-estree Issues related to @typescript-eslint/typescript-estree and removed package: eslint-plugin Issues related to @typescript-eslint/eslint-plugin labels Oct 15, 2019
@yoyo930021
Copy link
Contributor

I have a similar problem with macOS.
ESLint exit after completing a cli lint and waiting for a long time.

cpuprofile.zip

@bradzacher bradzacher unpinned this issue Oct 17, 2019
@kalbert312
Copy link
kalbert312 commented Oct 18, 2019

Running into this as well. macOS. Running with --debug produces eslint:cli-engine Linting complete in: 2301ms +6ms but it never exits.

For reference:

.eslintrc.json

{
	"plugins": ["@typescript-eslint"],
	"parser": "@typescript-eslint/parser",
	"parserOptions": {
		"project": "./tsconfig.eslint.json"
	},
	"extends": [
		"eslint:recommended",
		"plugin:@typescript-eslint/eslint-recommended",
		"plugin:@typescript-eslint/recommended",
		"plugin:@typescript-eslint/recommended-requiring-type-checking"
	],
	"env": {
		"amd": true,
		"browser": true,
		"node": true
	},
	"rules": {
		"prefer-const": ["warn", {
			"destructuring": "all"
		}],
		"require-await": "off",
		"@typescript-eslint/ban-ts-ignore": "warn",
		"@typescript-eslint/explicit-function-return-type": "off",
		"@typescript-eslint/interface-name-prefix": [
			"error",
			{
				"prefixWithI": "always",
				"allowUnderscorePrefix": false
			}
		],
		"@typescript-eslint/member-delimiter-style": [
			"error",
			{
				"multiline": {
					"delimiter": "comma",
					"requireLast": true
				},
				"singleline": {
					"delimiter": "comma",
					"requireLast": false
				}
			}
		],
		"@typescript-eslint/no-inferrable-types": "off",
		"@typescript-eslint/require-await": "warn"
	}
}

tsconfig.eslint.json

{
	"extends": "./tsconfig",
	"compilerOptions": {
		"allowJs": true
	},
	"include": [
		"src/**/*",
		"tests/**/*",
		"./*.js",
		"./*.ts"
	]
}

tsconfig.json

{
	"compilerOptions": {
		"target": "es5",
		"module": "commonjs",
		"lib": [
			"dom",
			"dom.iterable",
			"es2015"
		],
		"skipLibCheck": true,
		"sourceMap": true,
		"downlevelIteration": true,
		"strict": true,
		"allowSyntheticDefaultImports": true,
		"esModuleInterop": true,
		"inlineSources": true,
		"rootDir": "src"
	},
	"include": [
		"src/**/*"
	],
	"exclude": [
		"dist",
		"out"
	]
}

eslint 6.5.1
the 2.4.0 version of the eslint typescript packages.
It's in an archetype project so there's only 1 or 2 files to lint.
Reverting to 2.3.3 fixed the issue for me.

@favna
Copy link
favna commented Oct 18, 2019

Can confirm I am also facing this issue on MacOS Catalina (10.15), relevant config and versions are pretty much similar to those kalbert312. Passing PARSER_NO_WATCH=true does work but as a better solution I downgraded back to 2.3.3 for now.

NodeJS version: 12.12.0

favna added a commit to RWS-NL/air-node-packages that referenced this issue Oct 18, 2019
Also locked @typescript-eslint/parser to 2.3.3 until
typescript-eslint/typescript-eslint#1079 is fixed

Signed-off-by: Jeroen Claassens <j.claassens@cgi.com>
@amikheychik
Copy link

Can confirm this issue on MacOS 10.14 and NodeJS 10.16.0. Rollback to 2.3.3 helps.

@bradzacher
Copy link
Member

If you're looking for a more robust way to disable watching, there is also a config you can add to your eslintrc - parserOptions: {noWatch: true}.
You need not rollback to 2.3.3.

@bradzacher
Copy link
Member

I've just landed #1106, and it will be deployed to the canary tag imminently.
This change reworks the invalidation approach so that it doesn't need file watchers at all.

I'd appreciate if you could take a moment to test the canary tag on your codebases, and let me know if anything isn't working as expected for your real-world codebase.

@kbradl16
Copy link
kbradl16 commented Oct 20, 2019

Hello, I'm running windows and I was also having issues where eslint wouldn't exit after execution (it would print the problems, warnings and errors and then just hang). adding --debug I didn't see anything abnormal.

Adding the parserOptions: {noWatch: true} fixes the hanging issue and I would like to see if your commit(s) fixes the issue for me also. I'm not sure how to tell what commits went into a canary build though. Which canary build should I try? 2.4.1-alpha.8 or 2.4.1-alpha.9 or does it matter?

@bradzacher
Copy link
Member

The canary build is built off of master on every single commit.
There's no need to think about which version to use from the canary tag - just install the latest.

@doberkofler
Copy link
Contributor Author

@bradzacher Just a quick update while still running. I've just upgraded to 2.5.0 and unfortunately things got worse for me. When running eslint with the --debug flag it now takes up to 30 seconds (before the average was about 75ms) for each individual file to be parsed. I must parse over 700 files so it might take more than 6 hours to finish.

@bradzacher
Copy link
Member

Weird, I tested this extensively on CLI and I didn't see any perf regressions.
I was tuning this so that the new code path shouldn't get executed when running from the CLI.
You're running on windows, right? So #1111 won't make any difference for you, because windows is case sensitive.

Could you please do a run the debug flag DEBUG=typescript-eslint:* yarn lint and post the results? Don't worry about the --debug flag.

@doberkofler
Copy link
Contributor Author

My original run did actually end after 3016382ms and I will now run with DEBUG=typescript-eslint:* and let you know the results

@doberkofler
Copy link
Contributor Author

I now just started running eslint again and it again runs as slow as before and should probably take about 1 hours to finish.

The (intermediate) output looks as follows:

# npx eslint --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"
 
 typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\activity.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\activity.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for D:\MyDev\ljs_app\trunk\periscope\tsconfig.json. +1ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for D:\MyDev\ljs_app\trunk\periscope\tsconfig.selenium.json. +11s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +13s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +3s
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +168ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +3ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\alternativeaddress.ts +4s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\alternativeaddress.ts +53ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\alternativeaddress.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\alternativeaddress.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\book.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\book.ts +61ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\book.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\book.ts +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\campaign.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\campaign.ts +58ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\campaign.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\campaign.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\client.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\client.ts +228ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\client.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\client.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\clientinvoice.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\clientinvoice.ts +136ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\clientinvoice.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\clientinvoice.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\collaboration.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\collaboration.ts +199ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\collaboration.ts +1ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\collaboration.ts +4s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\costestimate.ts +4s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\costestimate.ts +90ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\costestimate.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\costestimate.ts +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\currency.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\currency.ts +216ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\currency.ts +2ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\currency.ts +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\employee.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\employee.ts +56ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\employee.ts +1ms
9E88

@doberkofler
Copy link
Contributor Author

You're running on windows, right? So #1111 won't make any difference for you, because windows is case sensitive.

Windows is not case sensitive

@AviVahl
Copy link
AviVahl commented Oct 21, 2019

Windows is not case sensitive, so it needs the canonical path as cache key.
Otherwise c:\my-project\file.txt and C:\MY-PROJECT\file.txt are not considered as pointing to the same path.

@doberkofler
Copy link
Contributor Author

Complete output
output.txt

@bradzacher
Copy link
Member
bradzacher commented Oct 21, 2019

You know what. I knew that. It's windows that's not case sensitive, and unix that is.
I've spent too much time across OS's.

I swear I saw typescript do some weird shit as well.
I need to go back and dig into it more, as I saw a number of cases where typescript was giving me lower case paths on unix...

Either way then, #1111 will fix this for you guys.
It's currently triggering a program update for every single cycle, but the above will stop that.
There wasn't any maintainers available to review my PR on Sunday, so I couldn't cram it in before the auto-release this morning.

@bradzacher
Copy link
Member

Just merged #1111 - there will be a canary version in ~15 minutes.

@doberkofler
Copy link
Contributor Author

I've just tested with the canary version and do not see any changes or speed improvements. It is still about 100x slower then before.

@doberkofler
Copy link
Contributor Author

I'm actually not sure if I should continue updating this SR as you already closed it. At least for me "this" issue is still unresolved and I can only revert back to 2.3.3 but I can also open a new one if you prefer keeping this ones closed.

@bradzacher
Copy link
Member

could you please dump the logs from another cli run using the DEBUG var?

@doberkofler
Copy link
Contributor Author

Sure. Already running but it will take a while. Just wanted to give you a quick update.

@doberkofler
Copy link
Contributor Author

It starts like this...

# npx eslint --report-unused-disable-directives "./**/*.{js,jsx,ts,tsx}"
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\activity.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\activity.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for D:\MyDev\ljs_app\trunk\periscope\tsconfig.json. +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for D:\MyDev\ljs_app\trunk\periscope\tsconfig.selenium.json. +10s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +13s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +3s
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +3ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\address.ts +5s
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +176ms
  typescript-eslint:typescript-estree:createWatchProgram File was not found in program - triggering folder update. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +1ms
  typescript-eslint:typescript-estree:createWatchProgram File was still not found in program after directory update - checking file deletions. D:\MyDev\ljs_app\trunk\periscope\src\data\agency.ts +5s
...

@bradzacher
Copy link
Member
bradzacher commented Oct 22, 2019

Mmm that's all I need. The rest will just be the same thing repeated for each file.
It's just not finding the files in the programs for some reason.

Because it's not finding anything, it's going through the resync branch, which is obv pretty intensive (resync TS program, check for source file (return if exists), trigger folder watchers, resync TS program, check for source file (return if exists), check for deleted files, resync TS program, check for source file (return if exists)).
The deleted files check is especially intensive, which is why it's a last resort.
The entire branch shouldn't ever be hit from the CLI obv, because the program should be correct from the get-go.

I'll try our repo on my old windows laptop and see what happens, as it doesn't repro on mac.

Do you have include in your tsconfig? Or do you have nothing?
Is this a monorepo, or is there just the two tsconfigs listed in the debug?

In the interim, I did add another flag to toggle this feature, as a just-in-case.
TSESTREE_NO_INVALIDATION=true yarn lint.

@doberkofler
Copy link
Contributor Author

My configuration:

eslint_typescript_project.js:

module.exports = {
	parserOptions: {
		project: [
			'./tsconfig.json',
			'./tsconfig.selenium.json'
		]
	}
};

tsconfig.json:

{
  "extends": "./configs/tsconfig_base",
  "compilerOptions": {
    "paths": {
      "Source_Data/*": ["./src/data/*"],
      "Source_Framework/*": ["./src/framework/*"],
      "Source_Modules/*": ["./src/modules/*"],
      "Source_ODS/*": ["./src/ods/*"],
      "Source_Vendor/*": ["./vendor/*"],
      "CoreImg/*": ["./img/*"]
    }
  },
  "include": [
    "src",
    "test/unittest"
  ]
}

tsconfig.selenium.json:

{
  "extends": "./configs/tsconfig_base",
  "compilerOptions": {
    "noEmit": false,
    "target": "es2018",
    "module": "commonjs"
  },
  "include": [
    "test/selenium/pageobjects",
    "test/selenium/tests",
    "test/selenium/util"
  ]
}

tsconfig_base.json:

{
  "compilerOptions": {
    "extendedDiagnostics": false,
    "noEmit": true,
    "downlevelIteration": true,
    "allowJs": true,
    "checkJs": false,
    "sourceMap": true,
    "strict": true,
    "noImplicitAny": true,
    "strictNullChecks": true,
    "noImplicitThis": false,
    "noUnusedLocals": true,
    "noUnusedParameters": false,
    "noImplicitReturns": true,
    "noFallthroughCasesInSwitch": true,
    "esModuleInterop": true,
    "declaration": false,
    "importHelpers": true,
    "moduleResolution": "node",
    "target": "es2015",
    "module": "es2015",
    "jsx": "react",
    "baseUrl": "../",
    "outDir": "../temp",
    "typeRoots": [
      "../node_modules/@types"
    ],
    "lib": [
      "es2017",
      "dom"
    ]
  },
  "compileOnSave": false,
  "exclude": [
    "node_modules"
  ]
}

@doberkofler
Copy link
Contributor Author

This is the output when running with TSESTREE_NO_INVALIDATION=true and it is a lot faster but maybe not as fast as originally.
output.txt

@bradzacher
Copy link
Member

Hmm, that output is still wrong, though.

Your tsconfig isn't missing anything like in #1110, so it's not hitting that case.
So, I don't know why you're getting a 100% cache miss rate.

It must be something different about TS's paths on windows.
I'll dig into it tonight on my windows machines.


As an FYI - during a CLI run, the debug output should look like this, in particular it should just be:

  • "Creating watch program" (once per tsconfig)
  • "Creating project program", then "Found existing program", repeated for every single file.
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/eslint-recommended.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram File did not belong to any existing programs, moving to create/update. /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/configs/eslint-recommended.ts +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/tsconfig.eslint.json. +0ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin-tslint/tsconfig.json. +909ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/tsconfig.json. +769ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/experimental-utils/tsconfig.json. +1s
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/parser/tsconfig.json. +624ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/shared-fixtures/tsconfig.json. +549ms
  typescript-eslint:typescript-estree:createWatchProgram Creating watch program for /Users/bradzacher/github/typescript-eslint/packages/typescript-estree/tsconfig.json. +398ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +5s
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/index.ts +684ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/adjacent-overload-signatures.ts +37ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/adjacent-overload-signatures.ts +39ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/array-type.ts +160ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/array-type.ts +163ms
  typescript-eslint:typescript-estree:createProjectProgram Creating project program for: /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/await-thenable.ts +310ms
  typescript-eslint:typescript-estree:createWatchProgram Found existing program for file. /Users/bradzacher/github/typescript-eslint/packages/eslint-plugin/src/rules/await-thenable.ts +305ms

kukhariev added a commit to kukhariev/node-uploadx that referenced this issue Oct 22, 2019
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 20, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working package: typescript-estree Issues related to @typescript-eslint/typescript-estree
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants
0