10000 Gathering environment information fails consistently · Issue #151 · sourcegraph/scip-python · GitHub
[go: up one dir, main page]
Skip to content

Gathering environment information fails consistently #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
affan-hussain opened this issue Feb 22, 2024 · 3 comments
Open

Gathering environment information fails consistently #151

affan-hussain opened this issue Feb 22, 2024 · 3 comments
Labels
graph/scip-python team/graph

Comments

@affan-hussain
Copy link
affan-hussain commented Feb 22, 2024 •
edited by varungandhi-src
Loading

Hey folks! I'm just trying out SCIP for the first time. I wanted to try running it on my python repo and used the command scip-python index . and it is consistenly failing. Running pip3 show -f on the CLI works fine.

(13:48:15) Evaluating python environment dependencies
(13:48:17)   Gathering environment information from `pip`


Experienced Fatal Error While Indexing:
Please create an issue at github.com/sourcegraph/scip-python: <ref *1> Error: spawnSync /bin/sh ENOBUFS
    at Object.spawnSync (node:internal/child_process:1124:20)
    at spawnSync (node:child_process:876:24)
    at Object.execSync (node:child_process:957:15)
    at /opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104453
    at t.withStatus (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:77273)
    at t.default (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104241)
    at M.index (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:26081)
    at f (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:38106)
    at I.<anonymous> (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:3948)
    at I._actionHandler (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/vendor.js:2:1059335) {
  errno: -55,
  code: 'ENOBUFS',
  syscall: 'spawnSync /bin/sh',
  path: '/bin/sh',
  spawnargs: [
    '-c',
    'pip3 show -f aiohttp aiosignal aiostream annotated-types anthropic anyio asgiref attrs beautifulsoup4 black bleach blinker boltons boto3 boto3-stubs botocore botocore-stubs bracex certifi cffi cfgv charset-normalizer click click-option-group cloudpickle colorama coverage cryptography dataclasses-json defusedxml Deprecated distlib distro dnspython docker exa-py face fastapi fastjsonschema filelock flake8 Flask frozenlist fsspec gitdb GitPython glom grpclib h11 h2 hpack httpcore httpx huggingface-hub hyperframe identify idna importlib-metadata iniconfig isort itsdangerous Jinja2 jira jmespath jsonpatch jsonpointer jsonschema jsonschema-specifications jupyter_client jupyter_core jupyterlab_pygments jwt langchain langchain-community langchain-core langchain-openai langchain-pinecone langsmith loguru markdown-it-py marko MarkupSafe marshmallow mccabe mdurl mistune modal multidict mypy mypy-extensions nbclient nbconvert nbformat networkx networkx-stubs nodeenv numpy oauthlib openai packaging pandas pandas-stubs pandocfilters pathspec peewee pillow pinecone-client pip platformdirs pluggy pre-commit protobuf pycodestyle pycparser pydantic pydantic_core pyflakes PyGithub Pygments PyJWT PyNaCl pytest pytest-json-report pytest-metadata pytest-mock python-dateutil python-dotenv pytz PyYAML pyzmq redis referencing regex requests requests-oauthlib requests-toolbelt rich rpds-py ruamel.yaml ruamel.yaml.clib ruff s3transfer semgrep setuptools sigtools six slack-bolt slack_sdk smmap sniffio soupsieve SQLAlchemy starlette synchronicity tblib tenacity tiktoken tinycss2 tokenizers toml tomli tornado tqdm traitlets tree-sitter tree-sitter-languages typer types-awscrt types-certifi types-pytz types-PyYAML types-requests types-s3transfer types-setuptools types-toml types-tree-sitter types-tree-sitter-languages types-urllib3 typing_extensions typing-inspect tzdata unidiff urllib3 uvicorn virtualenv watchfiles watchtower wcmatch webencodings Werkzeug wrapt yarl zipp'
  ],
  error: [Circular *1],
  status: null,
  signal: 'SIGTERM',
  output: [
    null,
    <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
    <Buffer >
  ],
  pid: 74461,
  stdout: <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
  stderr: <Buffer >
}
@zygi
Copy link
zygi commented Apr 24, 2024

Looks like it's because pip's output is too big to put in the buffer. A proper fix would involve rewriting this with child_process.spawn() (https://stackoverflow.com/questions/63796633/spawnsync-bin-sh-enobufs) but a workaround is to just give a larger buffer to exec:

diff --git a/packages/pyright-scip/src/virtualenv/environment.ts b/packages/pyright-scip/src/virtualenv/environment.ts
index 802ba9f8d..1abd0178b 100644
--- a/packages/pyright-scip/src/virtualenv/environment.ts
+++ b/packages/pyright-scip/src/virtualenv/environment.ts
@@ -29,13 +29,13 @@ let getPipCommand = () => {
 };
 
 function pipList(): PipInformation[] {
-    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`).toString()) as PipInformation[];
+    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`, { maxBuffer: 1024*1024*5 }).toString()) as PipInformation[];
 }
 
 function pipBulkShow(names: string[]): string[] {
     // TODO: This probably breaks with enough names. Should batch them into 512 or whatever the max for bash would be
     return child_process
-        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`)
+        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`, { maxBuffer: 1024*1024*5 })
         .toString()
         .split('\n---');
 }

@mmanela mmanela added the Migrated label May 6, 2024 — with Linear
@eseliger eseliger removed the Migrated label May 21, 2024
@drob
Copy link
drob commented May 29, 2024 •
edited
Loading

I'm running into this as well. Any workarounds? I'm wiping my venv but let me know if there's a better way.

@RheagalFire
Copy link

I'm facing the same issue , any updates on this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
graph/scip-python team/graph
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@mmanela @drob @zygi @affan-hussain @eseliger @RheagalFire @varungandhi-src
0