Readline completion of module names in import statements #69605
Comments
|
As mentioned in bpo-25209, here is a patch to complete module and attribute names in import statements. This is a rewrite of some personal code I have had kicking around for a few years, but I have added extra features (parsing “import x as alias”, supporting “from .relative import x”). All the existing completions should continue to work as long as the statement immediately preceding the cursor does not look like an “import” statement. When an import statement is detected, these other completions are disabled. When alternative completions are displayed, my code appends a dot (.) indicator to package names, but this is not inserted in the input line. Maybe people think this is a good idea or maybe not. This illustrates what it looks like in action. Text in square brackets was automatically completed: Python 3.6.0a0 (qbase qtip readline/complete-import.patch tip:65d2b721034a, Oct 16 2015, 10:02:3)
[GCC 5.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import io, h<Tab><Tab>
hashlib heapq hmac html. http.
>>> import io, htm[l].<Tab><Tab>
html.entities html.parser
>>> import io, html.e[ntities]
>>> from html import e<Tab><Tab>
entities escape
>>> from html import es[cape]The code works around bpo-16182 (byte vs code point indexing) in a very simplistic way. It probably won’t give useful completions (if any) for non-ASCII input involving imports. The patch currently does not complete built-in module names (sys.builtin_module_names). This should not be hard to add, but I need to decide whether the support should be added specially in rlcompleter, or generally in pkgutil.iter_modules() instead. I also have code to complete the following, which I could try to make patches for if there is interest:
|
|
I'm reviewing the patch, but it will take a time. Wouldn't be simpler to use regular expressions instead of tokenizer? For now Completer doesn't depends on the readline module, nor other global state. It can be used for completing in other environment, for example to complete words in IDE. Patched Completer retrieves additional information from the readline module. This can break existing code. It would be nice to decouple Completer from readline. In contrary to user .pythonrc.py file, we are free to invent new interfaces. May be add methods to the Completer class that provides needed additional information (nothing by default), and add Completer's subclass ReadlineCompleter that implements these methods using readline? Found presumable bugs: "import sy" doesn't suggest completion "sys". "import os.p" doesn't suggest completion "os.path". |
|
Is there a reason ipython's import completer couldn't be borrowed in its entirety? At work, I use a lightly adapted version of the code from ipython to do completion when I'm using the plain interactive interpreter (for whatever reason), and it works just fine. |
|
Josh: Thanks for pointing out I Python. I haven’t used it much myself, but it does seem to do a similar thing to my proposal. L 8000 ooks like the relevant code may be around module_completion() at <https://github.com/ipython/ipython/blob/ab929fe/IPython/core/completerlib.py#L209\>. The “sys” module is one of the builtin modules that I mentioned above. I plan to discuss changing pkgutil.iter_modules() to include it, in a separate bug report. The “os.path” case is more awkward. The “os” module is not actually a package. I believe “import os.path” only works because executing the “os” module modifies sys.modules. My code currently avoids importing non-packages, because I thought it would be annoying to accidentally run a script via tab completion (anything not protected by ‘if __name__ == "__main__" ’). On the other hand, I Python happily completes “os.path” (and many more non-submodule names). It also can be tricked into running scripts, e.g. if you do “import idlelib.__main__.” and press Tab. But maybe this is not a real problem, and I should stop being paranoid. I tend to avoid regular expressions if practical. But Serhiy you may be right that some simple string matching rules would reduce the need for tokenizing. It looks like I Python only has a few simple rules for the entire input line being “import x” and “from x import y”. The disadvantage is less accurate understanding of more complicated syntax, like “from x import y; from z import (a, bb as b, . . .”. It is a tradeoff between simpler code that only supports basic functionality versus complex code that supports more complete functionality. I hear your points about decoupling from Readline and backwards compatibility. I will consider the overall architecture more in a future update. It would be good to allow this stuff to be used in e.g. Idle (though I wouldn’t know where to wire it in myself). |
|
Added few cursory comments on Rietveld. "sys" and "os.path" cases are not critical and we can ignore them if it complicates the code. But preserving Completer independence from readline looks more important to me. We can just add methods get_line_buffer(), get_begidx(), get_endidx() and get_completion_type() to Completer, but may be better to add more high level methods. |
|
I moved all the calls targetting the readline module into a ReadlineCompleter subclass. However the logic for parsing “import” statements still exists in the base Completer class in private methods. An overview of the two classes: class Completer:
def complete(self, text, state):
self._get_matches(text)
def _get_matches(text):
# Only completes global and object.attr names, like before
def _code_matches(self, code, ...):
# Completes import statements, otherwise returns (None, ...)
class ReadlineCompleter(Completer): # New public class
def complete(self, text, state):
# Moved Yury’s Tab insertion logic here
return super().complete(...)
def _get_matches(text):
code = readline.get_line_buffer()[:readline.get_endidx()]
self._code_matches(code)
super()._get_matches(text) # Fallback to existing behaviourPerhaps the _code_matches() and related methods could be turned into more general public APIs, e.g. complete_code(code) -> list of modules, attributes, globals, etc. But that would affect global_matches() and attr_matches(), which I have not touched so far. Other changes: |
|
I changed my mind. I thought it would be easier to implement this with simple regular expressions and string operations, but handling the corner cases makes it difficult. I now think that using the My apologies, @vadmium. Are you still interesting in continuing this work? |
|
I don’t want to spend much time on this, but I’m happy for other people to build on my code. |
|
I should have some time in the next couple of days so I'd like to look into implementing this :) |
* origin/main: (111 commits) pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385) pythongh-131178: Add tests for `ast` command-line interface (python#133329) Regenerate pcbuild.sln in Visual Studio 2022 (python#133394) pythongh-133042: disable HACL* HMAC on Emscripten (python#133064) pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387) pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946) pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388) pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830) pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368) pythongh-132457: make staticmethod and classmethod generic (python#132460) pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812) pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490) pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517) pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628) pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358) bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226) pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287) pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364) pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027) pythonGH-9104 8000 8: Add utils for printing the call stack for asyncio tasks (python#133284) ...
|
Shouldn't we totally disable completion on import attributes? Since completions don't come from module attributes but current namespace, I find it pretty misleading, and unlikely to be useful. Eg. on 3.14 / current main, >>> from typing import Na<TAB>
>>> from typing import NameError(while the user probably wanted to import |
That sounds reasonable to me! Would you like to send a PR for that? We'd most likely need to modify
Currently we just return an empty list for both. |
Sure, I'll have a go at it, thanks for the pointers! |
|
Feel free to ping me if you need help! |
|
Done, it was pretty straightforward indeed! 🚀 Two other things I noted while working on it:
|
|
Thanks! I'll review it tomorrow.
|
Uh oh!
There was an error while loading. Please reload this page.
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
Linked PRs
The text was updated successfully, but these errors were encountered: