Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.8/site-packages/IPython/core/completer.py: 22%
1218 statements
« prev ^ index » next coverage.py v7.4.4, created at 2024-04-20 06:09 +0000
1"""Completion for IPython.
3This module started as fork of the rlcompleter module in the Python standard
4library. The original enhancements made to rlcompleter have been sent
5upstream and were accepted as of Python 2.3,
7This module now support a wide variety of completion mechanism both available
8for normal classic Python code, as well as completer for IPython specific
9Syntax like magics.
11Latex and Unicode completion
12============================
14IPython and compatible frontends not only can complete your code, but can help
15you to input a wide range of characters. In particular we allow you to insert
16a unicode character using the tab completion mechanism.
18Forward latex/unicode completion
19--------------------------------
21Forward completion allows you to easily type a unicode character using its latex
22name, or unicode long description. To do so type a backslash follow by the
23relevant name and press tab:
26Using latex completion:
28.. code::
30 \\alpha<tab>
31 α
33or using unicode completion:
36.. code::
38 \\GREEK SMALL LETTER ALPHA<tab>
39 α
42Only valid Python identifiers will complete. Combining characters (like arrow or
43dots) are also available, unlike latex they need to be put after the their
44counterpart that is to say, ``F\\\\vec<tab>`` is correct, not ``\\\\vec<tab>F``.
46Some browsers are known to display combining characters incorrectly.
48Backward latex completion
49-------------------------
51It is sometime challenging to know how to type a character, if you are using
52IPython, or any compatible frontend you can prepend backslash to the character
53and press :kbd:`Tab` to expand it to its latex form.
55.. code::
57 \\α<tab>
58 \\alpha
61Both forward and backward completions can be deactivated by setting the
62:std:configtrait:`Completer.backslash_combining_completions` option to
63``False``.
66Experimental
67============
69Starting with IPython 6.0, this module can make use of the Jedi library to
70generate completions both using static analysis of the code, and dynamically
71inspecting multiple namespaces. Jedi is an autocompletion and static analysis
72for Python. The APIs attached to this new mechanism is unstable and will
73raise unless use in an :any:`provisionalcompleter` context manager.
75You will find that the following are experimental:
77 - :any:`provisionalcompleter`
78 - :any:`IPCompleter.completions`
79 - :any:`Completion`
80 - :any:`rectify_completions`
82.. note::
84 better name for :any:`rectify_completions` ?
86We welcome any feedback on these new API, and we also encourage you to try this
87module in debug mode (start IPython with ``--Completer.debug=True``) in order
88to have extra logging information if :any:`jedi` is crashing, or if current
89IPython completer pending deprecations are returning results not yet handled
90by :any:`jedi`
92Using Jedi for tab completion allow snippets like the following to work without
93having to execute any code:
95 >>> myvar = ['hello', 42]
96 ... myvar[1].bi<tab>
98Tab completion will be able to infer that ``myvar[1]`` is a real number without
99executing almost any code unlike the deprecated :any:`IPCompleter.greedy`
100option.
102Be sure to update :any:`jedi` to the latest stable version or to try the
103current development version to get better completions.
105Matchers
106========
108All completions routines are implemented using unified *Matchers* API.
109The matchers API is provisional and subject to change without notice.
111The built-in matchers include:
113- :any:`IPCompleter.dict_key_matcher`: dictionary key completions,
114- :any:`IPCompleter.magic_matcher`: completions for magics,
115- :any:`IPCompleter.unicode_name_matcher`,
116 :any:`IPCompleter.fwd_unicode_matcher`
117 and :any:`IPCompleter.latex_name_matcher`: see `Forward latex/unicode completion`_,
118- :any:`back_unicode_name_matcher` and :any:`back_latex_name_matcher`: see `Backward latex completion`_,
119- :any:`IPCompleter.file_matcher`: paths to files and directories,
120- :any:`IPCompleter.python_func_kw_matcher` - function keywords,
121- :any:`IPCompleter.python_matches` - globals and attributes (v1 API),
122- ``IPCompleter.jedi_matcher`` - static analysis with Jedi,
123- :any:`IPCompleter.custom_completer_matcher` - pluggable completer with a default
124 implementation in :any:`InteractiveShell` which uses IPython hooks system
125 (`complete_command`) with string dispatch (including regular expressions).
126 Differently to other matchers, ``custom_completer_matcher`` will not suppress
127 Jedi results to match behaviour in earlier IPython versions.
129Custom matchers can be added by appending to ``IPCompleter.custom_matchers`` list.
131Matcher API
132-----------
134Simplifying some details, the ``Matcher`` interface can described as
136.. code-block::
138 MatcherAPIv1 = Callable[[str], list[str]]
139 MatcherAPIv2 = Callable[[CompletionContext], SimpleMatcherResult]
141 Matcher = MatcherAPIv1 | MatcherAPIv2
143The ``MatcherAPIv1`` reflects the matcher API as available prior to IPython 8.6.0
144and remains supported as a simplest way for generating completions. This is also
145currently the only API supported by the IPython hooks system `complete_command`.
147To distinguish between matcher versions ``matcher_api_version`` attribute is used.
148More precisely, the API allows to omit ``matcher_api_version`` for v1 Matchers,
149and requires a literal ``2`` for v2 Matchers.
151Once the API stabilises future versions may relax the requirement for specifying
152``matcher_api_version`` by switching to :any:`functools.singledispatch`, therefore
153please do not rely on the presence of ``matcher_api_version`` for any purposes.
155Suppression of competing matchers
156---------------------------------
158By default results from all matchers are combined, in the order determined by
159their priority. Matchers can request to suppress results from subsequent
160matchers by setting ``suppress`` to ``True`` in the ``MatcherResult``.
162When multiple matchers simultaneously request surpression, the results from of
163the matcher with higher priority will be returned.
165Sometimes it is desirable to suppress most but not all other matchers;
166this can be achieved by adding a set of identifiers of matchers which
167should not be suppressed to ``MatcherResult`` under ``do_not_suppress`` key.
169The suppression behaviour can is user-configurable via
170:std:configtrait:`IPCompleter.suppress_competing_matchers`.
171"""
174# Copyright (c) IPython Development Team.
175# Distributed under the terms of the Modified BSD License.
176#
177# Some of this code originated from rlcompleter in the Python standard library
178# Copyright (C) 2001 Python Software Foundation, www.python.org
180from __future__ import annotations
181import builtins as builtin_mod
182import enum
183import glob
184import inspect
185import itertools
186import keyword
187import os
188import re
189import string
190import sys
191import tokenize
192import time
193import unicodedata
194import uuid
195import warnings
196from ast import literal_eval
197from collections import defaultdict
198from contextlib import contextmanager
199from dataclasses import dataclass
200from functools import cached_property, partial
201from types import SimpleNamespace
202from typing import (
203 Iterable,
204 Iterator,
205 List,
206 Tuple,
207 Union,
208 Any,
209 Sequence,
210 Dict,
211 Optional,
212 TYPE_CHECKING,
213 Set,
214 Sized,
215 TypeVar,
216 Literal,
217)
219from IPython.core.guarded_eval import guarded_eval, EvaluationContext
220from IPython.core.error import TryNext
221from IPython.core.inputtransformer2 import ESC_MAGIC
222from IPython.core.latex_symbols import latex_symbols, reverse_latex_symbol
223from IPython.core.oinspect import InspectColors
224from IPython.testing.skipdoctest import skip_doctest
225from IPython.utils import generics
226from IPython.utils.decorators import sphinx_options
227from IPython.utils.dir2 import dir2, get_real_method
228from IPython.utils.docs import GENERATING_DOCUMENTATION
229from IPython.utils.path import ensure_dir_exists
230from IPython.utils.process import arg_split
231from traitlets import (
232 Bool,
233 Enum,
234 Int,
235 List as ListTrait,
236 Unicode,
237 Dict as DictTrait,
238 Union as UnionTrait,
239 observe,
240)
241from traitlets.config.configurable import Configurable
243import __main__
245# skip module docstests
246__skip_doctest__ = True
249try:
250 import jedi
251 jedi.settings.case_insensitive_completion = False
252 import jedi.api.helpers
253 import jedi.api.classes
254 JEDI_INSTALLED = True
255except ImportError:
256 JEDI_INSTALLED = False
259if TYPE_CHECKING or GENERATING_DOCUMENTATION and sys.version_info >= (3, 11):
260 from typing import cast
261 from typing_extensions import TypedDict, NotRequired, Protocol, TypeAlias, TypeGuard
262else:
263 from typing import Generic
265 def cast(type_, obj):
266 """Workaround for `TypeError: MatcherAPIv2() takes no arguments`"""
267 return obj
269 # do not require on runtime
270 NotRequired = Tuple # requires Python >=3.11
271 TypedDict = Dict # by extension of `NotRequired` requires 3.11 too
272 Protocol = object # requires Python >=3.8
273 TypeAlias = Any # requires Python >=3.10
274 TypeGuard = Generic # requires Python >=3.10
275if GENERATING_DOCUMENTATION:
276 from typing import TypedDict
278# -----------------------------------------------------------------------------
279# Globals
280#-----------------------------------------------------------------------------
282# ranges where we have most of the valid unicode names. We could be more finer
283# grained but is it worth it for performance While unicode have character in the
284# range 0, 0x110000, we seem to have name for about 10% of those. (131808 as I
285# write this). With below range we cover them all, with a density of ~67%
286# biggest next gap we consider only adds up about 1% density and there are 600
287# gaps that would need hard coding.
288_UNICODE_RANGES = [(32, 0x323B0), (0xE0001, 0xE01F0)]
290# Public API
291__all__ = ["Completer", "IPCompleter"]
293if sys.platform == 'win32':
294 PROTECTABLES = ' '
295else:
296 PROTECTABLES = ' ()[]{}?=\\|;:\'#*"^&'
298# Protect against returning an enormous number of completions which the frontend
299# may have trouble processing.
300MATCHES_LIMIT = 500
302# Completion type reported when no type can be inferred.
303_UNKNOWN_TYPE = "<unknown>"
305# sentinel value to signal lack of a match
306not_found = object()
308class ProvisionalCompleterWarning(FutureWarning):
309 """
310 Exception raise by an experimental feature in this module.
312 Wrap code in :any:`provisionalcompleter` context manager if you
313 are certain you want to use an unstable feature.
314 """
315 pass
317warnings.filterwarnings('error', category=ProvisionalCompleterWarning)
320@skip_doctest
321@contextmanager
322def provisionalcompleter(action='ignore'):
323 """
324 This context manager has to be used in any place where unstable completer
325 behavior and API may be called.
327 >>> with provisionalcompleter():
328 ... completer.do_experimental_things() # works
330 >>> completer.do_experimental_things() # raises.
332 .. note::
334 Unstable
336 By using this context manager you agree that the API in use may change
337 without warning, and that you won't complain if they do so.
339 You also understand that, if the API is not to your liking, you should report
340 a bug to explain your use case upstream.
342 We'll be happy to get your feedback, feature requests, and improvements on
343 any of the unstable APIs!
344 """
345 with warnings.catch_warnings():
346 warnings.filterwarnings(action, category=ProvisionalCompleterWarning)
347 yield
350def has_open_quotes(s):
351 """Return whether a string has open quotes.
353 This simply counts whether the number of quote characters of either type in
354 the string is odd.
356 Returns
357 -------
358 If there is an open quote, the quote character is returned. Else, return
359 False.
360 """
361 # We check " first, then ', so complex cases with nested quotes will get
362 # the " to take precedence.
363 if s.count('"') % 2:
364 return '"'
365 elif s.count("'") % 2:
366 return "'"
367 else:
368 return False
371def protect_filename(s, protectables=PROTECTABLES):
372 """Escape a string to protect certain characters."""
373 if set(s) & set(protectables):
374 if sys.platform == "win32":
375 return '"' + s + '"'
376 else:
377 return "".join(("\\" + c if c in protectables else c) for c in s)
378 else:
379 return s
382def expand_user(path:str) -> Tuple[str, bool, str]:
383 """Expand ``~``-style usernames in strings.
385 This is similar to :func:`os.path.expanduser`, but it computes and returns
386 extra information that will be useful if the input was being used in
387 computing completions, and you wish to return the completions with the
388 original '~' instead of its expanded value.
390 Parameters
391 ----------
392 path : str
393 String to be expanded. If no ~ is present, the output is the same as the
394 input.
396 Returns
397 -------
398 newpath : str
399 Result of ~ expansion in the input path.
400 tilde_expand : bool
401 Whether any expansion was performed or not.
402 tilde_val : str
403 The value that ~ was replaced with.
404 """
405 # Default values
406 tilde_expand = False
407 tilde_val = ''
408 newpath = path
410 if path.startswith('~'):
411 tilde_expand = True
412 rest = len(path)-1
413 newpath = os.path.expanduser(path)
414 if rest:
415 tilde_val = newpath[:-rest]
416 else:
417 tilde_val = newpath
419 return newpath, tilde_expand, tilde_val
422def compress_user(path:str, tilde_expand:bool, tilde_val:str) -> str:
423 """Does the opposite of expand_user, with its outputs.
424 """
425 if tilde_expand:
426 return path.replace(tilde_val, '~')
427 else:
428 return path
431def completions_sorting_key(word):
432 """key for sorting completions
434 This does several things:
436 - Demote any completions starting with underscores to the end
437 - Insert any %magic and %%cellmagic completions in the alphabetical order
438 by their name
439 """
440 prio1, prio2 = 0, 0
442 if word.startswith('__'):
443 prio1 = 2
444 elif word.startswith('_'):
445 prio1 = 1
447 if word.endswith('='):
448 prio1 = -1
450 if word.startswith('%%'):
451 # If there's another % in there, this is something else, so leave it alone
452 if not "%" in word[2:]:
453 word = word[2:]
454 prio2 = 2
455 elif word.startswith('%'):
456 if not "%" in word[1:]:
457 word = word[1:]
458 prio2 = 1
460 return prio1, word, prio2
463class _FakeJediCompletion:
464 """
465 This is a workaround to communicate to the UI that Jedi has crashed and to
466 report a bug. Will be used only id :any:`IPCompleter.debug` is set to true.
468 Added in IPython 6.0 so should likely be removed for 7.0
470 """
472 def __init__(self, name):
474 self.name = name
475 self.complete = name
476 self.type = 'crashed'
477 self.name_with_symbols = name
478 self.signature = ""
479 self._origin = "fake"
480 self.text = "crashed"
482 def __repr__(self):
483 return '<Fake completion object jedi has crashed>'
486_JediCompletionLike = Union["jedi.api.Completion", _FakeJediCompletion]
489class Completion:
490 """
491 Completion object used and returned by IPython completers.
493 .. warning::
495 Unstable
497 This function is unstable, API may change without warning.
498 It will also raise unless use in proper context manager.
500 This act as a middle ground :any:`Completion` object between the
501 :any:`jedi.api.classes.Completion` object and the Prompt Toolkit completion
502 object. While Jedi need a lot of information about evaluator and how the
503 code should be ran/inspected, PromptToolkit (and other frontend) mostly
504 need user facing information.
506 - Which range should be replaced replaced by what.
507 - Some metadata (like completion type), or meta information to displayed to
508 the use user.
510 For debugging purpose we can also store the origin of the completion (``jedi``,
511 ``IPython.python_matches``, ``IPython.magics_matches``...).
512 """
514 __slots__ = ['start', 'end', 'text', 'type', 'signature', '_origin']
516 def __init__(
517 self,
518 start: int,
519 end: int,
520 text: str,
521 *,
522 type: Optional[str] = None,
523 _origin="",
524 signature="",
525 ) -> None:
526 warnings.warn(
527 "``Completion`` is a provisional API (as of IPython 6.0). "
528 "It may change without warnings. "
529 "Use in corresponding context manager.",
530 category=ProvisionalCompleterWarning,
531 stacklevel=2,
532 )
534 self.start = start
535 self.end = end
536 self.text = text
537 self.type = type
538 self.signature = signature
539 self._origin = _origin
541 def __repr__(self):
542 return '<Completion start=%s end=%s text=%r type=%r, signature=%r,>' % \
543 (self.start, self.end, self.text, self.type or '?', self.signature or '?')
545 def __eq__(self, other) -> bool:
546 """
547 Equality and hash do not hash the type (as some completer may not be
548 able to infer the type), but are use to (partially) de-duplicate
549 completion.
551 Completely de-duplicating completion is a bit tricker that just
552 comparing as it depends on surrounding text, which Completions are not
553 aware of.
554 """
555 return self.start == other.start and \
556 self.end == other.end and \
557 self.text == other.text
559 def __hash__(self):
560 return hash((self.start, self.end, self.text))
563class SimpleCompletion:
564 """Completion item to be included in the dictionary returned by new-style Matcher (API v2).
566 .. warning::
568 Provisional
570 This class is used to describe the currently supported attributes of
571 simple completion items, and any additional implementation details
572 should not be relied on. Additional attributes may be included in
573 future versions, and meaning of text disambiguated from the current
574 dual meaning of "text to insert" and "text to used as a label".
575 """
577 __slots__ = ["text", "type"]
579 def __init__(self, text: str, *, type: Optional[str] = None):
580 self.text = text
581 self.type = type
583 def __repr__(self):
584 return f"<SimpleCompletion text={self.text!r} type={self.type!r}>"
587class _MatcherResultBase(TypedDict):
588 """Definition of dictionary to be returned by new-style Matcher (API v2)."""
590 #: Suffix of the provided ``CompletionContext.token``, if not given defaults to full token.
591 matched_fragment: NotRequired[str]
593 #: Whether to suppress results from all other matchers (True), some
594 #: matchers (set of identifiers) or none (False); default is False.
595 suppress: NotRequired[Union[bool, Set[str]]]
597 #: Identifiers of matchers which should NOT be suppressed when this matcher
598 #: requests to suppress all other matchers; defaults to an empty set.
599 do_not_suppress: NotRequired[Set[str]]
601 #: Are completions already ordered and should be left as-is? default is False.
602 ordered: NotRequired[bool]
605@sphinx_options(show_inherited_members=True, exclude_inherited_from=["dict"])
606class SimpleMatcherResult(_MatcherResultBase, TypedDict):
607 """Result of new-style completion matcher."""
609 # note: TypedDict is added again to the inheritance chain
610 # in order to get __orig_bases__ for documentation
612 #: List of candidate completions
613 completions: Sequence[SimpleCompletion] | Iterator[SimpleCompletion]
616class _JediMatcherResult(_MatcherResultBase):
617 """Matching result returned by Jedi (will be processed differently)"""
619 #: list of candidate completions
620 completions: Iterator[_JediCompletionLike]
623AnyMatcherCompletion = Union[_JediCompletionLike, SimpleCompletion]
624AnyCompletion = TypeVar("AnyCompletion", AnyMatcherCompletion, Completion)
627@dataclass
628class CompletionContext:
629 """Completion context provided as an argument to matchers in the Matcher API v2."""
631 # rationale: many legacy matchers relied on completer state (`self.text_until_cursor`)
632 # which was not explicitly visible as an argument of the matcher, making any refactor
633 # prone to errors; by explicitly passing `cursor_position` we can decouple the matchers
634 # from the completer, and make substituting them in sub-classes easier.
636 #: Relevant fragment of code directly preceding the cursor.
637 #: The extraction of token is implemented via splitter heuristic
638 #: (following readline behaviour for legacy reasons), which is user configurable
639 #: (by switching the greedy mode).
640 token: str
642 #: The full available content of the editor or buffer
643 full_text: str
645 #: Cursor position in the line (the same for ``full_text`` and ``text``).
646 cursor_position: int
648 #: Cursor line in ``full_text``.
649 cursor_line: int
651 #: The maximum number of completions that will be used downstream.
652 #: Matchers can use this information to abort early.
653 #: The built-in Jedi matcher is currently excepted from this limit.
654 # If not given, return all possible completions.
655 limit: Optional[int]
657 @cached_property
658 def text_until_cursor(self) -> str:
659 return self.line_with_cursor[: self.cursor_position]
661 @cached_property
662 def line_with_cursor(self) -> str:
663 return self.full_text.split("\n")[self.cursor_line]
666#: Matcher results for API v2.
667MatcherResult = Union[SimpleMatcherResult, _JediMatcherResult]
670class _MatcherAPIv1Base(Protocol):
671 def __call__(self, text: str) -> List[str]:
672 """Call signature."""
673 ...
675 #: Used to construct the default matcher identifier
676 __qualname__: str
679class _MatcherAPIv1Total(_MatcherAPIv1Base, Protocol):
680 #: API version
681 matcher_api_version: Optional[Literal[1]]
683 def __call__(self, text: str) -> List[str]:
684 """Call signature."""
685 ...
688#: Protocol describing Matcher API v1.
689MatcherAPIv1: TypeAlias = Union[_MatcherAPIv1Base, _MatcherAPIv1Total]
692class MatcherAPIv2(Protocol):
693 """Protocol describing Matcher API v2."""
695 #: API version
696 matcher_api_version: Literal[2] = 2
698 def __call__(self, context: CompletionContext) -> MatcherResult:
699 """Call signature."""
700 ...
702 #: Used to construct the default matcher identifier
703 __qualname__: str
706Matcher: TypeAlias = Union[MatcherAPIv1, MatcherAPIv2]
709def _is_matcher_v1(matcher: Matcher) -> TypeGuard[MatcherAPIv1]:
710 api_version = _get_matcher_api_version(matcher)
711 return api_version == 1
714def _is_matcher_v2(matcher: Matcher) -> TypeGuard[MatcherAPIv2]:
715 api_version = _get_matcher_api_version(matcher)
716 return api_version == 2
719def _is_sizable(value: Any) -> TypeGuard[Sized]:
720 """Determines whether objects is sizable"""
721 return hasattr(value, "__len__")
724def _is_iterator(value: Any) -> TypeGuard[Iterator]:
725 """Determines whether objects is sizable"""
726 return hasattr(value, "__next__")
729def has_any_completions(result: MatcherResult) -> bool:
730 """Check if any result includes any completions."""
731 completions = result["completions"]
732 if _is_sizable(completions):
733 return len(completions) != 0
734 if _is_iterator(completions):
735 try:
736 old_iterator = completions
737 first = next(old_iterator)
738 result["completions"] = cast(
739 Iterator[SimpleCompletion],
740 itertools.chain([first], old_iterator),
741 )
742 return True
743 except StopIteration:
744 return False
745 raise ValueError(
746 "Completions returned by matcher need to be an Iterator or a Sizable"
747 )
750def completion_matcher(
751 *,
752 priority: Optional[float] = None,
753 identifier: Optional[str] = None,
754 api_version: int = 1,
755):
756 """Adds attributes describing the matcher.
758 Parameters
759 ----------
760 priority : Optional[float]
761 The priority of the matcher, determines the order of execution of matchers.
762 Higher priority means that the matcher will be executed first. Defaults to 0.
763 identifier : Optional[str]
764 identifier of the matcher allowing users to modify the behaviour via traitlets,
765 and also used to for debugging (will be passed as ``origin`` with the completions).
767 Defaults to matcher function's ``__qualname__`` (for example,
768 ``IPCompleter.file_matcher`` for the built-in matched defined
769 as a ``file_matcher`` method of the ``IPCompleter`` class).
770 api_version: Optional[int]
771 version of the Matcher API used by this matcher.
772 Currently supported values are 1 and 2.
773 Defaults to 1.
774 """
776 def wrapper(func: Matcher):
777 func.matcher_priority = priority or 0 # type: ignore
778 func.matcher_identifier = identifier or func.__qualname__ # type: ignore
779 func.matcher_api_version = api_version # type: ignore
780 if TYPE_CHECKING:
781 if api_version == 1:
782 func = cast(MatcherAPIv1, func)
783 elif api_version == 2:
784 func = cast(MatcherAPIv2, func)
785 return func
787 return wrapper
790def _get_matcher_priority(matcher: Matcher):
791 return getattr(matcher, "matcher_priority", 0)
794def _get_matcher_id(matcher: Matcher):
795 return getattr(matcher, "matcher_identifier", matcher.__qualname__)
798def _get_matcher_api_version(matcher):
799 return getattr(matcher, "matcher_api_version", 1)
802context_matcher = partial(completion_matcher, api_version=2)
805_IC = Iterable[Completion]
808def _deduplicate_completions(text: str, completions: _IC)-> _IC:
809 """
810 Deduplicate a set of completions.
812 .. warning::
814 Unstable
816 This function is unstable, API may change without warning.
818 Parameters
819 ----------
820 text : str
821 text that should be completed.
822 completions : Iterator[Completion]
823 iterator over the completions to deduplicate
825 Yields
826 ------
827 `Completions` objects
828 Completions coming from multiple sources, may be different but end up having
829 the same effect when applied to ``text``. If this is the case, this will
830 consider completions as equal and only emit the first encountered.
831 Not folded in `completions()` yet for debugging purpose, and to detect when
832 the IPython completer does return things that Jedi does not, but should be
833 at some point.
834 """
835 completions = list(completions)
836 if not completions:
837 return
839 new_start = min(c.start for c in completions)
840 new_end = max(c.end for c in completions)
842 seen = set()
843 for c in completions:
844 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
845 if new_text not in seen:
846 yield c
847 seen.add(new_text)
850def rectify_completions(text: str, completions: _IC, *, _debug: bool = False) -> _IC:
851 """
852 Rectify a set of completions to all have the same ``start`` and ``end``
854 .. warning::
856 Unstable
858 This function is unstable, API may change without warning.
859 It will also raise unless use in proper context manager.
861 Parameters
862 ----------
863 text : str
864 text that should be completed.
865 completions : Iterator[Completion]
866 iterator over the completions to rectify
867 _debug : bool
868 Log failed completion
870 Notes
871 -----
872 :any:`jedi.api.classes.Completion` s returned by Jedi may not have the same start and end, though
873 the Jupyter Protocol requires them to behave like so. This will readjust
874 the completion to have the same ``start`` and ``end`` by padding both
875 extremities with surrounding text.
877 During stabilisation should support a ``_debug`` option to log which
878 completion are return by the IPython completer and not found in Jedi in
879 order to make upstream bug report.
880 """
881 warnings.warn("`rectify_completions` is a provisional API (as of IPython 6.0). "
882 "It may change without warnings. "
883 "Use in corresponding context manager.",
884 category=ProvisionalCompleterWarning, stacklevel=2)
886 completions = list(completions)
887 if not completions:
888 return
889 starts = (c.start for c in completions)
890 ends = (c.end for c in completions)
892 new_start = min(starts)
893 new_end = max(ends)
895 seen_jedi = set()
896 seen_python_matches = set()
897 for c in completions:
898 new_text = text[new_start:c.start] + c.text + text[c.end:new_end]
899 if c._origin == 'jedi':
900 seen_jedi.add(new_text)
901 elif c._origin == 'IPCompleter.python_matches':
902 seen_python_matches.add(new_text)
903 yield Completion(new_start, new_end, new_text, type=c.type, _origin=c._origin, signature=c.signature)
904 diff = seen_python_matches.difference(seen_jedi)
905 if diff and _debug:
906 print('IPython.python matches have extras:', diff)
909if sys.platform == 'win32':
910 DELIMS = ' \t\n`!@#$^&*()=+[{]}|;\'",<>?'
911else:
912 DELIMS = ' \t\n`!@#$^&*()=+[{]}\\|;:\'",<>?'
914GREEDY_DELIMS = ' =\r\n'
917class CompletionSplitter(object):
918 """An object to split an input line in a manner similar to readline.
920 By having our own implementation, we can expose readline-like completion in
921 a uniform manner to all frontends. This object only needs to be given the
922 line of text to be split and the cursor position on said line, and it
923 returns the 'word' to be completed on at the cursor after splitting the
924 entire line.
926 What characters are used as splitting delimiters can be controlled by
927 setting the ``delims`` attribute (this is a property that internally
928 automatically builds the necessary regular expression)"""
930 # Private interface
932 # A string of delimiter characters. The default value makes sense for
933 # IPython's most typical usage patterns.
934 _delims = DELIMS
936 # The expression (a normal string) to be compiled into a regular expression
937 # for actual splitting. We store it as an attribute mostly for ease of
938 # debugging, since this type of code can be so tricky to debug.
939 _delim_expr = None
941 # The regular expression that does the actual splitting
942 _delim_re = None
944 def __init__(self, delims=None):
945 delims = CompletionSplitter._delims if delims is None else delims
946 self.delims = delims
948 @property
949 def delims(self):
950 """Return the string of delimiter characters."""
951 return self._delims
953 @delims.setter
954 def delims(self, delims):
955 """Set the delimiters for line splitting."""
956 expr = '[' + ''.join('\\'+ c for c in delims) + ']'
957 self._delim_re = re.compile(expr)
958 self._delims = delims
959 self._delim_expr = expr
961 def split_line(self, line, cursor_pos=None):
962 """Split a line of text with a cursor at the given position.
963 """
964 l = line if cursor_pos is None else line[:cursor_pos]
965 return self._delim_re.split(l)[-1]
969class Completer(Configurable):
971 greedy = Bool(
972 False,
973 help="""Activate greedy completion.
975 .. deprecated:: 8.8
976 Use :std:configtrait:`Completer.evaluation` and :std:configtrait:`Completer.auto_close_dict_keys` instead.
978 When enabled in IPython 8.8 or newer, changes configuration as follows:
980 - ``Completer.evaluation = 'unsafe'``
981 - ``Completer.auto_close_dict_keys = True``
982 """,
983 ).tag(config=True)
985 evaluation = Enum(
986 ("forbidden", "minimal", "limited", "unsafe", "dangerous"),
987 default_value="limited",
988 help="""Policy for code evaluation under completion.
990 Successive options allow to enable more eager evaluation for better
991 completion suggestions, including for nested dictionaries, nested lists,
992 or even results of function calls.
993 Setting ``unsafe`` or higher can lead to evaluation of arbitrary user
994 code on :kbd:`Tab` with potentially unwanted or dangerous side effects.
996 Allowed values are:
998 - ``forbidden``: no evaluation of code is permitted,
999 - ``minimal``: evaluation of literals and access to built-in namespace;
1000 no item/attribute evaluationm no access to locals/globals,
1001 no evaluation of any operations or comparisons.
1002 - ``limited``: access to all namespaces, evaluation of hard-coded methods
1003 (for example: :any:`dict.keys`, :any:`object.__getattr__`,
1004 :any:`object.__getitem__`) on allow-listed objects (for example:
1005 :any:`dict`, :any:`list`, :any:`tuple`, ``pandas.Series``),
1006 - ``unsafe``: evaluation of all methods and function calls but not of
1007 syntax with side-effects like `del x`,
1008 - ``dangerous``: completely arbitrary evaluation.
1009 """,
1010 ).tag(config=True)
1012 use_jedi = Bool(default_value=JEDI_INSTALLED,
1013 help="Experimental: Use Jedi to generate autocompletions. "
1014 "Default to True if jedi is installed.").tag(config=True)
1016 jedi_compute_type_timeout = Int(default_value=400,
1017 help="""Experimental: restrict time (in milliseconds) during which Jedi can compute types.
1018 Set to 0 to stop computing types. Non-zero value lower than 100ms may hurt
1019 performance by preventing jedi to build its cache.
1020 """).tag(config=True)
1022 debug = Bool(default_value=False,
1023 help='Enable debug for the Completer. Mostly print extra '
1024 'information for experimental jedi integration.')\
1025 .tag(config=True)
1027 backslash_combining_completions = Bool(True,
1028 help="Enable unicode completions, e.g. \\alpha<tab> . "
1029 "Includes completion of latex commands, unicode names, and expanding "
1030 "unicode characters back to latex commands.").tag(config=True)
1032 auto_close_dict_keys = Bool(
1033 False,
1034 help="""
1035 Enable auto-closing dictionary keys.
1037 When enabled string keys will be suffixed with a final quote
1038 (matching the opening quote), tuple keys will also receive a
1039 separating comma if needed, and keys which are final will
1040 receive a closing bracket (``]``).
1041 """,
1042 ).tag(config=True)
1044 def __init__(self, namespace=None, global_namespace=None, **kwargs):
1045 """Create a new completer for the command line.
1047 Completer(namespace=ns, global_namespace=ns2) -> completer instance.
1049 If unspecified, the default namespace where completions are performed
1050 is __main__ (technically, __main__.__dict__). Namespaces should be
1051 given as dictionaries.
1053 An optional second namespace can be given. This allows the completer
1054 to handle cases where both the local and global scopes need to be
1055 distinguished.
1056 """
1058 # Don't bind to namespace quite yet, but flag whether the user wants a
1059 # specific namespace or to use __main__.__dict__. This will allow us
1060 # to bind to __main__.__dict__ at completion time, not now.
1061 if namespace is None:
1062 self.use_main_ns = True
1063 else:
1064 self.use_main_ns = False
1065 self.namespace = namespace
1067 # The global namespace, if given, can be bound directly
1068 if global_namespace is None:
1069 self.global_namespace = {}
1070 else:
1071 self.global_namespace = global_namespace
1073 self.custom_matchers = []
1075 super(Completer, self).__init__(**kwargs)
1077 def complete(self, text, state):
1078 """Return the next possible completion for 'text'.
1080 This is called successively with state == 0, 1, 2, ... until it
1081 returns None. The completion should begin with 'text'.
1083 """
1084 if self.use_main_ns:
1085 self.namespace = __main__.__dict__
1087 if state == 0:
1088 if "." in text:
1089 self.matches = self.attr_matches(text)
1090 else:
1091 self.matches = self.global_matches(text)
1092 try:
1093 return self.matches[state]
1094 except IndexError:
1095 return None
1097 def global_matches(self, text):
1098 """Compute matches when text is a simple name.
1100 Return a list of all keywords, built-in functions and names currently
1101 defined in self.namespace or self.global_namespace that match.
1103 """
1104 matches = []
1105 match_append = matches.append
1106 n = len(text)
1107 for lst in [
1108 keyword.kwlist,
1109 builtin_mod.__dict__.keys(),
1110 list(self.namespace.keys()),
1111 list(self.global_namespace.keys()),
1112 ]:
1113 for word in lst:
1114 if word[:n] == text and word != "__builtins__":
1115 match_append(word)
1117 snake_case_re = re.compile(r"[^_]+(_[^_]+)+?\Z")
1118 for lst in [list(self.namespace.keys()), list(self.global_namespace.keys())]:
1119 shortened = {
1120 "_".join([sub[0] for sub in word.split("_")]): word
1121 for word in lst
1122 if snake_case_re.match(word)
1123 }
1124 for word in shortened.keys():
1125 if word[:n] == text and word != "__builtins__":
1126 match_append(shortened[word])
1127 return matches
1129 def attr_matches(self, text):
1130 """Compute matches when text contains a dot.
1132 Assuming the text is of the form NAME.NAME....[NAME], and is
1133 evaluatable in self.namespace or self.global_namespace, it will be
1134 evaluated and its attributes (as revealed by dir()) are used as
1135 possible completions. (For class instances, class members are
1136 also considered.)
1138 WARNING: this can still invoke arbitrary C code, if an object
1139 with a __getattr__ hook is evaluated.
1141 """
1142 m2 = re.match(r"(.+)\.(\w*)$", self.line_buffer)
1143 if not m2:
1144 return []
1145 expr, attr = m2.group(1, 2)
1147 obj = self._evaluate_expr(expr)
1149 if obj is not_found:
1150 return []
1152 if self.limit_to__all__ and hasattr(obj, '__all__'):
1153 words = get__all__entries(obj)
1154 else:
1155 words = dir2(obj)
1157 try:
1158 words = generics.complete_object(obj, words)
1159 except TryNext:
1160 pass
1161 except AssertionError:
1162 raise
1163 except Exception:
1164 # Silence errors from completion function
1165 pass
1166 # Build match list to return
1167 n = len(attr)
1169 # Note: ideally we would just return words here and the prefix
1170 # reconciliator would know that we intend to append to rather than
1171 # replace the input text; this requires refactoring to return range
1172 # which ought to be replaced (as does jedi).
1173 tokens = _parse_tokens(expr)
1174 rev_tokens = reversed(tokens)
1175 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1176 name_turn = True
1178 parts = []
1179 for token in rev_tokens:
1180 if token.type in skip_over:
1181 continue
1182 if token.type == tokenize.NAME and name_turn:
1183 parts.append(token.string)
1184 name_turn = False
1185 elif token.type == tokenize.OP and token.string == "." and not name_turn:
1186 parts.append(token.string)
1187 name_turn = True
1188 else:
1189 # short-circuit if not empty nor name token
1190 break
1192 prefix_after_space = "".join(reversed(parts))
1194 return ["%s.%s" % (prefix_after_space, w) for w in words if w[:n] == attr]
1196 def _evaluate_expr(self, expr):
1197 obj = not_found
1198 done = False
1199 while not done and expr:
1200 try:
1201 obj = guarded_eval(
1202 expr,
1203 EvaluationContext(
1204 globals=self.global_namespace,
1205 locals=self.namespace,
1206 evaluation=self.evaluation,
1207 ),
1208 )
1209 done = True
1210 except Exception as e:
1211 if self.debug:
1212 print("Evaluation exception", e)
1213 # trim the expression to remove any invalid prefix
1214 # e.g. user starts `(d[`, so we get `expr = '(d'`,
1215 # where parenthesis is not closed.
1216 # TODO: make this faster by reusing parts of the computation?
1217 expr = expr[1:]
1218 return obj
1220def get__all__entries(obj):
1221 """returns the strings in the __all__ attribute"""
1222 try:
1223 words = getattr(obj, '__all__')
1224 except:
1225 return []
1227 return [w for w in words if isinstance(w, str)]
1230class _DictKeyState(enum.Flag):
1231 """Represent state of the key match in context of other possible matches.
1233 - given `d1 = {'a': 1}` completion on `d1['<tab>` will yield `{'a': END_OF_ITEM}` as there is no tuple.
1234 - given `d2 = {('a', 'b'): 1}`: `d2['a', '<tab>` will yield `{'b': END_OF_TUPLE}` as there is no tuple members to add beyond `'b'`.
1235 - given `d3 = {('a', 'b'): 1}`: `d3['<tab>` will yield `{'a': IN_TUPLE}` as `'a'` can be added.
1236 - given `d4 = {'a': 1, ('a', 'b'): 2}`: `d4['<tab>` will yield `{'a': END_OF_ITEM & END_OF_TUPLE}`
1237 """
1239 BASELINE = 0
1240 END_OF_ITEM = enum.auto()
1241 END_OF_TUPLE = enum.auto()
1242 IN_TUPLE = enum.auto()
1245def _parse_tokens(c):
1246 """Parse tokens even if there is an error."""
1247 tokens = []
1248 token_generator = tokenize.generate_tokens(iter(c.splitlines()).__next__)
1249 while True:
1250 try:
1251 tokens.append(next(token_generator))
1252 except tokenize.TokenError:
1253 return tokens
1254 except StopIteration:
1255 return tokens
1258def _match_number_in_dict_key_prefix(prefix: str) -> Union[str, None]:
1259 """Match any valid Python numeric literal in a prefix of dictionary keys.
1261 References:
1262 - https://docs.python.org/3/reference/lexical_analysis.html#numeric-literals
1263 - https://docs.python.org/3/library/tokenize.html
1264 """
1265 if prefix[-1].isspace():
1266 # if user typed a space we do not have anything to complete
1267 # even if there was a valid number token before
1268 return None
1269 tokens = _parse_tokens(prefix)
1270 rev_tokens = reversed(tokens)
1271 skip_over = {tokenize.ENDMARKER, tokenize.NEWLINE}
1272 number = None
1273 for token in rev_tokens:
1274 if token.type in skip_over:
1275 continue
1276 if number is None:
1277 if token.type == tokenize.NUMBER:
1278 number = token.string
1279 continue
1280 else:
1281 # we did not match a number
1282 return None
1283 if token.type == tokenize.OP:
1284 if token.string == ",":
1285 break
1286 if token.string in {"+", "-"}:
1287 number = token.string + number
1288 else:
1289 return None
1290 return number
1293_INT_FORMATS = {
1294 "0b": bin,
1295 "0o": oct,
1296 "0x": hex,
1297}
1300def match_dict_keys(
1301 keys: List[Union[str, bytes, Tuple[Union[str, bytes], ...]]],
1302 prefix: str,
1303 delims: str,
1304 extra_prefix: Optional[Tuple[Union[str, bytes], ...]] = None,
1305) -> Tuple[str, int, Dict[str, _DictKeyState]]:
1306 """Used by dict_key_matches, matching the prefix to a list of keys
1308 Parameters
1309 ----------
1310 keys
1311 list of keys in dictionary currently being completed.
1312 prefix
1313 Part of the text already typed by the user. E.g. `mydict[b'fo`
1314 delims
1315 String of delimiters to consider when finding the current key.
1316 extra_prefix : optional
1317 Part of the text already typed in multi-key index cases. E.g. for
1318 `mydict['foo', "bar", 'b`, this would be `('foo', 'bar')`.
1320 Returns
1321 -------
1322 A tuple of three elements: ``quote``, ``token_start``, ``matched``, with
1323 ``quote`` being the quote that need to be used to close current string.
1324 ``token_start`` the position where the replacement should start occurring,
1325 ``matches`` a dictionary of replacement/completion keys on keys and values
1326 indicating whether the state.
1327 """
1328 prefix_tuple = extra_prefix if extra_prefix else ()
1330 prefix_tuple_size = sum(
1331 [
1332 # for pandas, do not count slices as taking space
1333 not isinstance(k, slice)
1334 for k in prefix_tuple
1335 ]
1336 )
1337 text_serializable_types = (str, bytes, int, float, slice)
1339 def filter_prefix_tuple(key):
1340 # Reject too short keys
1341 if len(key) <= prefix_tuple_size:
1342 return False
1343 # Reject keys which cannot be serialised to text
1344 for k in key:
1345 if not isinstance(k, text_serializable_types):
1346 return False
1347 # Reject keys that do not match the prefix
1348 for k, pt in zip(key, prefix_tuple):
1349 if k != pt and not isinstance(pt, slice):
1350 return False
1351 # All checks passed!
1352 return True
1354 filtered_key_is_final: Dict[
1355 Union[str, bytes, int, float], _DictKeyState
1356 ] = defaultdict(lambda: _DictKeyState.BASELINE)
1358 for k in keys:
1359 # If at least one of the matches is not final, mark as undetermined.
1360 # This can happen with `d = {111: 'b', (111, 222): 'a'}` where
1361 # `111` appears final on first match but is not final on the second.
1363 if isinstance(k, tuple):
1364 if filter_prefix_tuple(k):
1365 key_fragment = k[prefix_tuple_size]
1366 filtered_key_is_final[key_fragment] |= (
1367 _DictKeyState.END_OF_TUPLE
1368 if len(k) == prefix_tuple_size + 1
1369 else _DictKeyState.IN_TUPLE
1370 )
1371 elif prefix_tuple_size > 0:
1372 # we are completing a tuple but this key is not a tuple,
1373 # so we should ignore it
1374 pass
1375 else:
1376 if isinstance(k, text_serializable_types):
1377 filtered_key_is_final[k] |= _DictKeyState.END_OF_ITEM
1379 filtered_keys = filtered_key_is_final.keys()
1381 if not prefix:
1382 return "", 0, {repr(k): v for k, v in filtered_key_is_final.items()}
1384 quote_match = re.search("(?:\"|')", prefix)
1385 is_user_prefix_numeric = False
1387 if quote_match:
1388 quote = quote_match.group()
1389 valid_prefix = prefix + quote
1390 try:
1391 prefix_str = literal_eval(valid_prefix)
1392 except Exception:
1393 return "", 0, {}
1394 else:
1395 # If it does not look like a string, let's assume
1396 # we are dealing with a number or variable.
1397 number_match = _match_number_in_dict_key_prefix(prefix)
1399 # We do not want the key matcher to suggest variable names so we yield:
1400 if number_match is None:
1401 # The alternative would be to assume that user forgort the quote
1402 # and if the substring matches, suggest adding it at the start.
1403 return "", 0, {}
1405 prefix_str = number_match
1406 is_user_prefix_numeric = True
1407 quote = ""
1409 pattern = '[^' + ''.join('\\' + c for c in delims) + ']*$'
1410 token_match = re.search(pattern, prefix, re.UNICODE)
1411 assert token_match is not None # silence mypy
1412 token_start = token_match.start()
1413 token_prefix = token_match.group()
1415 matched: Dict[str, _DictKeyState] = {}
1417 str_key: Union[str, bytes]
1419 for key in filtered_keys:
1420 if isinstance(key, (int, float)):
1421 # User typed a number but this key is not a number.
1422 if not is_user_prefix_numeric:
1423 continue
1424 str_key = str(key)
1425 if isinstance(key, int):
1426 int_base = prefix_str[:2].lower()
1427 # if user typed integer using binary/oct/hex notation:
1428 if int_base in _INT_FORMATS:
1429 int_format = _INT_FORMATS[int_base]
1430 str_key = int_format(key)
1431 else:
1432 # User typed a string but this key is a number.
1433 if is_user_prefix_numeric:
1434 continue
1435 str_key = key
1436 try:
1437 if not str_key.startswith(prefix_str):
1438 continue
1439 except (AttributeError, TypeError, UnicodeError) as e:
1440 # Python 3+ TypeError on b'a'.startswith('a') or vice-versa
1441 continue
1443 # reformat remainder of key to begin with prefix
1444 rem = str_key[len(prefix_str) :]
1445 # force repr wrapped in '
1446 rem_repr = repr(rem + '"') if isinstance(rem, str) else repr(rem + b'"')
1447 rem_repr = rem_repr[1 + rem_repr.index("'"):-2]
1448 if quote == '"':
1449 # The entered prefix is quoted with ",
1450 # but the match is quoted with '.
1451 # A contained " hence needs escaping for comparison:
1452 rem_repr = rem_repr.replace('"', '\\"')
1454 # then reinsert prefix from start of token
1455 match = "%s%s" % (token_prefix, rem_repr)
1457 matched[match] = filtered_key_is_final[key]
1458 return quote, token_start, matched
1461def cursor_to_position(text:str, line:int, column:int)->int:
1462 """
1463 Convert the (line,column) position of the cursor in text to an offset in a
1464 string.
1466 Parameters
1467 ----------
1468 text : str
1469 The text in which to calculate the cursor offset
1470 line : int
1471 Line of the cursor; 0-indexed
1472 column : int
1473 Column of the cursor 0-indexed
1475 Returns
1476 -------
1477 Position of the cursor in ``text``, 0-indexed.
1479 See Also
1480 --------
1481 position_to_cursor : reciprocal of this function
1483 """
1484 lines = text.split('\n')
1485 assert line <= len(lines), '{} <= {}'.format(str(line), str(len(lines)))
1487 return sum(len(l) + 1 for l in lines[:line]) + column
1489def position_to_cursor(text:str, offset:int)->Tuple[int, int]:
1490 """
1491 Convert the position of the cursor in text (0 indexed) to a line
1492 number(0-indexed) and a column number (0-indexed) pair
1494 Position should be a valid position in ``text``.
1496 Parameters
1497 ----------
1498 text : str
1499 The text in which to calculate the cursor offset
1500 offset : int
1501 Position of the cursor in ``text``, 0-indexed.
1503 Returns
1504 -------
1505 (line, column) : (int, int)
1506 Line of the cursor; 0-indexed, column of the cursor 0-indexed
1508 See Also
1509 --------
1510 cursor_to_position : reciprocal of this function
1512 """
1514 assert 0 <= offset <= len(text) , "0 <= %s <= %s" % (offset , len(text))
1516 before = text[:offset]
1517 blines = before.split('\n') # ! splitnes trim trailing \n
1518 line = before.count('\n')
1519 col = len(blines[-1])
1520 return line, col
1523def _safe_isinstance(obj, module, class_name, *attrs):
1524 """Checks if obj is an instance of module.class_name if loaded
1525 """
1526 if module in sys.modules:
1527 m = sys.modules[module]
1528 for attr in [class_name, *attrs]:
1529 m = getattr(m, attr)
1530 return isinstance(obj, m)
1533@context_matcher()
1534def back_unicode_name_matcher(context: CompletionContext):
1535 """Match Unicode characters back to Unicode name
1537 Same as :any:`back_unicode_name_matches`, but adopted to new Matcher API.
1538 """
1539 fragment, matches = back_unicode_name_matches(context.text_until_cursor)
1540 return _convert_matcher_v1_result_to_v2(
1541 matches, type="unicode", fragment=fragment, suppress_if_matches=True
1542 )
1545def back_unicode_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1546 """Match Unicode characters back to Unicode name
1548 This does ``☃`` -> ``\\snowman``
1550 Note that snowman is not a valid python3 combining character but will be expanded.
1551 Though it will not recombine back to the snowman character by the completion machinery.
1553 This will not either back-complete standard sequences like \\n, \\b ...
1555 .. deprecated:: 8.6
1556 You can use :meth:`back_unicode_name_matcher` instead.
1558 Returns
1559 =======
1561 Return a tuple with two elements:
1563 - The Unicode character that was matched (preceded with a backslash), or
1564 empty string,
1565 - a sequence (of 1), name for the match Unicode character, preceded by
1566 backslash, or empty if no match.
1567 """
1568 if len(text)<2:
1569 return '', ()
1570 maybe_slash = text[-2]
1571 if maybe_slash != '\\':
1572 return '', ()
1574 char = text[-1]
1575 # no expand on quote for completion in strings.
1576 # nor backcomplete standard ascii keys
1577 if char in string.ascii_letters or char in ('"',"'"):
1578 return '', ()
1579 try :
1580 unic = unicodedata.name(char)
1581 return '\\'+char,('\\'+unic,)
1582 except KeyError:
1583 pass
1584 return '', ()
1587@context_matcher()
1588def back_latex_name_matcher(context: CompletionContext):
1589 """Match latex characters back to unicode name
1591 Same as :any:`back_latex_name_matches`, but adopted to new Matcher API.
1592 """
1593 fragment, matches = back_latex_name_matches(context.text_until_cursor)
1594 return _convert_matcher_v1_result_to_v2(
1595 matches, type="latex", fragment=fragment, suppress_if_matches=True
1596 )
1599def back_latex_name_matches(text: str) -> Tuple[str, Sequence[str]]:
1600 """Match latex characters back to unicode name
1602 This does ``\\ℵ`` -> ``\\aleph``
1604 .. deprecated:: 8.6
1605 You can use :meth:`back_latex_name_matcher` instead.
1606 """
1607 if len(text)<2:
1608 return '', ()
1609 maybe_slash = text[-2]
1610 if maybe_slash != '\\':
1611 return '', ()
1614 char = text[-1]
1615 # no expand on quote for completion in strings.
1616 # nor backcomplete standard ascii keys
1617 if char in string.ascii_letters or char in ('"',"'"):
1618 return '', ()
1619 try :
1620 latex = reverse_latex_symbol[char]
1621 # '\\' replace the \ as well
1622 return '\\'+char,[latex]
1623 except KeyError:
1624 pass
1625 return '', ()
1628def _formatparamchildren(parameter) -> str:
1629 """
1630 Get parameter name and value from Jedi Private API
1632 Jedi does not expose a simple way to get `param=value` from its API.
1634 Parameters
1635 ----------
1636 parameter
1637 Jedi's function `Param`
1639 Returns
1640 -------
1641 A string like 'a', 'b=1', '*args', '**kwargs'
1643 """
1644 description = parameter.description
1645 if not description.startswith('param '):
1646 raise ValueError('Jedi function parameter description have change format.'
1647 'Expected "param ...", found %r".' % description)
1648 return description[6:]
1650def _make_signature(completion)-> str:
1651 """
1652 Make the signature from a jedi completion
1654 Parameters
1655 ----------
1656 completion : jedi.Completion
1657 object does not complete a function type
1659 Returns
1660 -------
1661 a string consisting of the function signature, with the parenthesis but
1662 without the function name. example:
1663 `(a, *args, b=1, **kwargs)`
1665 """
1667 # it looks like this might work on jedi 0.17
1668 if hasattr(completion, 'get_signatures'):
1669 signatures = completion.get_signatures()
1670 if not signatures:
1671 return '(?)'
1673 c0 = completion.get_signatures()[0]
1674 return '('+c0.to_string().split('(', maxsplit=1)[1]
1676 return '(%s)'% ', '.join([f for f in (_formatparamchildren(p) for signature in completion.get_signatures()
1677 for p in signature.defined_names()) if f])
1680_CompleteResult = Dict[str, MatcherResult]
1683DICT_MATCHER_REGEX = re.compile(
1684 r"""(?x)
1685( # match dict-referring - or any get item object - expression
1686 .+
1687)
1688\[ # open bracket
1689\s* # and optional whitespace
1690# Capture any number of serializable objects (e.g. "a", "b", 'c')
1691# and slices
1692((?:(?:
1693 (?: # closed string
1694 [uUbB]? # string prefix (r not handled)
1695 (?:
1696 '(?:[^']|(?<!\\)\\')*'
1697 |
1698 "(?:[^"]|(?<!\\)\\")*"
1699 )
1700 )
1701 |
1702 # capture integers and slices
1703 (?:[-+]?\d+)?(?::(?:[-+]?\d+)?){0,2}
1704 |
1705 # integer in bin/hex/oct notation
1706 0[bBxXoO]_?(?:\w|\d)+
1707 )
1708 \s*,\s*
1709)*)
1710((?:
1711 (?: # unclosed string
1712 [uUbB]? # string prefix (r not handled)
1713 (?:
1714 '(?:[^']|(?<!\\)\\')*
1715 |
1716 "(?:[^"]|(?<!\\)\\")*
1717 )
1718 )
1719 |
1720 # unfinished integer
1721 (?:[-+]?\d+)
1722 |
1723 # integer in bin/hex/oct notation
1724 0[bBxXoO]_?(?:\w|\d)+
1725 )
1726)?
1727$
1728"""
1729)
1732def _convert_matcher_v1_result_to_v2(
1733 matches: Sequence[str],
1734 type: str,
1735 fragment: Optional[str] = None,
1736 suppress_if_matches: bool = False,
1737) -> SimpleMatcherResult:
1738 """Utility to help with transition"""
1739 result = {
1740 "completions": [SimpleCompletion(text=match, type=type) for match in matches],
1741 "suppress": (True if matches else False) if suppress_if_matches else False,
1742 }
1743 if fragment is not None:
1744 result["matched_fragment"] = fragment
1745 return cast(SimpleMatcherResult, result)
1748class IPCompleter(Completer):
1749 """Extension of the completer class with IPython-specific features"""
1751 @observe('greedy')
1752 def _greedy_changed(self, change):
1753 """update the splitter and readline delims when greedy is changed"""
1754 if change["new"]:
1755 self.evaluation = "unsafe"
1756 self.auto_close_dict_keys = True
1757 self.splitter.delims = GREEDY_DELIMS
1758 else:
1759 self.evaluation = "limited"
1760 self.auto_close_dict_keys = False
1761 self.splitter.delims = DELIMS
1763 dict_keys_only = Bool(
1764 False,
1765 help="""
1766 Whether to show dict key matches only.
1768 (disables all matchers except for `IPCompleter.dict_key_matcher`).
1769 """,
1770 )
1772 suppress_competing_matchers = UnionTrait(
1773 [Bool(allow_none=True), DictTrait(Bool(None, allow_none=True))],
1774 default_value=None,
1775 help="""
1776 Whether to suppress completions from other *Matchers*.
1778 When set to ``None`` (default) the matchers will attempt to auto-detect
1779 whether suppression of other matchers is desirable. For example, at
1780 the beginning of a line followed by `%` we expect a magic completion
1781 to be the only applicable option, and after ``my_dict['`` we usually
1782 expect a completion with an existing dictionary key.
1784 If you want to disable this heuristic and see completions from all matchers,
1785 set ``IPCompleter.suppress_competing_matchers = False``.
1786 To disable the heuristic for specific matchers provide a dictionary mapping:
1787 ``IPCompleter.suppress_competing_matchers = {'IPCompleter.dict_key_matcher': False}``.
1789 Set ``IPCompleter.suppress_competing_matchers = True`` to limit
1790 completions to the set of matchers with the highest priority;
1791 this is equivalent to ``IPCompleter.merge_completions`` and
1792 can be beneficial for performance, but will sometimes omit relevant
1793 candidates from matchers further down the priority list.
1794 """,
1795 ).tag(config=True)
1797 merge_completions = Bool(
1798 True,
1799 help="""Whether to merge completion results into a single list
1801 If False, only the completion results from the first non-empty
1802 completer will be returned.
1804 As of version 8.6.0, setting the value to ``False`` is an alias for:
1805 ``IPCompleter.suppress_competing_matchers = True.``.
1806 """,
1807 ).tag(config=True)
1809 disable_matchers = ListTrait(
1810 Unicode(),
1811 help="""List of matchers to disable.
1813 The list should contain matcher identifiers (see :any:`completion_matcher`).
1814 """,
1815 ).tag(config=True)
1817 omit__names = Enum(
1818 (0, 1, 2),
1819 default_value=2,
1820 help="""Instruct the completer to omit private method names
1822 Specifically, when completing on ``object.<tab>``.
1824 When 2 [default]: all names that start with '_' will be excluded.
1826 When 1: all 'magic' names (``__foo__``) will be excluded.
1828 When 0: nothing will be excluded.
1829 """
1830 ).tag(config=True)
1831 limit_to__all__ = Bool(False,
1832 help="""
1833 DEPRECATED as of version 5.0.
1835 Instruct the completer to use __all__ for the completion
1837 Specifically, when completing on ``object.<tab>``.
1839 When True: only those names in obj.__all__ will be included.
1841 When False [default]: the __all__ attribute is ignored
1842 """,
1843 ).tag(config=True)
1845 profile_completions = Bool(
1846 default_value=False,
1847 help="If True, emit profiling data for completion subsystem using cProfile."
1848 ).tag(config=True)
1850 profiler_output_dir = Unicode(
1851 default_value=".completion_profiles",
1852 help="Template for path at which to output profile data for completions."
1853 ).tag(config=True)
1855 @observe('limit_to__all__')
1856 def _limit_to_all_changed(self, change):
1857 warnings.warn('`IPython.core.IPCompleter.limit_to__all__` configuration '
1858 'value has been deprecated since IPython 5.0, will be made to have '
1859 'no effects and then removed in future version of IPython.',
1860 UserWarning)
1862 def __init__(
1863 self, shell=None, namespace=None, global_namespace=None, config=None, **kwargs
1864 ):
1865 """IPCompleter() -> completer
1867 Return a completer object.
1869 Parameters
1870 ----------
1871 shell
1872 a pointer to the ipython shell itself. This is needed
1873 because this completer knows about magic functions, and those can
1874 only be accessed via the ipython instance.
1875 namespace : dict, optional
1876 an optional dict where completions are performed.
1877 global_namespace : dict, optional
1878 secondary optional dict for completions, to
1879 handle cases (such as IPython embedded inside functions) where
1880 both Python scopes are visible.
1881 config : Config
1882 traitlet's config object
1883 **kwargs
1884 passed to super class unmodified.
1885 """
1887 self.magic_escape = ESC_MAGIC
1888 self.splitter = CompletionSplitter()
1890 # _greedy_changed() depends on splitter and readline being defined:
1891 super().__init__(
1892 namespace=namespace,
1893 global_namespace=global_namespace,
1894 config=config,
1895 **kwargs,
1896 )
1898 # List where completion matches will be stored
1899 self.matches = []
1900 self.shell = shell
1901 # Regexp to split filenames with spaces in them
1902 self.space_name_re = re.compile(r'([^\\] )')
1903 # Hold a local ref. to glob.glob for speed
1904 self.glob = glob.glob
1906 # Determine if we are running on 'dumb' terminals, like (X)Emacs
1907 # buffers, to avoid completion problems.
1908 term = os.environ.get('TERM','xterm')
1909 self.dumb_terminal = term in ['dumb','emacs']
1911 # Special handling of backslashes needed in win32 platforms
1912 if sys.platform == "win32":
1913 self.clean_glob = self._clean_glob_win32
1914 else:
1915 self.clean_glob = self._clean_glob
1917 #regexp to parse docstring for function signature
1918 self.docstring_sig_re = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
1919 self.docstring_kwd_re = re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
1920 #use this if positional argument name is also needed
1921 #= re.compile(r'[\s|\[]*(\w+)(?:\s*=?\s*.*)')
1923 self.magic_arg_matchers = [
1924 self.magic_config_matcher,
1925 self.magic_color_matcher,
1926 ]
1928 # This is set externally by InteractiveShell
1929 self.custom_completers = None
1931 # This is a list of names of unicode characters that can be completed
1932 # into their corresponding unicode value. The list is large, so we
1933 # lazily initialize it on first use. Consuming code should access this
1934 # attribute through the `@unicode_names` property.
1935 self._unicode_names = None
1937 self._backslash_combining_matchers = [
1938 self.latex_name_matcher,
1939 self.unicode_name_matcher,
1940 back_latex_name_matcher,
1941 back_unicode_name_matcher,
1942 self.fwd_unicode_matcher,
1943 ]
1945 if not self.backslash_combining_completions:
1946 for matcher in self._backslash_combining_matchers:
1947 self.disable_matchers.append(_get_matcher_id(matcher))
1949 if not self.merge_completions:
1950 self.suppress_competing_matchers = True
1952 @property
1953 def matchers(self) -> List[Matcher]:
1954 """All active matcher routines for completion"""
1955 if self.dict_keys_only:
1956 return [self.dict_key_matcher]
1958 if self.use_jedi:
1959 return [
1960 *self.custom_matchers,
1961 *self._backslash_combining_matchers,
1962 *self.magic_arg_matchers,
1963 self.custom_completer_matcher,
1964 self.magic_matcher,
1965 self._jedi_matcher,
1966 self.dict_key_matcher,
1967 self.file_matcher,
1968 ]
1969 else:
1970 return [
1971 *self.custom_matchers,
1972 *self._backslash_combining_matchers,
1973 *self.magic_arg_matchers,
1974 self.custom_completer_matcher,
1975 self.dict_key_matcher,
1976 # TODO: convert python_matches to v2 API
1977 self.magic_matcher,
1978 self.python_matches,
1979 self.file_matcher,
1980 self.python_func_kw_matcher,
1981 ]
1983 def all_completions(self, text:str) -> List[str]:
1984 """
1985 Wrapper around the completion methods for the benefit of emacs.
1986 """
1987 prefix = text.rpartition('.')[0]
1988 with provisionalcompleter():
1989 return ['.'.join([prefix, c.text]) if prefix and self.use_jedi else c.text
1990 for c in self.completions(text, len(text))]
1992 return self.complete(text)[1]
1994 def _clean_glob(self, text:str):
1995 return self.glob("%s*" % text)
1997 def _clean_glob_win32(self, text:str):
1998 return [f.replace("\\","/")
1999 for f in self.glob("%s*" % text)]
2001 @context_matcher()
2002 def file_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2003 """Same as :any:`file_matches`, but adopted to new Matcher API."""
2004 matches = self.file_matches(context.token)
2005 # TODO: add a heuristic for suppressing (e.g. if it has OS-specific delimiter,
2006 # starts with `/home/`, `C:\`, etc)
2007 return _convert_matcher_v1_result_to_v2(matches, type="path")
2009 def file_matches(self, text: str) -> List[str]:
2010 """Match filenames, expanding ~USER type strings.
2012 Most of the seemingly convoluted logic in this completer is an
2013 attempt to handle filenames with spaces in them. And yet it's not
2014 quite perfect, because Python's readline doesn't expose all of the
2015 GNU readline details needed for this to be done correctly.
2017 For a filename with a space in it, the printed completions will be
2018 only the parts after what's already been typed (instead of the
2019 full completions, as is normally done). I don't think with the
2020 current (as of Python 2.3) Python readline it's possible to do
2021 better.
2023 .. deprecated:: 8.6
2024 You can use :meth:`file_matcher` instead.
2025 """
2027 # chars that require escaping with backslash - i.e. chars
2028 # that readline treats incorrectly as delimiters, but we
2029 # don't want to treat as delimiters in filename matching
2030 # when escaped with backslash
2031 if text.startswith('!'):
2032 text = text[1:]
2033 text_prefix = u'!'
2034 else:
2035 text_prefix = u''
2037 text_until_cursor = self.text_until_cursor
2038 # track strings with open quotes
2039 open_quotes = has_open_quotes(text_until_cursor)
2041 if '(' in text_until_cursor or '[' in text_until_cursor:
2042 lsplit = text
2043 else:
2044 try:
2045 # arg_split ~ shlex.split, but with unicode bugs fixed by us
2046 lsplit = arg_split(text_until_cursor)[-1]
2047 except ValueError:
2048 # typically an unmatched ", or backslash without escaped char.
2049 if open_quotes:
2050 lsplit = text_until_cursor.split(open_quotes)[-1]
2051 else:
2052 return []
2053 except IndexError:
2054 # tab pressed on empty line
2055 lsplit = ""
2057 if not open_quotes and lsplit != protect_filename(lsplit):
2058 # if protectables are found, do matching on the whole escaped name
2059 has_protectables = True
2060 text0,text = text,lsplit
2061 else:
2062 has_protectables = False
2063 text = os.path.expanduser(text)
2065 if text == "":
2066 return [text_prefix + protect_filename(f) for f in self.glob("*")]
2068 # Compute the matches from the filesystem
2069 if sys.platform == 'win32':
2070 m0 = self.clean_glob(text)
2071 else:
2072 m0 = self.clean_glob(text.replace('\\', ''))
2074 if has_protectables:
2075 # If we had protectables, we need to revert our changes to the
2076 # beginning of filename so that we don't double-write the part
2077 # of the filename we have so far
2078 len_lsplit = len(lsplit)
2079 matches = [text_prefix + text0 +
2080 protect_filename(f[len_lsplit:]) for f in m0]
2081 else:
2082 if open_quotes:
2083 # if we have a string with an open quote, we don't need to
2084 # protect the names beyond the quote (and we _shouldn't_, as
2085 # it would cause bugs when the filesystem call is made).
2086 matches = m0 if sys.platform == "win32" else\
2087 [protect_filename(f, open_quotes) for f in m0]
2088 else:
2089 matches = [text_prefix +
2090 protect_filename(f) for f in m0]
2092 # Mark directories in input list by appending '/' to their names.
2093 return [x+'/' if os.path.isdir(x) else x for x in matches]
2095 @context_matcher()
2096 def magic_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2097 """Match magics."""
2098 text = context.token
2099 matches = self.magic_matches(text)
2100 result = _convert_matcher_v1_result_to_v2(matches, type="magic")
2101 is_magic_prefix = len(text) > 0 and text[0] == "%"
2102 result["suppress"] = is_magic_prefix and bool(result["completions"])
2103 return result
2105 def magic_matches(self, text: str):
2106 """Match magics.
2108 .. deprecated:: 8.6
2109 You can use :meth:`magic_matcher` instead.
2110 """
2111 # Get all shell magics now rather than statically, so magics loaded at
2112 # runtime show up too.
2113 lsm = self.shell.magics_manager.lsmagic()
2114 line_magics = lsm['line']
2115 cell_magics = lsm['cell']
2116 pre = self.magic_escape
2117 pre2 = pre+pre
2119 explicit_magic = text.startswith(pre)
2121 # Completion logic:
2122 # - user gives %%: only do cell magics
2123 # - user gives %: do both line and cell magics
2124 # - no prefix: do both
2125 # In other words, line magics are skipped if the user gives %% explicitly
2126 #
2127 # We also exclude magics that match any currently visible names:
2128 # https://github.com/ipython/ipython/issues/4877, unless the user has
2129 # typed a %:
2130 # https://github.com/ipython/ipython/issues/10754
2131 bare_text = text.lstrip(pre)
2132 global_matches = self.global_matches(bare_text)
2133 if not explicit_magic:
2134 def matches(magic):
2135 """
2136 Filter magics, in particular remove magics that match
2137 a name present in global namespace.
2138 """
2139 return ( magic.startswith(bare_text) and
2140 magic not in global_matches )
2141 else:
2142 def matches(magic):
2143 return magic.startswith(bare_text)
2145 comp = [ pre2+m for m in cell_magics if matches(m)]
2146 if not text.startswith(pre2):
2147 comp += [ pre+m for m in line_magics if matches(m)]
2149 return comp
2151 @context_matcher()
2152 def magic_config_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2153 """Match class names and attributes for %config magic."""
2154 # NOTE: uses `line_buffer` equivalent for compatibility
2155 matches = self.magic_config_matches(context.line_with_cursor)
2156 return _convert_matcher_v1_result_to_v2(matches, type="param")
2158 def magic_config_matches(self, text: str) -> List[str]:
2159 """Match class names and attributes for %config magic.
2161 .. deprecated:: 8.6
2162 You can use :meth:`magic_config_matcher` instead.
2163 """
2164 texts = text.strip().split()
2166 if len(texts) > 0 and (texts[0] == 'config' or texts[0] == '%config'):
2167 # get all configuration classes
2168 classes = sorted(set([ c for c in self.shell.configurables
2169 if c.__class__.class_traits(config=True)
2170 ]), key=lambda x: x.__class__.__name__)
2171 classnames = [ c.__class__.__name__ for c in classes ]
2173 # return all classnames if config or %config is given
2174 if len(texts) == 1:
2175 return classnames
2177 # match classname
2178 classname_texts = texts[1].split('.')
2179 classname = classname_texts[0]
2180 classname_matches = [ c for c in classnames
2181 if c.startswith(classname) ]
2183 # return matched classes or the matched class with attributes
2184 if texts[1].find('.') < 0:
2185 return classname_matches
2186 elif len(classname_matches) == 1 and \
2187 classname_matches[0] == classname:
2188 cls = classes[classnames.index(classname)].__class__
2189 help = cls.class_get_help()
2190 # strip leading '--' from cl-args:
2191 help = re.sub(re.compile(r'^--', re.MULTILINE), '', help)
2192 return [ attr.split('=')[0]
2193 for attr in help.strip().splitlines()
2194 if attr.startswith(texts[1]) ]
2195 return []
2197 @context_matcher()
2198 def magic_color_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2199 """Match color schemes for %colors magic."""
2200 # NOTE: uses `line_buffer` equivalent for compatibility
2201 matches = self.magic_color_matches(context.line_with_cursor)
2202 return _convert_matcher_v1_result_to_v2(matches, type="param")
2204 def magic_color_matches(self, text: str) -> List[str]:
2205 """Match color schemes for %colors magic.
2207 .. deprecated:: 8.6
2208 You can use :meth:`magic_color_matcher` instead.
2209 """
2210 texts = text.split()
2211 if text.endswith(' '):
2212 # .split() strips off the trailing whitespace. Add '' back
2213 # so that: '%colors ' -> ['%colors', '']
2214 texts.append('')
2216 if len(texts) == 2 and (texts[0] == 'colors' or texts[0] == '%colors'):
2217 prefix = texts[1]
2218 return [ color for color in InspectColors.keys()
2219 if color.startswith(prefix) ]
2220 return []
2222 @context_matcher(identifier="IPCompleter.jedi_matcher")
2223 def _jedi_matcher(self, context: CompletionContext) -> _JediMatcherResult:
2224 matches = self._jedi_matches(
2225 cursor_column=context.cursor_position,
2226 cursor_line=context.cursor_line,
2227 text=context.full_text,
2228 )
2229 return {
2230 "completions": matches,
2231 # static analysis should not suppress other matchers
2232 "suppress": False,
2233 }
2235 def _jedi_matches(
2236 self, cursor_column: int, cursor_line: int, text: str
2237 ) -> Iterator[_JediCompletionLike]:
2238 """
2239 Return a list of :any:`jedi.api.Completion`s object from a ``text`` and
2240 cursor position.
2242 Parameters
2243 ----------
2244 cursor_column : int
2245 column position of the cursor in ``text``, 0-indexed.
2246 cursor_line : int
2247 line position of the cursor in ``text``, 0-indexed
2248 text : str
2249 text to complete
2251 Notes
2252 -----
2253 If ``IPCompleter.debug`` is ``True`` may return a :any:`_FakeJediCompletion`
2254 object containing a string with the Jedi debug information attached.
2256 .. deprecated:: 8.6
2257 You can use :meth:`_jedi_matcher` instead.
2258 """
2259 namespaces = [self.namespace]
2260 if self.global_namespace is not None:
2261 namespaces.append(self.global_namespace)
2263 completion_filter = lambda x:x
2264 offset = cursor_to_position(text, cursor_line, cursor_column)
2265 # filter output if we are completing for object members
2266 if offset:
2267 pre = text[offset-1]
2268 if pre == '.':
2269 if self.omit__names == 2:
2270 completion_filter = lambda c:not c.name.startswith('_')
2271 elif self.omit__names == 1:
2272 completion_filter = lambda c:not (c.name.startswith('__') and c.name.endswith('__'))
2273 elif self.omit__names == 0:
2274 completion_filter = lambda x:x
2275 else:
2276 raise ValueError("Don't understand self.omit__names == {}".format(self.omit__names))
2278 interpreter = jedi.Interpreter(text[:offset], namespaces)
2279 try_jedi = True
2281 try:
2282 # find the first token in the current tree -- if it is a ' or " then we are in a string
2283 completing_string = False
2284 try:
2285 first_child = next(c for c in interpreter._get_module().tree_node.children if hasattr(c, 'value'))
2286 except StopIteration:
2287 pass
2288 else:
2289 # note the value may be ', ", or it may also be ''' or """, or
2290 # in some cases, """what/you/typed..., but all of these are
2291 # strings.
2292 completing_string = len(first_child.value) > 0 and first_child.value[0] in {"'", '"'}
2294 # if we are in a string jedi is likely not the right candidate for
2295 # now. Skip it.
2296 try_jedi = not completing_string
2297 except Exception as e:
2298 # many of things can go wrong, we are using private API just don't crash.
2299 if self.debug:
2300 print("Error detecting if completing a non-finished string :", e, '|')
2302 if not try_jedi:
2303 return iter([])
2304 try:
2305 return filter(completion_filter, interpreter.complete(column=cursor_column, line=cursor_line + 1))
2306 except Exception as e:
2307 if self.debug:
2308 return iter(
2309 [
2310 _FakeJediCompletion(
2311 'Oops Jedi has crashed, please report a bug with the following:\n"""\n%s\ns"""'
2312 % (e)
2313 )
2314 ]
2315 )
2316 else:
2317 return iter([])
2319 @completion_matcher(api_version=1)
2320 def python_matches(self, text: str) -> Iterable[str]:
2321 """Match attributes or global python names"""
2322 if "." in text:
2323 try:
2324 matches = self.attr_matches(text)
2325 if text.endswith('.') and self.omit__names:
2326 if self.omit__names == 1:
2327 # true if txt is _not_ a __ name, false otherwise:
2328 no__name = (lambda txt:
2329 re.match(r'.*\.__.*?__',txt) is None)
2330 else:
2331 # true if txt is _not_ a _ name, false otherwise:
2332 no__name = (lambda txt:
2333 re.match(r'\._.*?',txt[txt.rindex('.'):]) is None)
2334 matches = filter(no__name, matches)
2335 except NameError:
2336 # catches <undefined attributes>.<tab>
2337 matches = []
2338 else:
2339 matches = self.global_matches(text)
2340 return matches
2342 def _default_arguments_from_docstring(self, doc):
2343 """Parse the first line of docstring for call signature.
2345 Docstring should be of the form 'min(iterable[, key=func])\n'.
2346 It can also parse cython docstring of the form
2347 'Minuit.migrad(self, int ncall=10000, resume=True, int nsplit=1)'.
2348 """
2349 if doc is None:
2350 return []
2352 #care only the firstline
2353 line = doc.lstrip().splitlines()[0]
2355 #p = re.compile(r'^[\w|\s.]+\(([^)]*)\).*')
2356 #'min(iterable[, key=func])\n' -> 'iterable[, key=func]'
2357 sig = self.docstring_sig_re.search(line)
2358 if sig is None:
2359 return []
2360 # iterable[, key=func]' -> ['iterable[' ,' key=func]']
2361 sig = sig.groups()[0].split(',')
2362 ret = []
2363 for s in sig:
2364 #re.compile(r'[\s|\[]*(\w+)(?:\s*=\s*.*)')
2365 ret += self.docstring_kwd_re.findall(s)
2366 return ret
2368 def _default_arguments(self, obj):
2369 """Return the list of default arguments of obj if it is callable,
2370 or empty list otherwise."""
2371 call_obj = obj
2372 ret = []
2373 if inspect.isbuiltin(obj):
2374 pass
2375 elif not (inspect.isfunction(obj) or inspect.ismethod(obj)):
2376 if inspect.isclass(obj):
2377 #for cython embedsignature=True the constructor docstring
2378 #belongs to the object itself not __init__
2379 ret += self._default_arguments_from_docstring(
2380 getattr(obj, '__doc__', ''))
2381 # for classes, check for __init__,__new__
2382 call_obj = (getattr(obj, '__init__', None) or
2383 getattr(obj, '__new__', None))
2384 # for all others, check if they are __call__able
2385 elif hasattr(obj, '__call__'):
2386 call_obj = obj.__call__
2387 ret += self._default_arguments_from_docstring(
2388 getattr(call_obj, '__doc__', ''))
2390 _keeps = (inspect.Parameter.KEYWORD_ONLY,
2391 inspect.Parameter.POSITIONAL_OR_KEYWORD)
2393 try:
2394 sig = inspect.signature(obj)
2395 ret.extend(k for k, v in sig.parameters.items() if
2396 v.kind in _keeps)
2397 except ValueError:
2398 pass
2400 return list(set(ret))
2402 @context_matcher()
2403 def python_func_kw_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2404 """Match named parameters (kwargs) of the last open function."""
2405 matches = self.python_func_kw_matches(context.token)
2406 return _convert_matcher_v1_result_to_v2(matches, type="param")
2408 def python_func_kw_matches(self, text):
2409 """Match named parameters (kwargs) of the last open function.
2411 .. deprecated:: 8.6
2412 You can use :meth:`python_func_kw_matcher` instead.
2413 """
2415 if "." in text: # a parameter cannot be dotted
2416 return []
2417 try: regexp = self.__funcParamsRegex
2418 except AttributeError:
2419 regexp = self.__funcParamsRegex = re.compile(r'''
2420 '.*?(?<!\\)' | # single quoted strings or
2421 ".*?(?<!\\)" | # double quoted strings or
2422 \w+ | # identifier
2423 \S # other characters
2424 ''', re.VERBOSE | re.DOTALL)
2425 # 1. find the nearest identifier that comes before an unclosed
2426 # parenthesis before the cursor
2427 # e.g. for "foo (1+bar(x), pa<cursor>,a=1)", the candidate is "foo"
2428 tokens = regexp.findall(self.text_until_cursor)
2429 iterTokens = reversed(tokens); openPar = 0
2431 for token in iterTokens:
2432 if token == ')':
2433 openPar -= 1
2434 elif token == '(':
2435 openPar += 1
2436 if openPar > 0:
2437 # found the last unclosed parenthesis
2438 break
2439 else:
2440 return []
2441 # 2. Concatenate dotted names ("foo.bar" for "foo.bar(x, pa" )
2442 ids = []
2443 isId = re.compile(r'\w+$').match
2445 while True:
2446 try:
2447 ids.append(next(iterTokens))
2448 if not isId(ids[-1]):
2449 ids.pop(); break
2450 if not next(iterTokens) == '.':
2451 break
2452 except StopIteration:
2453 break
2455 # Find all named arguments already assigned to, as to avoid suggesting
2456 # them again
2457 usedNamedArgs = set()
2458 par_level = -1
2459 for token, next_token in zip(tokens, tokens[1:]):
2460 if token == '(':
2461 par_level += 1
2462 elif token == ')':
2463 par_level -= 1
2465 if par_level != 0:
2466 continue
2468 if next_token != '=':
2469 continue
2471 usedNamedArgs.add(token)
2473 argMatches = []
2474 try:
2475 callableObj = '.'.join(ids[::-1])
2476 namedArgs = self._default_arguments(eval(callableObj,
2477 self.namespace))
2479 # Remove used named arguments from the list, no need to show twice
2480 for namedArg in set(namedArgs) - usedNamedArgs:
2481 if namedArg.startswith(text):
2482 argMatches.append("%s=" %namedArg)
2483 except:
2484 pass
2486 return argMatches
2488 @staticmethod
2489 def _get_keys(obj: Any) -> List[Any]:
2490 # Objects can define their own completions by defining an
2491 # _ipy_key_completions_() method.
2492 method = get_real_method(obj, '_ipython_key_completions_')
2493 if method is not None:
2494 return method()
2496 # Special case some common in-memory dict-like types
2497 if isinstance(obj, dict) or _safe_isinstance(obj, "pandas", "DataFrame"):
2498 try:
2499 return list(obj.keys())
2500 except Exception:
2501 return []
2502 elif _safe_isinstance(obj, "pandas", "core", "indexing", "_LocIndexer"):
2503 try:
2504 return list(obj.obj.keys())
2505 except Exception:
2506 return []
2507 elif _safe_isinstance(obj, 'numpy', 'ndarray') or\
2508 _safe_isinstance(obj, 'numpy', 'void'):
2509 return obj.dtype.names or []
2510 return []
2512 @context_matcher()
2513 def dict_key_matcher(self, context: CompletionContext) -> SimpleMatcherResult:
2514 """Match string keys in a dictionary, after e.g. ``foo[``."""
2515 matches = self.dict_key_matches(context.token)
2516 return _convert_matcher_v1_result_to_v2(
2517 matches, type="dict key", suppress_if_matches=True
2518 )
2520 def dict_key_matches(self, text: str) -> List[str]:
2521 """Match string keys in a dictionary, after e.g. ``foo[``.
2523 .. deprecated:: 8.6
2524 You can use :meth:`dict_key_matcher` instead.
2525 """
2527 # Short-circuit on closed dictionary (regular expression would
2528 # not match anyway, but would take quite a while).
2529 if self.text_until_cursor.strip().endswith("]"):
2530 return []
2532 match = DICT_MATCHER_REGEX.search(self.text_until_cursor)
2534 if match is None:
2535 return []
2537 expr, prior_tuple_keys, key_prefix = match.groups()
2539 obj = self._evaluate_expr(expr)
2541 if obj is not_found:
2542 return []
2544 keys = self._get_keys(obj)
2545 if not keys:
2546 return keys
2548 tuple_prefix = guarded_eval(
2549 prior_tuple_keys,
2550 EvaluationContext(
2551 globals=self.global_namespace,
2552 locals=self.namespace,
2553 evaluation=self.evaluation,
2554 in_subscript=True,
2555 ),
2556 )
2558 closing_quote, token_offset, matches = match_dict_keys(
2559 keys, key_prefix, self.splitter.delims, extra_prefix=tuple_prefix
2560 )
2561 if not matches:
2562 return []
2564 # get the cursor position of
2565 # - the text being completed
2566 # - the start of the key text
2567 # - the start of the completion
2568 text_start = len(self.text_until_cursor) - len(text)
2569 if key_prefix:
2570 key_start = match.start(3)
2571 completion_start = key_start + token_offset
2572 else:
2573 key_start = completion_start = match.end()
2575 # grab the leading prefix, to make sure all completions start with `text`
2576 if text_start > key_start:
2577 leading = ''
2578 else:
2579 leading = text[text_start:completion_start]
2581 # append closing quote and bracket as appropriate
2582 # this is *not* appropriate if the opening quote or bracket is outside
2583 # the text given to this method, e.g. `d["""a\nt
2584 can_close_quote = False
2585 can_close_bracket = False
2587 continuation = self.line_buffer[len(self.text_until_cursor) :].strip()
2589 if continuation.startswith(closing_quote):
2590 # do not close if already closed, e.g. `d['a<tab>'`
2591 continuation = continuation[len(closing_quote) :]
2592 else:
2593 can_close_quote = True
2595 continuation = continuation.strip()
2597 # e.g. `pandas.DataFrame` has different tuple indexer behaviour,
2598 # handling it is out of scope, so let's avoid appending suffixes.
2599 has_known_tuple_handling = isinstance(obj, dict)
2601 can_close_bracket = (
2602 not continuation.startswith("]") and self.auto_close_dict_keys
2603 )
2604 can_close_tuple_item = (
2605 not continuation.startswith(",")
2606 and has_known_tuple_handling
2607 and self.auto_close_dict_keys
2608 )
2609 can_close_quote = can_close_quote and self.auto_close_dict_keys
2611 # fast path if closing qoute should be appended but not suffix is allowed
2612 if not can_close_quote and not can_close_bracket and closing_quote:
2613 return [leading + k for k in matches]
2615 results = []
2617 end_of_tuple_or_item = _DictKeyState.END_OF_TUPLE | _DictKeyState.END_OF_ITEM
2619 for k, state_flag in matches.items():
2620 result = leading + k
2621 if can_close_quote and closing_quote:
2622 result += closing_quote
2624 if state_flag == end_of_tuple_or_item:
2625 # We do not know which suffix to add,
2626 # e.g. both tuple item and string
2627 # match this item.
2628 pass
2630 if state_flag in end_of_tuple_or_item and can_close_bracket:
2631 result += "]"
2632 if state_flag == _DictKeyState.IN_TUPLE and can_close_tuple_item:
2633 result += ", "
2634 results.append(result)
2635 return results
2637 @context_matcher()
2638 def unicode_name_matcher(self, context: CompletionContext):
2639 """Same as :any:`unicode_name_matches`, but adopted to new Matcher API."""
2640 fragment, matches = self.unicode_name_matches(context.text_until_cursor)
2641 return _convert_matcher_v1_result_to_v2(
2642 matches, type="unicode", fragment=fragment, suppress_if_matches=True
2643 )
2645 @staticmethod
2646 def unicode_name_matches(text: str) -> Tuple[str, List[str]]:
2647 """Match Latex-like syntax for unicode characters base
2648 on the name of the character.
2650 This does ``\\GREEK SMALL LETTER ETA`` -> ``η``
2652 Works only on valid python 3 identifier, or on combining characters that
2653 will combine to form a valid identifier.
2654 """
2655 slashpos = text.rfind('\\')
2656 if slashpos > -1:
2657 s = text[slashpos+1:]
2658 try :
2659 unic = unicodedata.lookup(s)
2660 # allow combining chars
2661 if ('a'+unic).isidentifier():
2662 return '\\'+s,[unic]
2663 except KeyError:
2664 pass
2665 return '', []
2667 @context_matcher()
2668 def latex_name_matcher(self, context: CompletionContext):
2669 """Match Latex syntax for unicode characters.
2671 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2672 """
2673 fragment, matches = self.latex_matches(context.text_until_cursor)
2674 return _convert_matcher_v1_result_to_v2(
2675 matches, type="latex", fragment=fragment, suppress_if_matches=True
2676 )
2678 def latex_matches(self, text: str) -> Tuple[str, Sequence[str]]:
2679 """Match Latex syntax for unicode characters.
2681 This does both ``\\alp`` -> ``\\alpha`` and ``\\alpha`` -> ``α``
2683 .. deprecated:: 8.6
2684 You can use :meth:`latex_name_matcher` instead.
2685 """
2686 slashpos = text.rfind('\\')
2687 if slashpos > -1:
2688 s = text[slashpos:]
2689 if s in latex_symbols:
2690 # Try to complete a full latex symbol to unicode
2691 # \\alpha -> α
2692 return s, [latex_symbols[s]]
2693 else:
2694 # If a user has partially typed a latex symbol, give them
2695 # a full list of options \al -> [\aleph, \alpha]
2696 matches = [k for k in latex_symbols if k.startswith(s)]
2697 if matches:
2698 return s, matches
2699 return '', ()
2701 @context_matcher()
2702 def custom_completer_matcher(self, context):
2703 """Dispatch custom completer.
2705 If a match is found, suppresses all other matchers except for Jedi.
2706 """
2707 matches = self.dispatch_custom_completer(context.token) or []
2708 result = _convert_matcher_v1_result_to_v2(
2709 matches, type=_UNKNOWN_TYPE, suppress_if_matches=True
2710 )
2711 result["ordered"] = True
2712 result["do_not_suppress"] = {_get_matcher_id(self._jedi_matcher)}
2713 return result
2715 def dispatch_custom_completer(self, text):
2716 """
2717 .. deprecated:: 8.6
2718 You can use :meth:`custom_completer_matcher` instead.
2719 """
2720 if not self.custom_completers:
2721 return
2723 line = self.line_buffer
2724 if not line.strip():
2725 return None
2727 # Create a little structure to pass all the relevant information about
2728 # the current completion to any custom completer.
2729 event = SimpleNamespace()
2730 event.line = line
2731 event.symbol = text
2732 cmd = line.split(None,1)[0]
2733 event.command = cmd
2734 event.text_until_cursor = self.text_until_cursor
2736 # for foo etc, try also to find completer for %foo
2737 if not cmd.startswith(self.magic_escape):
2738 try_magic = self.custom_completers.s_matches(
2739 self.magic_escape + cmd)
2740 else:
2741 try_magic = []
2743 for c in itertools.chain(self.custom_completers.s_matches(cmd),
2744 try_magic,
2745 self.custom_completers.flat_matches(self.text_until_cursor)):
2746 try:
2747 res = c(event)
2748 if res:
2749 # first, try case sensitive match
2750 withcase = [r for r in res if r.startswith(text)]
2751 if withcase:
2752 return withcase
2753 # if none, then case insensitive ones are ok too
2754 text_low = text.lower()
2755 return [r for r in res if r.lower().startswith(text_low)]
2756 except TryNext:
2757 pass
2758 except KeyboardInterrupt:
2759 """
2760 If custom completer take too long,
2761 let keyboard interrupt abort and return nothing.
2762 """
2763 break
2765 return None
2767 def completions(self, text: str, offset: int)->Iterator[Completion]:
2768 """
2769 Returns an iterator over the possible completions
2771 .. warning::
2773 Unstable
2775 This function is unstable, API may change without warning.
2776 It will also raise unless use in proper context manager.
2778 Parameters
2779 ----------
2780 text : str
2781 Full text of the current input, multi line string.
2782 offset : int
2783 Integer representing the position of the cursor in ``text``. Offset
2784 is 0-based indexed.
2786 Yields
2787 ------
2788 Completion
2790 Notes
2791 -----
2792 The cursor on a text can either be seen as being "in between"
2793 characters or "On" a character depending on the interface visible to
2794 the user. For consistency the cursor being on "in between" characters X
2795 and Y is equivalent to the cursor being "on" character Y, that is to say
2796 the character the cursor is on is considered as being after the cursor.
2798 Combining characters may span more that one position in the
2799 text.
2801 .. note::
2803 If ``IPCompleter.debug`` is :any:`True` will yield a ``--jedi/ipython--``
2804 fake Completion token to distinguish completion returned by Jedi
2805 and usual IPython completion.
2807 .. note::
2809 Completions are not completely deduplicated yet. If identical
2810 completions are coming from different sources this function does not
2811 ensure that each completion object will only be present once.
2812 """
2813 warnings.warn("_complete is a provisional API (as of IPython 6.0). "
2814 "It may change without warnings. "
2815 "Use in corresponding context manager.",
2816 category=ProvisionalCompleterWarning, stacklevel=2)
2818 seen = set()
2819 profiler:Optional[cProfile.Profile]
2820 try:
2821 if self.profile_completions:
2822 import cProfile
2823 profiler = cProfile.Profile()
2824 profiler.enable()
2825 else:
2826 profiler = None
2828 for c in self._completions(text, offset, _timeout=self.jedi_compute_type_timeout/1000):
2829 if c and (c in seen):
2830 continue
2831 yield c
2832 seen.add(c)
2833 except KeyboardInterrupt:
2834 """if completions take too long and users send keyboard interrupt,
2835 do not crash and return ASAP. """
2836 pass
2837 finally:
2838 if profiler is not None:
2839 profiler.disable()
2840 ensure_dir_exists(self.profiler_output_dir)
2841 output_path = os.path.join(self.profiler_output_dir, str(uuid.uuid4()))
2842 print("Writing profiler output to", output_path)
2843 profiler.dump_stats(output_path)
2845 def _completions(self, full_text: str, offset: int, *, _timeout) -> Iterator[Completion]:
2846 """
2847 Core completion module.Same signature as :any:`completions`, with the
2848 extra `timeout` parameter (in seconds).
2850 Computing jedi's completion ``.type`` can be quite expensive (it is a
2851 lazy property) and can require some warm-up, more warm up than just
2852 computing the ``name`` of a completion. The warm-up can be :
2854 - Long warm-up the first time a module is encountered after
2855 install/update: actually build parse/inference tree.
2857 - first time the module is encountered in a session: load tree from
2858 disk.
2860 We don't want to block completions for tens of seconds so we give the
2861 completer a "budget" of ``_timeout`` seconds per invocation to compute
2862 completions types, the completions that have not yet been computed will
2863 be marked as "unknown" an will have a chance to be computed next round
2864 are things get cached.
2866 Keep in mind that Jedi is not the only thing treating the completion so
2867 keep the timeout short-ish as if we take more than 0.3 second we still
2868 have lots of processing to do.
2870 """
2871 deadline = time.monotonic() + _timeout
2873 before = full_text[:offset]
2874 cursor_line, cursor_column = position_to_cursor(full_text, offset)
2876 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
2878 def is_non_jedi_result(
2879 result: MatcherResult, identifier: str
2880 ) -> TypeGuard[SimpleMatcherResult]:
2881 return identifier != jedi_matcher_id
2883 results = self._complete(
2884 full_text=full_text, cursor_line=cursor_line, cursor_pos=cursor_column
2885 )
2887 non_jedi_results: Dict[str, SimpleMatcherResult] = {
2888 identifier: result
2889 for identifier, result in results.items()
2890 if is_non_jedi_result(result, identifier)
2891 }
2893 jedi_matches = (
2894 cast(_JediMatcherResult, results[jedi_matcher_id])["completions"]
2895 if jedi_matcher_id in results
2896 else ()
2897 )
2899 iter_jm = iter(jedi_matches)
2900 if _timeout:
2901 for jm in iter_jm:
2902 try:
2903 type_ = jm.type
2904 except Exception:
2905 if self.debug:
2906 print("Error in Jedi getting type of ", jm)
2907 type_ = None
2908 delta = len(jm.name_with_symbols) - len(jm.complete)
2909 if type_ == 'function':
2910 signature = _make_signature(jm)
2911 else:
2912 signature = ''
2913 yield Completion(start=offset - delta,
2914 end=offset,
2915 text=jm.name_with_symbols,
2916 type=type_,
2917 signature=signature,
2918 _origin='jedi')
2920 if time.monotonic() > deadline:
2921 break
2923 for jm in iter_jm:
2924 delta = len(jm.name_with_symbols) - len(jm.complete)
2925 yield Completion(
2926 start=offset - delta,
2927 end=offset,
2928 text=jm.name_with_symbols,
2929 type=_UNKNOWN_TYPE, # don't compute type for speed
2930 _origin="jedi",
2931 signature="",
2932 )
2934 # TODO:
2935 # Suppress this, right now just for debug.
2936 if jedi_matches and non_jedi_results and self.debug:
2937 some_start_offset = before.rfind(
2938 next(iter(non_jedi_results.values()))["matched_fragment"]
2939 )
2940 yield Completion(
2941 start=some_start_offset,
2942 end=offset,
2943 text="--jedi/ipython--",
2944 _origin="debug",
2945 type="none",
2946 signature="",
2947 )
2949 ordered: List[Completion] = []
2950 sortable: List[Completion] = []
2952 for origin, result in non_jedi_results.items():
2953 matched_text = result["matched_fragment"]
2954 start_offset = before.rfind(matched_text)
2955 is_ordered = result.get("ordered", False)
2956 container = ordered if is_ordered else sortable
2958 # I'm unsure if this is always true, so let's assert and see if it
2959 # crash
2960 assert before.endswith(matched_text)
2962 for simple_completion in result["completions"]:
2963 completion = Completion(
2964 start=start_offset,
2965 end=offset,
2966 text=simple_completion.text,
2967 _origin=origin,
2968 signature="",
2969 type=simple_completion.type or _UNKNOWN_TYPE,
2970 )
2971 container.append(completion)
2973 yield from list(self._deduplicate(ordered + self._sort(sortable)))[
2974 :MATCHES_LIMIT
2975 ]
2977 def complete(self, text=None, line_buffer=None, cursor_pos=None) -> Tuple[str, Sequence[str]]:
2978 """Find completions for the given text and line context.
2980 Note that both the text and the line_buffer are optional, but at least
2981 one of them must be given.
2983 Parameters
2984 ----------
2985 text : string, optional
2986 Text to perform the completion on. If not given, the line buffer
2987 is split using the instance's CompletionSplitter object.
2988 line_buffer : string, optional
2989 If not given, the completer attempts to obtain the current line
2990 buffer via readline. This keyword allows clients which are
2991 requesting for text completions in non-readline contexts to inform
2992 the completer of the entire text.
2993 cursor_pos : int, optional
2994 Index of the cursor in the full line buffer. Should be provided by
2995 remote frontends where kernel has no access to frontend state.
2997 Returns
2998 -------
2999 Tuple of two items:
3000 text : str
3001 Text that was actually used in the completion.
3002 matches : list
3003 A list of completion matches.
3005 Notes
3006 -----
3007 This API is likely to be deprecated and replaced by
3008 :any:`IPCompleter.completions` in the future.
3010 """
3011 warnings.warn('`Completer.complete` is pending deprecation since '
3012 'IPython 6.0 and will be replaced by `Completer.completions`.',
3013 PendingDeprecationWarning)
3014 # potential todo, FOLD the 3rd throw away argument of _complete
3015 # into the first 2 one.
3016 # TODO: Q: does the above refer to jedi completions (i.e. 0-indexed?)
3017 # TODO: should we deprecate now, or does it stay?
3019 results = self._complete(
3020 line_buffer=line_buffer, cursor_pos=cursor_pos, text=text, cursor_line=0
3021 )
3023 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3025 return self._arrange_and_extract(
3026 results,
3027 # TODO: can we confirm that excluding Jedi here was a deliberate choice in previous version?
3028 skip_matchers={jedi_matcher_id},
3029 # this API does not support different start/end positions (fragments of token).
3030 abort_if_offset_changes=True,
3031 )
3033 def _arrange_and_extract(
3034 self,
3035 results: Dict[str, MatcherResult],
3036 skip_matchers: Set[str],
3037 abort_if_offset_changes: bool,
3038 ):
3039 sortable: List[AnyMatcherCompletion] = []
3040 ordered: List[AnyMatcherCompletion] = []
3041 most_recent_fragment = None
3042 for identifier, result in results.items():
3043 if identifier in skip_matchers:
3044 continue
3045 if not result["completions"]:
3046 continue
3047 if not most_recent_fragment:
3048 most_recent_fragment = result["matched_fragment"]
3049 if (
3050 abort_if_offset_changes
3051 and result["matched_fragment"] != most_recent_fragment
3052 ):
3053 break
3054 if result.get("ordered", False):
3055 ordered.extend(result["completions"])
3056 else:
3057 sortable.extend(result["completions"])
3059 if not most_recent_fragment:
3060 most_recent_fragment = "" # to satisfy typechecker (and just in case)
3062 return most_recent_fragment, [
3063 m.text for m in self._deduplicate(ordered + self._sort(sortable))
3064 ]
3066 def _complete(self, *, cursor_line, cursor_pos, line_buffer=None, text=None,
3067 full_text=None) -> _CompleteResult:
3068 """
3069 Like complete but can also returns raw jedi completions as well as the
3070 origin of the completion text. This could (and should) be made much
3071 cleaner but that will be simpler once we drop the old (and stateful)
3072 :any:`complete` API.
3074 With current provisional API, cursor_pos act both (depending on the
3075 caller) as the offset in the ``text`` or ``line_buffer``, or as the
3076 ``column`` when passing multiline strings this could/should be renamed
3077 but would add extra noise.
3079 Parameters
3080 ----------
3081 cursor_line
3082 Index of the line the cursor is on. 0 indexed.
3083 cursor_pos
3084 Position of the cursor in the current line/line_buffer/text. 0
3085 indexed.
3086 line_buffer : optional, str
3087 The current line the cursor is in, this is mostly due to legacy
3088 reason that readline could only give a us the single current line.
3089 Prefer `full_text`.
3090 text : str
3091 The current "token" the cursor is in, mostly also for historical
3092 reasons. as the completer would trigger only after the current line
3093 was parsed.
3094 full_text : str
3095 Full text of the current cell.
3097 Returns
3098 -------
3099 An ordered dictionary where keys are identifiers of completion
3100 matchers and values are ``MatcherResult``s.
3101 """
3103 # if the cursor position isn't given, the only sane assumption we can
3104 # make is that it's at the end of the line (the common case)
3105 if cursor_pos is None:
3106 cursor_pos = len(line_buffer) if text is None else len(text)
3108 if self.use_main_ns:
3109 self.namespace = __main__.__dict__
3111 # if text is either None or an empty string, rely on the line buffer
3112 if (not line_buffer) and full_text:
3113 line_buffer = full_text.split('\n')[cursor_line]
3114 if not text: # issue #11508: check line_buffer before calling split_line
3115 text = (
3116 self.splitter.split_line(line_buffer, cursor_pos) if line_buffer else ""
3117 )
3119 # If no line buffer is given, assume the input text is all there was
3120 if line_buffer is None:
3121 line_buffer = text
3123 # deprecated - do not use `line_buffer` in new code.
3124 self.line_buffer = line_buffer
3125 self.text_until_cursor = self.line_buffer[:cursor_pos]
3127 if not full_text:
3128 full_text = line_buffer
3130 context = CompletionContext(
3131 full_text=full_text,
3132 cursor_position=cursor_pos,
3133 cursor_line=cursor_line,
3134 token=text,
3135 limit=MATCHES_LIMIT,
3136 )
3138 # Start with a clean slate of completions
3139 results: Dict[str, MatcherResult] = {}
3141 jedi_matcher_id = _get_matcher_id(self._jedi_matcher)
3143 suppressed_matchers: Set[str] = set()
3145 matchers = {
3146 _get_matcher_id(matcher): matcher
3147 for matcher in sorted(
3148 self.matchers, key=_get_matcher_priority, reverse=True
3149 )
3150 }
3152 for matcher_id, matcher in matchers.items():
3153 matcher_id = _get_matcher_id(matcher)
3155 if matcher_id in self.disable_matchers:
3156 continue
3158 if matcher_id in results:
3159 warnings.warn(f"Duplicate matcher ID: {matcher_id}.")
3161 if matcher_id in suppressed_matchers:
3162 continue
3164 result: MatcherResult
3165 try:
3166 if _is_matcher_v1(matcher):
3167 result = _convert_matcher_v1_result_to_v2(
3168 matcher(text), type=_UNKNOWN_TYPE
3169 )
3170 elif _is_matcher_v2(matcher):
3171 result = matcher(context)
3172 else:
3173 api_version = _get_matcher_api_version(matcher)
3174 raise ValueError(f"Unsupported API version {api_version}")
3175 except:
3176 # Show the ugly traceback if the matcher causes an
3177 # exception, but do NOT crash the kernel!
3178 sys.excepthook(*sys.exc_info())
3179 continue
3181 # set default value for matched fragment if suffix was not selected.
3182 result["matched_fragment"] = result.get("matched_fragment", context.token)
3184 if not suppressed_matchers:
3185 suppression_recommended: Union[bool, Set[str]] = result.get(
3186 "suppress", False
3187 )
3189 suppression_config = (
3190 self.suppress_competing_matchers.get(matcher_id, None)
3191 if isinstance(self.suppress_competing_matchers, dict)
3192 else self.suppress_competing_matchers
3193 )
3194 should_suppress = (
3195 (suppression_config is True)
3196 or (suppression_recommended and (suppression_config is not False))
3197 ) and has_any_completions(result)
3199 if should_suppress:
3200 suppression_exceptions: Set[str] = result.get(
3201 "do_not_suppress", set()
3202 )
3203 if isinstance(suppression_recommended, Iterable):
3204 to_suppress = set(suppression_recommended)
3205 else:
3206 to_suppress = set(matchers)
3207 suppressed_matchers = to_suppress - suppression_exceptions
3209 new_results = {}
3210 for previous_matcher_id, previous_result in results.items():
3211 if previous_matcher_id not in suppressed_matchers:
3212 new_results[previous_matcher_id] = previous_result
3213 results = new_results
3215 results[matcher_id] = result
3217 _, matches = self._arrange_and_extract(
3218 results,
3219 # TODO Jedi completions non included in legacy stateful API; was this deliberate or omission?
3220 # if it was omission, we can remove the filtering step, otherwise remove this comment.
3221 skip_matchers={jedi_matcher_id},
3222 abort_if_offset_changes=False,
3223 )
3225 # populate legacy stateful API
3226 self.matches = matches
3228 return results
3230 @staticmethod
3231 def _deduplicate(
3232 matches: Sequence[AnyCompletion],
3233 ) -> Iterable[AnyCompletion]:
3234 filtered_matches: Dict[str, AnyCompletion] = {}
3235 for match in matches:
3236 text = match.text
3237 if (
3238 text not in filtered_matches
3239 or filtered_matches[text].type == _UNKNOWN_TYPE
3240 ):
3241 filtered_matches[text] = match
3243 return filtered_matches.values()
3245 @staticmethod
3246 def _sort(matches: Sequence[AnyCompletion]):
3247 return sorted(matches, key=lambda x: completions_sorting_key(x.text))
3249 @context_matcher()
3250 def fwd_unicode_matcher(self, context: CompletionContext):
3251 """Same as :any:`fwd_unicode_match`, but adopted to new Matcher API."""
3252 # TODO: use `context.limit` to terminate early once we matched the maximum
3253 # number that will be used downstream; can be added as an optional to
3254 # `fwd_unicode_match(text: str, limit: int = None)` or we could re-implement here.
3255 fragment, matches = self.fwd_unicode_match(context.text_until_cursor)
3256 return _convert_matcher_v1_result_to_v2(
3257 matches, type="unicode", fragment=fragment, suppress_if_matches=True
3258 )
3260 def fwd_unicode_match(self, text: str) -> Tuple[str, Sequence[str]]:
3261 """
3262 Forward match a string starting with a backslash with a list of
3263 potential Unicode completions.
3265 Will compute list of Unicode character names on first call and cache it.
3267 .. deprecated:: 8.6
3268 You can use :meth:`fwd_unicode_matcher` instead.
3270 Returns
3271 -------
3272 At tuple with:
3273 - matched text (empty if no matches)
3274 - list of potential completions, empty tuple otherwise)
3275 """
3276 # TODO: self.unicode_names is here a list we traverse each time with ~100k elements.
3277 # We could do a faster match using a Trie.
3279 # Using pygtrie the following seem to work:
3281 # s = PrefixSet()
3283 # for c in range(0,0x10FFFF + 1):
3284 # try:
3285 # s.add(unicodedata.name(chr(c)))
3286 # except ValueError:
3287 # pass
3288 # [''.join(k) for k in s.iter(prefix)]
3290 # But need to be timed and adds an extra dependency.
3292 slashpos = text.rfind('\\')
3293 # if text starts with slash
3294 if slashpos > -1:
3295 # PERF: It's important that we don't access self._unicode_names
3296 # until we're inside this if-block. _unicode_names is lazily
3297 # initialized, and it takes a user-noticeable amount of time to
3298 # initialize it, so we don't want to initialize it unless we're
3299 # actually going to use it.
3300 s = text[slashpos + 1 :]
3301 sup = s.upper()
3302 candidates = [x for x in self.unicode_names if x.startswith(sup)]
3303 if candidates:
3304 return s, candidates
3305 candidates = [x for x in self.unicode_names if sup in x]
3306 if candidates:
3307 return s, candidates
3308 splitsup = sup.split(" ")
3309 candidates = [
3310 x for x in self.unicode_names if all(u in x for u in splitsup)
3311 ]
3312 if candidates:
3313 return s, candidates
3315 return "", ()
3317 # if text does not start with slash
3318 else:
3319 return '', ()
3321 @property
3322 def unicode_names(self) -> List[str]:
3323 """List of names of unicode code points that can be completed.
3325 The list is lazily initialized on first access.
3326 """
3327 if self._unicode_names is None:
3328 names = []
3329 for c in range(0,0x10FFFF + 1):
3330 try:
3331 names.append(unicodedata.name(chr(c)))
3332 except ValueError:
3333 pass
3334 self._unicode_names = _unicode_name_compute(_UNICODE_RANGES)
3336 return self._unicode_names
3338def _unicode_name_compute(ranges:List[Tuple[int,int]]) -> List[str]:
3339 names = []
3340 for start,stop in ranges:
3341 for c in range(start, stop) :
3342 try:
3343 names.append(unicodedata.name(chr(c)))
3344 except ValueError:
3345 pass
3346 return names