Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.11/site-packages/regex/_main.py: 43%
Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1#
2# Secret Labs' Regular Expression Engine
3#
4# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
5#
6# This version of the SRE library can be redistributed under CNRI's
7# Python 1.6 license. For any other use, please contact Secret Labs
8# AB (info@pythonware.com).
9#
10# Portions of this engine have been developed in cooperation with
11# CNRI. Hewlett-Packard provided funding for 1.6 integration and
12# other compatibility work.
13#
14# 2010-01-16 mrab Python front-end re-written and extended
16r"""Support for regular expressions (RE).
18This module provides regular expression matching operations similar to those
19found in Perl. It supports both 8-bit and Unicode strings; both the pattern and
20the strings being processed can contain null bytes and characters outside the
21US ASCII range.
23Regular expressions can contain both special and ordinary characters. Most
24ordinary characters, like "A", "a", or "0", are the simplest regular
25expressions; they simply match themselves. You can concatenate ordinary
26characters, so last matches the string 'last'.
28There are a few differences between the old (legacy) behaviour and the new
29(enhanced) behaviour, which are indicated by VERSION0 or VERSION1.
31The special characters are:
32 "." Matches any character except a newline.
33 "^" Matches the start of the string.
34 "$" Matches the end of the string or just before the
35 newline at the end of the string.
36 "*" Matches 0 or more (greedy) repetitions of the preceding
37 RE. Greedy means that it will match as many repetitions
38 as possible.
39 "+" Matches 1 or more (greedy) repetitions of the preceding
40 RE.
41 "?" Matches 0 or 1 (greedy) of the preceding RE.
42 *?,+?,?? Non-greedy versions of the previous three special
43 characters.
44 *+,++,?+ Possessive versions of the previous three special
45 characters.
46 {m,n} Matches from m to n repetitions of the preceding RE.
47 {m,n}? Non-greedy version of the above.
48 {m,n}+ Possessive version of the above.
49 {...} Fuzzy matching constraints.
50 "\\" Either escapes special characters or signals a special
51 sequence.
52 [...] Indicates a set of characters. A "^" as the first
53 character indicates a complementing set.
54 "|" A|B, creates an RE that will match either A or B.
55 (...) Matches the RE inside the parentheses. The contents are
56 captured and can be retrieved or matched later in the
57 string.
58 (?flags-flags) VERSION1: Sets/clears the flags for the remainder of
59 the group or pattern; VERSION0: Sets the flags for the
60 entire pattern.
61 (?:...) Non-capturing version of regular parentheses.
62 (?>...) Atomic non-capturing version of regular parentheses.
63 (?flags-flags:...) Non-capturing version of regular parentheses with local
64 flags.
65 (?P<name>...) The substring matched by the group is accessible by
66 name.
67 (?<name>...) The substring matched by the group is accessible by
68 name.
69 (?P=name) Matches the text matched earlier by the group named
70 name.
71 (?#...) A comment; ignored.
72 (?=...) Matches if ... matches next, but doesn't consume the
73 string.
74 (?!...) Matches if ... doesn't match next.
75 (?<=...) Matches if preceded by ....
76 (?<!...) Matches if not preceded by ....
77 (?(id)yes|no) Matches yes pattern if group id matched, the (optional)
78 no pattern otherwise.
79 (?(DEFINE)...) If there's no group called "DEFINE", then ... will be
80 ignored, but any group definitions will be available.
81 (?|...|...) (?|A|B), creates an RE that will match either A or B,
82 but reuses capture group numbers across the
83 alternatives.
84 (*FAIL) Forces matching to fail, which means immediate
85 backtracking.
86 (*F) Abbreviation for (*FAIL).
87 (*PRUNE) Discards the current backtracking information. Its
88 effect doesn't extend outside an atomic group or a
89 lookaround.
90 (*SKIP) Similar to (*PRUNE), except that it also sets where in
91 the text the next attempt at matching the entire
92 pattern will start. Its effect doesn't extend outside
93 an atomic group or a lookaround.
95The fuzzy matching constraints are: "i" to permit insertions, "d" to permit
96deletions, "s" to permit substitutions, "e" to permit any of these. Limits are
97optional with "<=" and "<". If any type of error is provided then any type not
98provided is not permitted.
100A cost equation may be provided.
102Examples:
103 (?:fuzzy){i<=2}
104 (?:fuzzy){i<=1,s<=2,d<=1,1i+1s+1d<3}
106VERSION1: Set operators are supported, and a set can include nested sets. The
107set operators, in order of increasing precedence, are:
108 || Set union ("x||y" means "x or y").
109 ~~ (double tilde) Symmetric set difference ("x~~y" means "x or y, but not
110 both").
111 && Set intersection ("x&&y" means "x and y").
112 -- (double dash) Set difference ("x--y" means "x but not y").
114Implicit union, ie, simple juxtaposition like in [ab], has the highest
115precedence.
117VERSION0 and VERSION1:
118The special sequences consist of "\\" and a character from the list below. If
119the ordinary character is not on the list, then the resulting RE will match the
120second character.
121 \number Matches the contents of the group of the same number if
122 number is no more than 2 digits, otherwise the character
123 with the 3-digit octal code.
124 \a Matches the bell character.
125 \A Matches only at the start of the string.
126 \b Matches the empty string, but only at the start or end of a
127 word.
128 \B Matches the empty string, but not at the start or end of a
129 word.
130 \d Matches any decimal digit; equivalent to the set [0-9] when
131 matching a bytestring or a Unicode string with the ASCII
132 flag, or the whole range of Unicode digits when matching a
133 Unicode string.
134 \D Matches any non-digit character; equivalent to [^\d].
135 \f Matches the formfeed character.
136 \g<name> Matches the text matched by the group named name.
137 \G Matches the empty string, but only at the position where
138 the search started.
139 \h Matches horizontal whitespace.
140 \K Keeps only what follows for the entire match.
141 \L<name> Named list. The list is provided as a keyword argument.
142 \m Matches the empty string, but only at the start of a word.
143 \M Matches the empty string, but only at the end of a word.
144 \n Matches the newline character.
145 \N{name} Matches the named character.
146 \p{name=value} Matches the character if its property has the specified
147 value.
148 \P{name=value} Matches the character if its property hasn't the specified
149 value.
150 \r Matches the carriage-return character.
151 \s Matches any whitespace character; equivalent to
152 [ \t\n\r\f\v].
153 \S Matches any non-whitespace character; equivalent to [^\s].
154 \t Matches the tab character.
155 \uXXXX Matches the Unicode codepoint with 4-digit hex code XXXX.
156 \UXXXXXXXX Matches the Unicode codepoint with 8-digit hex code
157 XXXXXXXX.
158 \v Matches the vertical tab character.
159 \w Matches any alphanumeric character; equivalent to
160 [a-zA-Z0-9_] when matching a bytestring or a Unicode string
161 with the ASCII flag, or the whole range of Unicode
162 alphanumeric characters (letters plus digits plus
163 underscore) when matching a Unicode string. With LOCALE, it
164 will match the set [0-9_] plus characters defined as
165 letters for the current locale.
166 \W Matches the complement of \w; equivalent to [^\w].
167 \xXX Matches the character with 2-digit hex code XX.
168 \X Matches a grapheme.
169 \Z Matches only at the end of the string.
170 \z Matches only at the end of the string. Alias of \Z.
171 \\ Matches a literal backslash.
173This module exports the following functions:
174 match Match a regular expression pattern at the beginning of a string.
175 prefixmatch Match a regular expression pattern at the beginning of a string.
176 Alias of match.
177 fullmatch Match a regular expression pattern against all of a string.
178 search Search a string for the presence of a pattern.
179 sub Substitute occurrences of a pattern found in a string using a
180 template string.
181 subf Substitute occurrences of a pattern found in a string using a
182 format string.
183 subn Same as sub, but also return the number of substitutions made.
184 subfn Same as subf, but also return the number of substitutions made.
185 split Split a string by the occurrences of a pattern. VERSION1: will
186 split at zero-width match; VERSION0: won't split at zero-width
187 match.
188 splititer Return an iterator yielding the parts of a split string.
189 findall Find all occurrences of a pattern in a string.
190 finditer Return an iterator yielding a match object for each match.
191 compile Compile a pattern into a Pattern object.
192 purge Clear the regular expression cache.
193 escape Backslash all non-alphanumerics or special characters in a
194 string.
196Most of the functions support a concurrent parameter: if True, the GIL will be
197released during matching, allowing other Python threads to run concurrently. If
198the string changes during matching, the behaviour is undefined. This parameter
199is not needed when working on the builtin (immutable) string classes.
201Some of the functions in this module take flags as optional parameters. Most of
202these flags can also be set within an RE:
203 A a ASCII Make \w, \W, \b, \B, \d, and \D match the
204 corresponding ASCII character categories. Default
205 when matching a bytestring.
206 B b BESTMATCH Find the best fuzzy match (default is first).
207 D DEBUG Print the parsed pattern.
208 E e ENHANCEMATCH Attempt to improve the fit after finding the first
209 fuzzy match.
210 F f FULLCASE Use full case-folding when performing
211 case-insensitive matching in Unicode.
212 I i IGNORECASE Perform case-insensitive matching.
213 L L LOCALE Make \w, \W, \b, \B, \d, and \D dependent on the
214 current locale. (One byte per character only.)
215 M m MULTILINE "^" matches the beginning of lines (after a newline)
216 as well as the string. "$" matches the end of lines
217 (before a newline) as well as the end of the string.
218 P p POSIX Perform POSIX-standard matching (leftmost longest).
219 R r REVERSE Searches backwards.
220 S s DOTALL "." matches any character at all, including the
221 newline.
222 U u UNICODE Make \w, \W, \b, \B, \d, and \D dependent on the
223 Unicode locale. Default when matching a Unicode
224 string.
225 V0 V0 VERSION0 Turn on the old legacy behaviour.
226 V1 V1 VERSION1 Turn on the new enhanced behaviour. This flag
227 includes the FULLCASE flag.
228 W w WORD Make \b and \B work with default Unicode word breaks
229 and make ".", "^" and "$" work with Unicode line
230 breaks.
231 X x VERBOSE Ignore whitespace and comments for nicer looking REs.
233This module also defines an exception 'error'.
235"""
237# Public symbols.
238__all__ = ["cache_all", "compile", "DEFAULT_VERSION", "escape", "findall",
239 "finditer", "fullmatch", "match", "prefixmatch", "purge", "search", "split", "splititer",
240 "sub", "subf", "subfn", "subn", "template", "Scanner", "A", "ASCII", "B",
241 "BESTMATCH", "D", "DEBUG", "E", "ENHANCEMATCH", "S", "DOTALL", "F",
242 "FULLCASE", "I", "IGNORECASE", "L", "LOCALE", "M", "MULTILINE", "P", "POSIX",
243 "R", "REVERSE", "T", "TEMPLATE", "U", "UNICODE", "V0", "VERSION0", "V1",
244 "VERSION1", "X", "VERBOSE", "W", "WORD", "error", "Regex", "__version__",
245 "__doc__", "RegexFlag"]
247__version__ = "2026.7.19"
249# --------------------------------------------------------------------
250# Public interface.
252def match(pattern, string, flags=0, pos=None, endpos=None, partial=False,
253 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
254 """Try to apply the pattern at the start of the string, returning a match
255 object, or None if no match was found."""
256 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
257 return pat.match(string, pos, endpos, concurrent, partial, timeout)
259def prefixmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
260 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
261 """Try to apply the pattern at the start of the string, returning a match
262 object, or None if no match was found. Alias of match()."""
263 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
264 return pat.match(string, pos, endpos, concurrent, partial, timeout)
266def fullmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
267 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
268 """Try to apply the pattern against all of the string, returning a match
269 object, or None if no match was found."""
270 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
271 return pat.fullmatch(string, pos, endpos, concurrent, partial, timeout)
273def search(pattern, string, flags=0, pos=None, endpos=None, partial=False,
274 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
275 """Search through string looking for a match to the pattern, returning a
276 match object, or None if no match was found."""
277 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
278 return pat.search(string, pos, endpos, concurrent, partial, timeout)
280def sub(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
281 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
282 """Return the string obtained by replacing the leftmost (or rightmost with a
283 reverse pattern) non-overlapping occurrences of the pattern in string by the
284 replacement repl. repl can be either a string or a callable; if a string,
285 backslash escapes in it are processed; if a callable, it's passed the match
286 object and must return a replacement string to be used."""
287 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
288 return pat.sub(repl, string, count, pos, endpos, concurrent, timeout)
290def subf(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
291 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
292 """Return the string obtained by replacing the leftmost (or rightmost with a
293 reverse pattern) non-overlapping occurrences of the pattern in string by the
294 replacement format. format can be either a string or a callable; if a string,
295 it's treated as a format string; if a callable, it's passed the match object
296 and must return a replacement string to be used."""
297 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
298 return pat.subf(format, string, count, pos, endpos, concurrent, timeout)
300def subn(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
301 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
302 """Return a 2-tuple containing (new_string, number). new_string is the string
303 obtained by replacing the leftmost (or rightmost with a reverse pattern)
304 non-overlapping occurrences of the pattern in the source string by the
305 replacement repl. number is the number of substitutions that were made. repl
306 can be either a string or a callable; if a string, backslash escapes in it
307 are processed; if a callable, it's passed the match object and must return a
308 replacement string to be used."""
309 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
310 return pat.subn(repl, string, count, pos, endpos, concurrent, timeout)
312def subfn(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
313 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
314 """Return a 2-tuple containing (new_string, number). new_string is the string
315 obtained by replacing the leftmost (or rightmost with a reverse pattern)
316 non-overlapping occurrences of the pattern in the source string by the
317 replacement format. number is the number of substitutions that were made. format
318 can be either a string or a callable; if a string, it's treated as a format
319 string; if a callable, it's passed the match object and must return a
320 replacement string to be used."""
321 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
322 return pat.subfn(format, string, count, pos, endpos, concurrent, timeout)
324def split(pattern, string, maxsplit=0, flags=0, concurrent=None, timeout=None,
325 ignore_unused=False, **kwargs):
326 """Split the source string by the occurrences of the pattern, returning a
327 list containing the resulting substrings. If capturing parentheses are used
328 in pattern, then the text of all groups in the pattern are also returned as
329 part of the resulting list. If maxsplit is nonzero, at most maxsplit splits
330 occur, and the remainder of the string is returned as the final element of
331 the list."""
332 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
333 return pat.split(string, maxsplit, concurrent, timeout)
335def splititer(pattern, string, maxsplit=0, flags=0, concurrent=None,
336 timeout=None, ignore_unused=False, **kwargs):
337 "Return an iterator yielding the parts of a split string."
338 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
339 return pat.splititer(string, maxsplit, concurrent, timeout)
341def findall(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
342 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
343 """Return a list of all matches in the string. The matches may be overlapped
344 if overlapped is True. If one or more groups are present in the pattern,
345 return a list of groups; this will be a list of tuples if the pattern has
346 more than one group. Empty matches are included in the result."""
347 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
348 return pat.findall(string, pos, endpos, overlapped, concurrent, timeout)
350def finditer(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
351 partial=False, concurrent=None, timeout=None, ignore_unused=False, **kwargs):
352 """Return an iterator over all matches in the string. The matches may be
353 overlapped if overlapped is True. For each match, the iterator returns a
354 match object. Empty matches are included in the result."""
355 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
356 return pat.finditer(string, pos, endpos, overlapped, concurrent, partial,
357 timeout)
359def compile(pattern, flags=0, ignore_unused=False, cache_pattern=None, **kwargs):
360 "Compile a regular expression pattern, returning a pattern object."
361 if cache_pattern is None:
362 cache_pattern = _cache_all
363 return _compile(pattern, flags, ignore_unused, kwargs, cache_pattern)
365def purge():
366 "Clear the regular expression cache"
367 with _cache_lock:
368 _cache.clear()
369 _locale_sensitive.clear()
371# Whether to cache all patterns.
372_cache_all = True
374def cache_all(value=True):
375 """Sets whether to cache all patterns, even those are compiled explicitly.
376 Passing None has no effect, but returns the current setting."""
377 global _cache_all
379 if value is None:
380 return _cache_all
382 _cache_all = value
384def template(pattern, flags=0):
385 "Compile a template pattern, returning a pattern object."
386 return _compile(pattern, flags | TEMPLATE, False, {}, False)
388def escape(pattern, special_only=True, literal_spaces=False):
389 """Escape a string for use as a literal in a pattern. If special_only is
390 True, escape only special characters, else escape all non-alphanumeric
391 characters. If literal_spaces is True, don't escape spaces."""
392 # Convert it to Unicode.
393 if isinstance(pattern, bytes):
394 p = pattern.decode("latin-1")
395 else:
396 p = pattern
398 s = []
399 if special_only:
400 for c in p:
401 if c == " " and literal_spaces:
402 s.append(c)
403 elif c in _METACHARS or c.isspace():
404 s.append("\\")
405 s.append(c)
406 else:
407 s.append(c)
408 else:
409 for c in p:
410 if c == " " and literal_spaces:
411 s.append(c)
412 elif c in _ALNUM:
413 s.append(c)
414 else:
415 s.append("\\")
416 s.append(c)
418 r = "".join(s)
419 # Convert it back to bytes if necessary.
420 if isinstance(pattern, bytes):
421 r = r.encode("latin-1")
423 return r
425# --------------------------------------------------------------------
426# Internals.
428from regex import _regex_core
429from regex import _regex
430from threading import RLock as _RLock
431from locale import getpreferredencoding as _getpreferredencoding
432from regex._regex_core import *
433from regex._regex_core import (_ALL_VERSIONS, _ALL_ENCODINGS, _FirstSetError,
434 _UnscopedFlagSet, _check_group_features, _compile_firstset,
435 _compile_replacement, _flatten_code, _fold_case, _get_required_string,
436 _parse_pattern, _shrink_cache)
437from regex._regex_core import (ALNUM as _ALNUM, Info as _Info, OP as _OP, Source
438 as _Source, Fuzzy as _Fuzzy)
440# Version 0 is the old behaviour, compatible with the original 're' module.
441# Version 1 is the new behaviour, which differs slightly.
443DEFAULT_VERSION = RegexFlag.VERSION0
445_METACHARS = frozenset("()[]{}?*+|^$\\.-#&~")
447_regex_core.DEFAULT_VERSION = DEFAULT_VERSION
449# Caches for the patterns and replacements.
450_cache = {}
451_cache_lock = _RLock()
452_named_args = {}
453_replacement_cache = {}
454_locale_sensitive = {}
456# Maximum size of the cache.
457_MAXCACHE = 500
458_MAXREPCACHE = 500
460def _compile(pattern, flags, ignore_unused, kwargs, cache_it):
461 "Compiles a regular expression to a PatternObject."
463 global DEFAULT_VERSION
464 try:
465 from regex import DEFAULT_VERSION
466 except ImportError:
467 pass
469 # We won't bother to cache the pattern if we're debugging.
470 if (flags & DEBUG) != 0:
471 cache_it = False
473 # What locale is this pattern using?
474 locale_key = (type(pattern), pattern)
475 if _locale_sensitive.get(locale_key, True) or (flags & LOCALE) != 0:
476 # This pattern is, or might be, locale-sensitive.
477 pattern_locale = _getpreferredencoding(False)
478 else:
479 # This pattern is definitely not locale-sensitive.
480 pattern_locale = None
482 def complain_unused_args():
483 if ignore_unused:
484 return
486 # Complain about any unused keyword arguments, possibly resulting from a typo.
487 unused_kwargs = set(kwargs) - {k for k, v in args_needed}
488 if unused_kwargs:
489 any_one = next(iter(unused_kwargs))
490 raise ValueError('unused keyword argument {!a}'.format(any_one))
492 if cache_it:
493 try:
494 # Do we know what keyword arguments are needed?
495 args_key = pattern, type(pattern), flags
496 args_needed = _named_args[args_key]
498 # Are we being provided with its required keyword arguments?
499 args_supplied = set()
500 if args_needed:
501 for k, v in args_needed:
502 try:
503 args_supplied.add((k, frozenset(kwargs[k])))
504 except KeyError:
505 raise error("missing named list: {!r}".format(k))
507 complain_unused_args()
509 args_supplied = frozenset(args_supplied)
511 # Have we already seen this regular expression and named list?
512 pattern_key = (pattern, type(pattern), flags, args_supplied,
513 DEFAULT_VERSION, pattern_locale)
514 return _cache[pattern_key]
515 except KeyError:
516 # It's a new pattern, or new named list for a known pattern.
517 pass
519 # Guess the encoding from the class of the pattern string.
520 if isinstance(pattern, str):
521 guess_encoding = UNICODE
522 elif isinstance(pattern, bytes):
523 guess_encoding = ASCII
524 elif isinstance(pattern, Pattern):
525 if flags:
526 raise ValueError("cannot process flags argument with a compiled pattern")
528 return pattern
529 else:
530 raise TypeError("first argument must be a string or compiled pattern")
532 # Set the default version in the core code in case it has been changed.
533 _regex_core.DEFAULT_VERSION = DEFAULT_VERSION
535 global_flags = flags
537 while True:
538 caught_exception = None
539 try:
540 source = _Source(pattern)
541 info = _Info(global_flags, source.char_type, kwargs)
542 info.guess_encoding = guess_encoding
543 source.ignore_space = bool(info.flags & VERBOSE)
544 parsed = _parse_pattern(source, info)
545 break
546 except _UnscopedFlagSet:
547 # Remember the global flags for the next attempt.
548 global_flags = info.global_flags
549 except error as e:
550 caught_exception = e
552 if caught_exception:
553 raise error(caught_exception.msg, caught_exception.pattern,
554 caught_exception.pos)
556 if not source.at_end():
557 raise error("unbalanced parenthesis", pattern, source.pos)
559 # Check the global flags for conflicts.
560 version = (info.flags & _ALL_VERSIONS) or DEFAULT_VERSION
561 if version not in (0, VERSION0, VERSION1):
562 raise ValueError("VERSION0 and VERSION1 flags are mutually incompatible")
564 if (info.flags & _ALL_ENCODINGS) not in (0, ASCII, LOCALE, UNICODE):
565 raise ValueError("ASCII, LOCALE and UNICODE flags are mutually incompatible")
567 if isinstance(pattern, bytes) and (info.flags & UNICODE):
568 raise ValueError("cannot use UNICODE flag with a bytes pattern")
570 if not (info.flags & _ALL_ENCODINGS):
571 if isinstance(pattern, str):
572 info.flags |= UNICODE
573 else:
574 info.flags |= ASCII
576 reverse = bool(info.flags & REVERSE)
577 fuzzy = isinstance(parsed, _Fuzzy)
579 # Remember whether this pattern as an inline locale flag.
580 _locale_sensitive[locale_key] = info.inline_locale
582 # Fix the group references.
583 caught_exception = None
584 try:
585 parsed.fix_groups(pattern, reverse, False)
586 except error as e:
587 caught_exception = e
589 if caught_exception:
590 raise error(caught_exception.msg, caught_exception.pattern,
591 caught_exception.pos)
593 # Should we print the parsed pattern?
594 if flags & DEBUG:
595 parsed.dump(indent=0, reverse=reverse)
597 # Optimise the parsed pattern.
598 parsed = parsed.optimise(info, reverse)
599 parsed = parsed.pack_characters(info)
601 # Get the required string.
602 req_offset, req_chars, req_flags = _get_required_string(parsed, info.flags)
604 # Build the named lists.
605 named_lists = {}
606 named_list_indexes = [None] * len(info.named_lists_used)
607 args_needed = set()
608 for key, index in info.named_lists_used.items():
609 name, case_flags = key
610 values = frozenset(kwargs[name])
611 if case_flags:
612 items = frozenset(_fold_case(info, v) for v in values)
613 else:
614 items = values
615 named_lists[name] = values
616 named_list_indexes[index] = items
617 args_needed.add((name, values))
619 complain_unused_args()
621 # Check the features of the groups.
622 _check_group_features(info, parsed)
624 # Compile the parsed pattern. The result is a list of tuples.
625 code = parsed.compile(reverse)
627 # Is there a group call to the pattern as a whole?
628 key = (0, reverse, fuzzy)
629 ref = info.call_refs.get(key)
630 if ref is not None:
631 code = [(_OP.CALL_REF, ref)] + code + [(_OP.END, )]
633 # Add the final 'success' opcode.
634 code += [(_OP.SUCCESS, )]
636 # Compile the additional copies of the groups that we need.
637 for group, rev, fuz in info.additional_groups:
638 code += group.compile(rev, fuz)
640 # Flatten the code into a list of ints.
641 code = _flatten_code(code)
643 if not parsed.has_simple_start():
644 # Get the first set, if possible.
645 try:
646 fs_code = _compile_firstset(info, parsed.get_firstset(reverse))
647 fs_code = _flatten_code(fs_code)
648 code = fs_code + code
649 except _FirstSetError:
650 pass
652 # The named capture groups.
653 index_group = dict((v, n) for n, v in info.group_index.items())
655 # Create the PatternObject.
656 #
657 # Local flags like IGNORECASE affect the code generation, but aren't needed
658 # by the PatternObject itself. Conversely, global flags like LOCALE _don't_
659 # affect the code generation but _are_ needed by the PatternObject.
660 compiled_pattern = _regex.compile(pattern, info.flags | version, code,
661 info.group_index, index_group, named_lists, named_list_indexes,
662 req_offset, req_chars, req_flags, info.group_count)
664 # Do we need to reduce the size of the cache?
665 if len(_cache) >= _MAXCACHE:
666 with _cache_lock:
667 _shrink_cache(_cache, _named_args, _locale_sensitive, _MAXCACHE)
669 if cache_it:
670 if (info.flags & LOCALE) == 0:
671 pattern_locale = None
673 args_needed = frozenset(args_needed)
675 # Store this regular expression and named list.
676 pattern_key = (pattern, type(pattern), flags, args_needed,
677 DEFAULT_VERSION, pattern_locale)
679 with _cache_lock:
680 _cache[pattern_key] = compiled_pattern
682 # Store what keyword arguments are needed.
683 _named_args[args_key] = args_needed
685 return compiled_pattern
687def _compile_replacement_helper(pattern, template):
688 "Compiles a replacement template."
689 # This function is called by the _regex module.
691 # Have we seen this before?
692 key = pattern.pattern, pattern.flags, template
693 compiled = _replacement_cache.get(key)
694 if compiled is not None:
695 return compiled
697 if len(_replacement_cache) >= _MAXREPCACHE:
698 _replacement_cache.clear()
700 is_unicode = isinstance(template, str)
701 source = _Source(template)
702 if is_unicode:
703 def make_string(char_codes):
704 return "".join(chr(c) for c in char_codes)
705 else:
706 def make_string(char_codes):
707 return bytes(char_codes)
709 compiled = []
710 literal = []
711 while True:
712 ch = source.get()
713 if not ch:
714 break
715 if ch == "\\":
716 # '_compile_replacement' will return either an int group reference
717 # or a string literal. It returns items (plural) in order to handle
718 # a 2-character literal (an invalid escape sequence).
719 is_group, items = _compile_replacement(source, pattern, is_unicode)
720 if is_group:
721 # It's a group, so first flush the literal.
722 if literal:
723 compiled.append(make_string(literal))
724 literal = []
725 compiled.extend(items)
726 else:
727 literal.extend(items)
728 else:
729 literal.append(ord(ch))
731 # Flush the literal.
732 if literal:
733 compiled.append(make_string(literal))
735 _replacement_cache[key] = compiled
737 return compiled
739# We define Pattern here after all the support objects have been defined.
740_pat = _compile('', 0, False, {}, False)
741Pattern = type(_pat)
742Match = type(_pat.match(''))
743del _pat
745# Make Pattern public for typing annotations.
746__all__.append("Pattern")
747__all__.append("Match")
749# We'll define an alias for the 'compile' function so that the repr of a
750# pattern object is eval-able.
751Regex = compile
753# Register myself for pickling.
754import copyreg as _copy_reg
756def _pickle(pattern):
757 return _regex.compile, pattern._pickled_data
759_copy_reg.pickle(Pattern, _pickle)