Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.11/site-packages/regex/_main.py: 51%
Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1#
2# Secret Labs' Regular Expression Engine
3#
4# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
5#
6# This version of the SRE library can be redistributed under CNRI's
7# Python 1.6 license. For any other use, please contact Secret Labs
8# AB (info@pythonware.com).
9#
10# Portions of this engine have been developed in cooperation with
11# CNRI. Hewlett-Packard provided funding for 1.6 integration and
12# other compatibility work.
13#
14# 2010-01-16 mrab Python front-end re-written and extended
16r"""Support for regular expressions (RE).
18This module provides regular expression matching operations similar to those
19found in Perl. It supports both 8-bit and Unicode strings; both the pattern and
20the strings being processed can contain null bytes and characters outside the
21US ASCII range.
23Regular expressions can contain both special and ordinary characters. Most
24ordinary characters, like "A", "a", or "0", are the simplest regular
25expressions; they simply match themselves. You can concatenate ordinary
26characters, so last matches the string 'last'.
28There are a few differences between the old (legacy) behaviour and the new
29(enhanced) behaviour, which are indicated by VERSION0 or VERSION1.
31The special characters are:
32 "." Matches any character except a newline.
33 "^" Matches the start of the string.
34 "$" Matches the end of the string or just before the
35 newline at the end of the string.
36 "*" Matches 0 or more (greedy) repetitions of the preceding
37 RE. Greedy means that it will match as many repetitions
38 as possible.
39 "+" Matches 1 or more (greedy) repetitions of the preceding
40 RE.
41 "?" Matches 0 or 1 (greedy) of the preceding RE.
42 *?,+?,?? Non-greedy versions of the previous three special
43 characters.
44 *+,++,?+ Possessive versions of the previous three special
45 characters.
46 {m,n} Matches from m to n repetitions of the preceding RE.
47 {m,n}? Non-greedy version of the above.
48 {m,n}+ Possessive version of the above.
49 {...} Fuzzy matching constraints.
50 "\\" Either escapes special characters or signals a special
51 sequence.
52 [...] Indicates a set of characters. A "^" as the first
53 character indicates a complementing set.
54 "|" A|B, creates an RE that will match either A or B.
55 (...) Matches the RE inside the parentheses. The contents are
56 captured and can be retrieved or matched later in the
57 string.
58 (?flags-flags) VERSION1: Sets/clears the flags for the remainder of
59 the group or pattern; VERSION0: Sets the flags for the
60 entire pattern.
61 (?:...) Non-capturing version of regular parentheses.
62 (?>...) Atomic non-capturing version of regular parentheses.
63 (?flags-flags:...) Non-capturing version of regular parentheses with local
64 flags.
65 (?P<name>...) The substring matched by the group is accessible by
66 name.
67 (?<name>...) The substring matched by the group is accessible by
68 name.
69 (?P=name) Matches the text matched earlier by the group named
70 name.
71 (?#...) A comment; ignored.
72 (?=...) Matches if ... matches next, but doesn't consume the
73 string.
74 (?!...) Matches if ... doesn't match next.
75 (?<=...) Matches if preceded by ....
76 (?<!...) Matches if not preceded by ....
77 (?(id)yes|no) Matches yes pattern if group id matched, the (optional)
78 no pattern otherwise.
79 (?(DEFINE)...) If there's no group called "DEFINE", then ... will be
80 ignored, but any group definitions will be available.
81 (?|...|...) (?|A|B), creates an RE that will match either A or B,
82 but reuses capture group numbers across the
83 alternatives.
84 (*FAIL) Forces matching to fail, which means immediate
85 backtracking.
86 (*F) Abbreviation for (*FAIL).
87 (*PRUNE) Discards the current backtracking information. Its
88 effect doesn't extend outside an atomic group or a
89 lookaround.
90 (*SKIP) Similar to (*PRUNE), except that it also sets where in
91 the text the next attempt at matching the entire
92 pattern will start. Its effect doesn't extend outside
93 an atomic group or a lookaround.
95The fuzzy matching constraints are: "i" to permit insertions, "d" to permit
96deletions, "s" to permit substitutions, "e" to permit any of these. Limits are
97optional with "<=" and "<". If any type of error is provided then any type not
98provided is not permitted.
100A cost equation may be provided.
102Examples:
103 (?:fuzzy){i<=2}
104 (?:fuzzy){i<=1,s<=2,d<=1,1i+1s+1d<3}
106VERSION1: Set operators are supported, and a set can include nested sets. The
107set operators, in order of increasing precedence, are:
108 || Set union ("x||y" means "x or y").
109 ~~ (double tilde) Symmetric set difference ("x~~y" means "x or y, but not
110 both").
111 && Set intersection ("x&&y" means "x and y").
112 -- (double dash) Set difference ("x--y" means "x but not y").
114Implicit union, ie, simple juxtaposition like in [ab], has the highest
115precedence.
117VERSION0 and VERSION1:
118The special sequences consist of "\\" and a character from the list below. If
119the ordinary character is not on the list, then the resulting RE will match the
120second character.
121 \number Matches the contents of the group of the same number if
122 number is no more than 2 digits, otherwise the character
123 with the 3-digit octal code.
124 \a Matches the bell character.
125 \A Matches only at the start of the string.
126 \b Matches the empty string, but only at the start or end of a
127 word.
128 \B Matches the empty string, but not at the start or end of a
129 word.
130 \d Matches any decimal digit; equivalent to the set [0-9] when
131 matching a bytestring or a Unicode string with the ASCII
132 flag, or the whole range of Unicode digits when matching a
133 Unicode string.
134 \D Matches any non-digit character; equivalent to [^\d].
135 \f Matches the formfeed character.
136 \g<name> Matches the text matched by the group named name.
137 \G Matches the empty string, but only at the position where
138 the search started.
139 \h Matches horizontal whitespace.
140 \K Keeps only what follows for the entire match.
141 \L<name> Named list. The list is provided as a keyword argument.
142 \m Matches the empty string, but only at the start of a word.
143 \M Matches the empty string, but only at the end of a word.
144 \n Matches the newline character.
145 \N{name} Matches the named character.
146 \p{name=value} Matches the character if its property has the specified
147 value.
148 \P{name=value} Matches the character if its property hasn't the specified
149 value.
150 \r Matches the carriage-return character.
151 \s Matches any whitespace character; equivalent to
152 [ \t\n\r\f\v].
153 \S Matches any non-whitespace character; equivalent to [^\s].
154 \t Matches the tab character.
155 \uXXXX Matches the Unicode codepoint with 4-digit hex code XXXX.
156 \UXXXXXXXX Matches the Unicode codepoint with 8-digit hex code
157 XXXXXXXX.
158 \v Matches the vertical tab character.
159 \w Matches any alphanumeric character; equivalent to
160 [a-zA-Z0-9_] when matching a bytestring or a Unicode string
161 with the ASCII flag, or the whole range of Unicode
162 alphanumeric characters (letters plus digits plus
163 underscore) when matching a Unicode string. With LOCALE, it
164 will match the set [0-9_] plus characters defined as
165 letters for the current locale.
166 \W Matches the complement of \w; equivalent to [^\w].
167 \xXX Matches the character with 2-digit hex code XX.
168 \X Matches a grapheme.
169 \Z Matches only at the end of the string.
170 \z Matches only at the end of the string. Alias of \Z.
171 \\ Matches a literal backslash.
173This module exports the following functions:
174 match Match a regular expression pattern at the beginning of a string.
175 prefixmatch Match a regular expression pattern at the beginning of a string.
176 Alias of match.
177 fullmatch Match a regular expression pattern against all of a string.
178 search Search a string for the presence of a pattern.
179 sub Substitute occurrences of a pattern found in a string using a
180 template string.
181 subf Substitute occurrences of a pattern found in a string using a
182 format string.
183 subn Same as sub, but also return the number of substitutions made.
184 subfn Same as subf, but also return the number of substitutions made.
185 split Split a string by the occurrences of a pattern. VERSION1: will
186 split at zero-width match; VERSION0: won't split at zero-width
187 match.
188 splititer Return an iterator yielding the parts of a split string.
189 findall Find all occurrences of a pattern in a string.
190 finditer Return an iterator yielding a match object for each match.
191 compile Compile a pattern into a Pattern object.
192 purge Clear the regular expression cache.
193 escape Backslash all non-alphanumerics or special characters in a
194 string.
196Most of the functions support a concurrent parameter: if True, the GIL will be
197released during matching, allowing other Python threads to run concurrently. If
198the string changes during matching, the behaviour is undefined. This parameter
199is not needed when working on the builtin (immutable) string classes.
201Some of the functions in this module take flags as optional parameters. Most of
202these flags can also be set within an RE:
203 A a ASCII Make \w, \W, \b, \B, \d, and \D match the
204 corresponding ASCII character categories. Default
205 when matching a bytestring.
206 B b BESTMATCH Find the best fuzzy match (default is first).
207 D DEBUG Print the parsed pattern.
208 E e ENHANCEMATCH Attempt to improve the fit after finding the first
209 fuzzy match.
210 F f FULLCASE Use full case-folding when performing
211 case-insensitive matching in Unicode.
212 I i IGNORECASE Perform case-insensitive matching.
213 L L LOCALE Make \w, \W, \b, \B, \d, and \D dependent on the
214 current locale. (One byte per character only.)
215 M m MULTILINE "^" matches the beginning of lines (after a newline)
216 as well as the string. "$" matches the end of lines
217 (before a newline) as well as the end of the string.
218 P p POSIX Perform POSIX-standard matching (leftmost longest).
219 R r REVERSE Searches backwards.
220 S s DOTALL "." matches any character at all, including the
221 newline.
222 U u UNICODE Make \w, \W, \b, \B, \d, and \D dependent on the
223 Unicode locale. Default when matching a Unicode
224 string.
225 V0 V0 VERSION0 Turn on the old legacy behaviour.
226 V1 V1 VERSION1 Turn on the new enhanced behaviour. This flag
227 includes the FULLCASE flag.
228 W w WORD Make \b and \B work with default Unicode word breaks
229 and make ".", "^" and "$" work with Unicode line
230 breaks.
231 X x VERBOSE Ignore whitespace and comments for nicer looking REs.
233This module also defines an exception 'error'.
235"""
237# Public symbols.
238__all__ = ["cache_all", "compile", "DEFAULT_VERSION", "escape", "findall",
239 "finditer", "fullmatch", "match", "prefixmatch", "purge", "search", "split", "splititer",
240 "sub", "subf", "subfn", "subn", "template", "Scanner", "A", "ASCII", "B",
241 "BESTMATCH", "D", "DEBUG", "E", "ENHANCEMATCH", "S", "DOTALL", "F",
242 "FULLCASE", "I", "IGNORECASE", "L", "LOCALE", "M", "MULTILINE", "P", "POSIX",
243 "R", "REVERSE", "T", "TEMPLATE", "U", "UNICODE", "V0", "VERSION0", "V1",
244 "VERSION1", "X", "VERBOSE", "W", "WORD", "error", "Regex", "__version__",
245 "__doc__", "RegexFlag"]
247__version__ = "2026.3.32"
249# --------------------------------------------------------------------
250# Public interface.
252def match(pattern, string, flags=0, pos=None, endpos=None, partial=False,
253 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
254 """Try to apply the pattern at the start of the string, returning a match
255 object, or None if no match was found."""
256 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
257 return pat.match(string, pos, endpos, concurrent, partial, timeout)
259def prefixmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
260 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
261 """Try to apply the pattern at the start of the string, returning a match
262 object, or None if no match was found. Alias of match()."""
263 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
264 return pat.match(string, pos, endpos, concurrent, partial, timeout)
266def fullmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
267 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
268 """Try to apply the pattern against all of the string, returning a match
269 object, or None if no match was found."""
270 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
271 return pat.fullmatch(string, pos, endpos, concurrent, partial, timeout)
273def search(pattern, string, flags=0, pos=None, endpos=None, partial=False,
274 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
275 """Search through string looking for a match to the pattern, returning a
276 match object, or None if no match was found."""
277 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
278 return pat.search(string, pos, endpos, concurrent, partial, timeout)
280def sub(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
281 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
282 """Return the string obtained by replacing the leftmost (or rightmost with a
283 reverse pattern) non-overlapping occurrences of the pattern in string by the
284 replacement repl. repl can be either a string or a callable; if a string,
285 backslash escapes in it are processed; if a callable, it's passed the match
286 object and must return a replacement string to be used."""
287 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
288 return pat.sub(repl, string, count, pos, endpos, concurrent, timeout)
290def subf(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
291 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
292 """Return the string obtained by replacing the leftmost (or rightmost with a
293 reverse pattern) non-overlapping occurrences of the pattern in string by the
294 replacement format. format can be either a string or a callable; if a string,
295 it's treated as a format string; if a callable, it's passed the match object
296 and must return a replacement string to be used."""
297 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
298 return pat.subf(format, string, count, pos, endpos, concurrent, timeout)
300def subn(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
301 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
302 """Return a 2-tuple containing (new_string, number). new_string is the string
303 obtained by replacing the leftmost (or rightmost with a reverse pattern)
304 non-overlapping occurrences of the pattern in the source string by the
305 replacement repl. number is the number of substitutions that were made. repl
306 can be either a string or a callable; if a string, backslash escapes in it
307 are processed; if a callable, it's passed the match object and must return a
308 replacement string to be used."""
309 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
310 return pat.subn(repl, string, count, pos, endpos, concurrent, timeout)
312def subfn(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
313 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
314 """Return a 2-tuple containing (new_string, number). new_string is the string
315 obtained by replacing the leftmost (or rightmost with a reverse pattern)
316 non-overlapping occurrences of the pattern in the source string by the
317 replacement format. number is the number of substitutions that were made. format
318 can be either a string or a callable; if a string, it's treated as a format
319 string; if a callable, it's passed the match object and must return a
320 replacement string to be used."""
321 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
322 return pat.subfn(format, string, count, pos, endpos, concurrent, timeout)
324def split(pattern, string, maxsplit=0, flags=0, concurrent=None, timeout=None,
325 ignore_unused=False, **kwargs):
326 """Split the source string by the occurrences of the pattern, returning a
327 list containing the resulting substrings. If capturing parentheses are used
328 in pattern, then the text of all groups in the pattern are also returned as
329 part of the resulting list. If maxsplit is nonzero, at most maxsplit splits
330 occur, and the remainder of the string is returned as the final element of
331 the list."""
332 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
333 return pat.split(string, maxsplit, concurrent, timeout)
335def splititer(pattern, string, maxsplit=0, flags=0, concurrent=None,
336 timeout=None, ignore_unused=False, **kwargs):
337 "Return an iterator yielding the parts of a split string."
338 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
339 return pat.splititer(string, maxsplit, concurrent, timeout)
341def findall(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
342 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
343 """Return a list of all matches in the string. The matches may be overlapped
344 if overlapped is True. If one or more groups are present in the pattern,
345 return a list of groups; this will be a list of tuples if the pattern has
346 more than one group. Empty matches are included in the result."""
347 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
348 return pat.findall(string, pos, endpos, overlapped, concurrent, timeout)
350def finditer(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
351 partial=False, concurrent=None, timeout=None, ignore_unused=False, **kwargs):
352 """Return an iterator over all matches in the string. The matches may be
353 overlapped if overlapped is True. For each match, the iterator returns a
354 match object. Empty matches are included in the result."""
355 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
356 return pat.finditer(string, pos, endpos, overlapped, concurrent, partial,
357 timeout)
359def compile(pattern, flags=0, ignore_unused=False, cache_pattern=None, **kwargs):
360 "Compile a regular expression pattern, returning a pattern object."
361 if cache_pattern is None:
362 cache_pattern = _cache_all
363 return _compile(pattern, flags, ignore_unused, kwargs, cache_pattern)
365def purge():
366 "Clear the regular expression cache"
367 _cache.clear()
368 _locale_sensitive.clear()
370# Whether to cache all patterns.
371_cache_all = True
373def cache_all(value=True):
374 """Sets whether to cache all patterns, even those are compiled explicitly.
375 Passing None has no effect, but returns the current setting."""
376 global _cache_all
378 if value is None:
379 return _cache_all
381 _cache_all = value
383def template(pattern, flags=0):
384 "Compile a template pattern, returning a pattern object."
385 return _compile(pattern, flags | TEMPLATE, False, {}, False)
387def escape(pattern, special_only=True, literal_spaces=False):
388 """Escape a string for use as a literal in a pattern. If special_only is
389 True, escape only special characters, else escape all non-alphanumeric
390 characters. If literal_spaces is True, don't escape spaces."""
391 # Convert it to Unicode.
392 if isinstance(pattern, bytes):
393 p = pattern.decode("latin-1")
394 else:
395 p = pattern
397 s = []
398 if special_only:
399 for c in p:
400 if c == " " and literal_spaces:
401 s.append(c)
402 elif c in _METACHARS or c.isspace():
403 s.append("\\")
404 s.append(c)
405 else:
406 s.append(c)
407 else:
408 for c in p:
409 if c == " " and literal_spaces:
410 s.append(c)
411 elif c in _ALNUM:
412 s.append(c)
413 else:
414 s.append("\\")
415 s.append(c)
417 r = "".join(s)
418 # Convert it back to bytes if necessary.
419 if isinstance(pattern, bytes):
420 r = r.encode("latin-1")
422 return r
424# --------------------------------------------------------------------
425# Internals.
427from regex import _regex_core
428from regex import _regex
429from threading import RLock as _RLock
430from locale import getpreferredencoding as _getpreferredencoding
431from regex._regex_core import *
432from regex._regex_core import (_ALL_VERSIONS, _ALL_ENCODINGS, _FirstSetError,
433 _UnscopedFlagSet, _check_group_features, _compile_firstset,
434 _compile_replacement, _flatten_code, _fold_case, _get_required_string,
435 _parse_pattern, _shrink_cache)
436from regex._regex_core import (ALNUM as _ALNUM, Info as _Info, OP as _OP, Source
437 as _Source, Fuzzy as _Fuzzy)
439# Version 0 is the old behaviour, compatible with the original 're' module.
440# Version 1 is the new behaviour, which differs slightly.
442DEFAULT_VERSION = RegexFlag.VERSION0
444_METACHARS = frozenset("()[]{}?*+|^$\\.-#&~")
446_regex_core.DEFAULT_VERSION = DEFAULT_VERSION
448# Caches for the patterns and replacements.
449_cache = {}
450_cache_lock = _RLock()
451_named_args = {}
452_replacement_cache = {}
453_locale_sensitive = {}
455# Maximum size of the cache.
456_MAXCACHE = 500
457_MAXREPCACHE = 500
459def _compile(pattern, flags, ignore_unused, kwargs, cache_it):
460 "Compiles a regular expression to a PatternObject."
462 global DEFAULT_VERSION
463 try:
464 from regex import DEFAULT_VERSION
465 except ImportError:
466 pass
468 # We won't bother to cache the pattern if we're debugging.
469 if (flags & DEBUG) != 0:
470 cache_it = False
472 # What locale is this pattern using?
473 locale_key = (type(pattern), pattern)
474 if _locale_sensitive.get(locale_key, True) or (flags & LOCALE) != 0:
475 # This pattern is, or might be, locale-sensitive.
476 pattern_locale = _getpreferredencoding()
477 else:
478 # This pattern is definitely not locale-sensitive.
479 pattern_locale = None
481 def complain_unused_args():
482 if ignore_unused:
483 return
485 # Complain about any unused keyword arguments, possibly resulting from a typo.
486 unused_kwargs = set(kwargs) - {k for k, v in args_needed}
487 if unused_kwargs:
488 any_one = next(iter(unused_kwargs))
489 raise ValueError('unused keyword argument {!a}'.format(any_one))
491 if cache_it:
492 try:
493 # Do we know what keyword arguments are needed?
494 args_key = pattern, type(pattern), flags
495 args_needed = _named_args[args_key]
497 # Are we being provided with its required keyword arguments?
498 args_supplied = set()
499 if args_needed:
500 for k, v in args_needed:
501 try:
502 args_supplied.add((k, frozenset(kwargs[k])))
503 except KeyError:
504 raise error("missing named list: {!r}".format(k))
506 complain_unused_args()
508 args_supplied = frozenset(args_supplied)
510 # Have we already seen this regular expression and named list?
511 pattern_key = (pattern, type(pattern), flags, args_supplied,
512 DEFAULT_VERSION, pattern_locale)
513 return _cache[pattern_key]
514 except KeyError:
515 # It's a new pattern, or new named list for a known pattern.
516 pass
518 # Guess the encoding from the class of the pattern string.
519 if isinstance(pattern, str):
520 guess_encoding = UNICODE
521 elif isinstance(pattern, bytes):
522 guess_encoding = ASCII
523 elif isinstance(pattern, Pattern):
524 if flags:
525 raise ValueError("cannot process flags argument with a compiled pattern")
527 return pattern
528 else:
529 raise TypeError("first argument must be a string or compiled pattern")
531 # Set the default version in the core code in case it has been changed.
532 _regex_core.DEFAULT_VERSION = DEFAULT_VERSION
534 global_flags = flags
536 while True:
537 caught_exception = None
538 try:
539 source = _Source(pattern)
540 info = _Info(global_flags, source.char_type, kwargs)
541 info.guess_encoding = guess_encoding
542 source.ignore_space = bool(info.flags & VERBOSE)
543 parsed = _parse_pattern(source, info)
544 break
545 except _UnscopedFlagSet:
546 # Remember the global flags for the next attempt.
547 global_flags = info.global_flags
548 except error as e:
549 caught_exception = e
551 if caught_exception:
552 raise error(caught_exception.msg, caught_exception.pattern,
553 caught_exception.pos)
555 if not source.at_end():
556 raise error("unbalanced parenthesis", pattern, source.pos)
558 # Check the global flags for conflicts.
559 version = (info.flags & _ALL_VERSIONS) or DEFAULT_VERSION
560 if version not in (0, VERSION0, VERSION1):
561 raise ValueError("VERSION0 and VERSION1 flags are mutually incompatible")
563 if (info.flags & _ALL_ENCODINGS) not in (0, ASCII, LOCALE, UNICODE):
564 raise ValueError("ASCII, LOCALE and UNICODE flags are mutually incompatible")
566 if isinstance(pattern, bytes) and (info.flags & UNICODE):
567 raise ValueError("cannot use UNICODE flag with a bytes pattern")
569 if not (info.flags & _ALL_ENCODINGS):
570 if isinstance(pattern, str):
571 info.flags |= UNICODE
572 else:
573 info.flags |= ASCII
575 reverse = bool(info.flags & REVERSE)
576 fuzzy = isinstance(parsed, _Fuzzy)
578 # Remember whether this pattern as an inline locale flag.
579 _locale_sensitive[locale_key] = info.inline_locale
581 # Fix the group references.
582 caught_exception = None
583 try:
584 parsed.fix_groups(pattern, reverse, False)
585 except error as e:
586 caught_exception = e
588 if caught_exception:
589 raise error(caught_exception.msg, caught_exception.pattern,
590 caught_exception.pos)
592 # Should we print the parsed pattern?
593 if flags & DEBUG:
594 parsed.dump(indent=0, reverse=reverse)
596 # Optimise the parsed pattern.
597 parsed = parsed.optimise(info, reverse)
598 parsed = parsed.pack_characters(info)
600 # Get the required string.
601 req_offset, req_chars, req_flags = _get_required_string(parsed, info.flags)
603 # Build the named lists.
604 named_lists = {}
605 named_list_indexes = [None] * len(info.named_lists_used)
606 args_needed = set()
607 for key, index in info.named_lists_used.items():
608 name, case_flags = key
609 values = frozenset(kwargs[name])
610 if case_flags:
611 items = frozenset(_fold_case(info, v) for v in values)
612 else:
613 items = values
614 named_lists[name] = values
615 named_list_indexes[index] = items
616 args_needed.add((name, values))
618 complain_unused_args()
620 # Check the features of the groups.
621 _check_group_features(info, parsed)
623 # Compile the parsed pattern. The result is a list of tuples.
624 code = parsed.compile(reverse)
626 # Is there a group call to the pattern as a whole?
627 key = (0, reverse, fuzzy)
628 ref = info.call_refs.get(key)
629 if ref is not None:
630 code = [(_OP.CALL_REF, ref)] + code + [(_OP.END, )]
632 # Add the final 'success' opcode.
633 code += [(_OP.SUCCESS, )]
635 # Compile the additional copies of the groups that we need.
636 for group, rev, fuz in info.additional_groups:
637 code += group.compile(rev, fuz)
639 # Flatten the code into a list of ints.
640 code = _flatten_code(code)
642 if not parsed.has_simple_start():
643 # Get the first set, if possible.
644 try:
645 fs_code = _compile_firstset(info, parsed.get_firstset(reverse))
646 fs_code = _flatten_code(fs_code)
647 code = fs_code + code
648 except _FirstSetError:
649 pass
651 # The named capture groups.
652 index_group = dict((v, n) for n, v in info.group_index.items())
654 # Create the PatternObject.
655 #
656 # Local flags like IGNORECASE affect the code generation, but aren't needed
657 # by the PatternObject itself. Conversely, global flags like LOCALE _don't_
658 # affect the code generation but _are_ needed by the PatternObject.
659 compiled_pattern = _regex.compile(pattern, info.flags | version, code,
660 info.group_index, index_group, named_lists, named_list_indexes,
661 req_offset, req_chars, req_flags, info.group_count)
663 # Do we need to reduce the size of the cache?
664 if len(_cache) >= _MAXCACHE:
665 with _cache_lock:
666 _shrink_cache(_cache, _named_args, _locale_sensitive, _MAXCACHE)
668 if cache_it:
669 if (info.flags & LOCALE) == 0:
670 pattern_locale = None
672 args_needed = frozenset(args_needed)
674 # Store this regular expression and named list.
675 pattern_key = (pattern, type(pattern), flags, args_needed,
676 DEFAULT_VERSION, pattern_locale)
677 _cache[pattern_key] = compiled_pattern
679 # Store what keyword arguments are needed.
680 _named_args[args_key] = args_needed
682 return compiled_pattern
684def _compile_replacement_helper(pattern, template):
685 "Compiles a replacement template."
686 # This function is called by the _regex module.
688 # Have we seen this before?
689 key = pattern.pattern, pattern.flags, template
690 compiled = _replacement_cache.get(key)
691 if compiled is not None:
692 return compiled
694 if len(_replacement_cache) >= _MAXREPCACHE:
695 _replacement_cache.clear()
697 is_unicode = isinstance(template, str)
698 source = _Source(template)
699 if is_unicode:
700 def make_string(char_codes):
701 return "".join(chr(c) for c in char_codes)
702 else:
703 def make_string(char_codes):
704 return bytes(char_codes)
706 compiled = []
707 literal = []
708 while True:
709 ch = source.get()
710 if not ch:
711 break
712 if ch == "\\":
713 # '_compile_replacement' will return either an int group reference
714 # or a string literal. It returns items (plural) in order to handle
715 # a 2-character literal (an invalid escape sequence).
716 is_group, items = _compile_replacement(source, pattern, is_unicode)
717 if is_group:
718 # It's a group, so first flush the literal.
719 if literal:
720 compiled.append(make_string(literal))
721 literal = []
722 compiled.extend(items)
723 else:
724 literal.extend(items)
725 else:
726 literal.append(ord(ch))
728 # Flush the literal.
729 if literal:
730 compiled.append(make_string(literal))
732 _replacement_cache[key] = compiled
734 return compiled
736# We define Pattern here after all the support objects have been defined.
737_pat = _compile('', 0, False, {}, False)
738Pattern = type(_pat)
739Match = type(_pat.match(''))
740del _pat
742# Make Pattern public for typing annotations.
743__all__.append("Pattern")
744__all__.append("Match")
746# We'll define an alias for the 'compile' function so that the repr of a
747# pattern object is eval-able.
748Regex = compile
750# Register myself for pickling.
751import copyreg as _copy_reg
753def _pickle(pattern):
754 return _regex.compile, pattern._pickled_data
756_copy_reg.pickle(Pattern, _pickle)