Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.8/site-packages/regex/regex.py: 73%
Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1#
2# Secret Labs' Regular Expression Engine
3#
4# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
5#
6# This version of the SRE library can be redistributed under CNRI's
7# Python 1.6 license. For any other use, please contact Secret Labs
8# AB (info@pythonware.com).
9#
10# Portions of this engine have been developed in cooperation with
11# CNRI. Hewlett-Packard provided funding for 1.6 integration and
12# other compatibility work.
13#
14# 2010-01-16 mrab Python front-end re-written and extended
16r"""Support for regular expressions (RE).
18This module provides regular expression matching operations similar to those
19found in Perl. It supports both 8-bit and Unicode strings; both the pattern and
20the strings being processed can contain null bytes and characters outside the
21US ASCII range.
23Regular expressions can contain both special and ordinary characters. Most
24ordinary characters, like "A", "a", or "0", are the simplest regular
25expressions; they simply match themselves. You can concatenate ordinary
26characters, so last matches the string 'last'.
28There are a few differences between the old (legacy) behaviour and the new
29(enhanced) behaviour, which are indicated by VERSION0 or VERSION1.
31The special characters are:
32 "." Matches any character except a newline.
33 "^" Matches the start of the string.
34 "$" Matches the end of the string or just before the
35 newline at the end of the string.
36 "*" Matches 0 or more (greedy) repetitions of the preceding
37 RE. Greedy means that it will match as many repetitions
38 as possible.
39 "+" Matches 1 or more (greedy) repetitions of the preceding
40 RE.
41 "?" Matches 0 or 1 (greedy) of the preceding RE.
42 *?,+?,?? Non-greedy versions of the previous three special
43 characters.
44 *+,++,?+ Possessive versions of the previous three special
45 characters.
46 {m,n} Matches from m to n repetitions of the preceding RE.
47 {m,n}? Non-greedy version of the above.
48 {m,n}+ Possessive version of the above.
49 {...} Fuzzy matching constraints.
50 "\\" Either escapes special characters or signals a special
51 sequence.
52 [...] Indicates a set of characters. A "^" as the first
53 character indicates a complementing set.
54 "|" A|B, creates an RE that will match either A or B.
55 (...) Matches the RE inside the parentheses. The contents are
56 captured and can be retrieved or matched later in the
57 string.
58 (?flags-flags) VERSION1: Sets/clears the flags for the remainder of
59 the group or pattern; VERSION0: Sets the flags for the
60 entire pattern.
61 (?:...) Non-capturing version of regular parentheses.
62 (?>...) Atomic non-capturing version of regular parentheses.
63 (?flags-flags:...) Non-capturing version of regular parentheses with local
64 flags.
65 (?P<name>...) The substring matched by the group is accessible by
66 name.
67 (?<name>...) The substring matched by the group is accessible by
68 name.
69 (?P=name) Matches the text matched earlier by the group named
70 name.
71 (?#...) A comment; ignored.
72 (?=...) Matches if ... matches next, but doesn't consume the
73 string.
74 (?!...) Matches if ... doesn't match next.
75 (?<=...) Matches if preceded by ....
76 (?<!...) Matches if not preceded by ....
77 (?(id)yes|no) Matches yes pattern if group id matched, the (optional)
78 no pattern otherwise.
79 (?(DEFINE)...) If there's no group called "DEFINE", then ... will be
80 ignored, but any group definitions will be available.
81 (?|...|...) (?|A|B), creates an RE that will match either A or B,
82 but reuses capture group numbers across the
83 alternatives.
84 (*FAIL) Forces matching to fail, which means immediate
85 backtracking.
86 (*F) Abbreviation for (*FAIL).
87 (*PRUNE) Discards the current backtracking information. Its
88 effect doesn't extend outside an atomic group or a
89 lookaround.
90 (*SKIP) Similar to (*PRUNE), except that it also sets where in
91 the text the next attempt at matching the entire
92 pattern will start. Its effect doesn't extend outside
93 an atomic group or a lookaround.
95The fuzzy matching constraints are: "i" to permit insertions, "d" to permit
96deletions, "s" to permit substitutions, "e" to permit any of these. Limits are
97optional with "<=" and "<". If any type of error is provided then any type not
98provided is not permitted.
100A cost equation may be provided.
102Examples:
103 (?:fuzzy){i<=2}
104 (?:fuzzy){i<=1,s<=2,d<=1,1i+1s+1d<3}
106VERSION1: Set operators are supported, and a set can include nested sets. The
107set operators, in order of increasing precedence, are:
108 || Set union ("x||y" means "x or y").
109 ~~ (double tilde) Symmetric set difference ("x~~y" means "x or y, but not
110 both").
111 && Set intersection ("x&&y" means "x and y").
112 -- (double dash) Set difference ("x--y" means "x but not y").
114Implicit union, ie, simple juxtaposition like in [ab], has the highest
115precedence.
117VERSION0 and VERSION1:
118The special sequences consist of "\\" and a character from the list below. If
119the ordinary character is not on the list, then the resulting RE will match the
120second character.
121 \number Matches the contents of the group of the same number if
122 number is no more than 2 digits, otherwise the character
123 with the 3-digit octal code.
124 \a Matches the bell character.
125 \A Matches only at the start of the string.
126 \b Matches the empty string, but only at the start or end of a
127 word.
128 \B Matches the empty string, but not at the start or end of a
129 word.
130 \d Matches any decimal digit; equivalent to the set [0-9] when
131 matching a bytestring or a Unicode string with the ASCII
132 flag, or the whole range of Unicode digits when matching a
133 Unicode string.
134 \D Matches any non-digit character; equivalent to [^\d].
135 \f Matches the formfeed character.
136 \g<name> Matches the text matched by the group named name.
137 \G Matches the empty string, but only at the position where
138 the search started.
139 \h Matches horizontal whitespace.
140 \K Keeps only what follows for the entire match.
141 \L<name> Named list. The list is provided as a keyword argument.
142 \m Matches the empty string, but only at the start of a word.
143 \M Matches the empty string, but only at the end of a word.
144 \n Matches the newline character.
145 \N{name} Matches the named character.
146 \p{name=value} Matches the character if its property has the specified
147 value.
148 \P{name=value} Matches the character if its property hasn't the specified
149 value.
150 \r Matches the carriage-return character.
151 \s Matches any whitespace character; equivalent to
152 [ \t\n\r\f\v].
153 \S Matches any non-whitespace character; equivalent to [^\s].
154 \t Matches the tab character.
155 \uXXXX Matches the Unicode codepoint with 4-digit hex code XXXX.
156 \UXXXXXXXX Matches the Unicode codepoint with 8-digit hex code
157 XXXXXXXX.
158 \v Matches the vertical tab character.
159 \w Matches any alphanumeric character; equivalent to
160 [a-zA-Z0-9_] when matching a bytestring or a Unicode string
161 with the ASCII flag, or the whole range of Unicode
162 alphanumeric characters (letters plus digits plus
163 underscore) when matching a Unicode string. With LOCALE, it
164 will match the set [0-9_] plus characters defined as
165 letters for the current locale.
166 \W Matches the complement of \w; equivalent to [^\w].
167 \xXX Matches the character with 2-digit hex code XX.
168 \X Matches a grapheme.
169 \Z Matches only at the end of the string.
170 \\ Matches a literal backslash.
172This module exports the following functions:
173 match Match a regular expression pattern at the beginning of a string.
174 fullmatch Match a regular expression pattern against all of a string.
175 search Search a string for the presence of a pattern.
176 sub Substitute occurrences of a pattern found in a string using a
177 template string.
178 subf Substitute occurrences of a pattern found in a string using a
179 format string.
180 subn Same as sub, but also return the number of substitutions made.
181 subfn Same as subf, but also return the number of substitutions made.
182 split Split a string by the occurrences of a pattern. VERSION1: will
183 split at zero-width match; VERSION0: won't split at zero-width
184 match.
185 splititer Return an iterator yielding the parts of a split string.
186 findall Find all occurrences of a pattern in a string.
187 finditer Return an iterator yielding a match object for each match.
188 compile Compile a pattern into a Pattern object.
189 purge Clear the regular expression cache.
190 escape Backslash all non-alphanumerics or special characters in a
191 string.
193Most of the functions support a concurrent parameter: if True, the GIL will be
194released during matching, allowing other Python threads to run concurrently. If
195the string changes during matching, the behaviour is undefined. This parameter
196is not needed when working on the builtin (immutable) string classes.
198Some of the functions in this module take flags as optional parameters. Most of
199these flags can also be set within an RE:
200 A a ASCII Make \w, \W, \b, \B, \d, and \D match the
201 corresponding ASCII character categories. Default
202 when matching a bytestring.
203 B b BESTMATCH Find the best fuzzy match (default is first).
204 D DEBUG Print the parsed pattern.
205 E e ENHANCEMATCH Attempt to improve the fit after finding the first
206 fuzzy match.
207 F f FULLCASE Use full case-folding when performing
208 case-insensitive matching in Unicode.
209 I i IGNORECASE Perform case-insensitive matching.
210 L L LOCALE Make \w, \W, \b, \B, \d, and \D dependent on the
211 current locale. (One byte per character only.)
212 M m MULTILINE "^" matches the beginning of lines (after a newline)
213 as well as the string. "$" matches the end of lines
214 (before a newline) as well as the end of the string.
215 P p POSIX Perform POSIX-standard matching (leftmost longest).
216 R r REVERSE Searches backwards.
217 S s DOTALL "." matches any character at all, including the
218 newline.
219 U u UNICODE Make \w, \W, \b, \B, \d, and \D dependent on the
220 Unicode locale. Default when matching a Unicode
221 string.
222 V0 V0 VERSION0 Turn on the old legacy behaviour.
223 V1 V1 VERSION1 Turn on the new enhanced behaviour. This flag
224 includes the FULLCASE flag.
225 W w WORD Make \b and \B work with default Unicode word breaks
226 and make ".", "^" and "$" work with Unicode line
227 breaks.
228 X x VERBOSE Ignore whitespace and comments for nicer looking REs.
230This module also defines an exception 'error'.
232"""
234# Public symbols.
235__all__ = ["cache_all", "compile", "DEFAULT_VERSION", "escape", "findall",
236 "finditer", "fullmatch", "match", "purge", "search", "split", "splititer",
237 "sub", "subf", "subfn", "subn", "template", "Scanner", "A", "ASCII", "B",
238 "BESTMATCH", "D", "DEBUG", "E", "ENHANCEMATCH", "S", "DOTALL", "F",
239 "FULLCASE", "I", "IGNORECASE", "L", "LOCALE", "M", "MULTILINE", "P", "POSIX",
240 "R", "REVERSE", "T", "TEMPLATE", "U", "UNICODE", "V0", "VERSION0", "V1",
241 "VERSION1", "X", "VERBOSE", "W", "WORD", "error", "Regex", "__version__",
242 "__doc__", "RegexFlag"]
244__version__ = "2.5.146"
246# --------------------------------------------------------------------
247# Public interface.
249def match(pattern, string, flags=0, pos=None, endpos=None, partial=False,
250 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
251 """Try to apply the pattern at the start of the string, returning a match
252 object, or None if no match was found."""
253 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
254 return pat.match(string, pos, endpos, concurrent, partial, timeout)
256def fullmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
257 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
258 """Try to apply the pattern against all of the string, returning a match
259 object, or None if no match was found."""
260 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
261 return pat.fullmatch(string, pos, endpos, concurrent, partial, timeout)
263def search(pattern, string, flags=0, pos=None, endpos=None, partial=False,
264 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
265 """Search through string looking for a match to the pattern, returning a
266 match object, or None if no match was found."""
267 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
268 return pat.search(string, pos, endpos, concurrent, partial, timeout)
270def sub(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
271 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
272 """Return the string obtained by replacing the leftmost (or rightmost with a
273 reverse pattern) non-overlapping occurrences of the pattern in string by the
274 replacement repl. repl can be either a string or a callable; if a string,
275 backslash escapes in it are processed; if a callable, it's passed the match
276 object and must return a replacement string to be used."""
277 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
278 return pat.sub(repl, string, count, pos, endpos, concurrent, timeout)
280def subf(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
281 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
282 """Return the string obtained by replacing the leftmost (or rightmost with a
283 reverse pattern) non-overlapping occurrences of the pattern in string by the
284 replacement format. format can be either a string or a callable; if a string,
285 it's treated as a format string; if a callable, it's passed the match object
286 and must return a replacement string to be used."""
287 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
288 return pat.subf(format, string, count, pos, endpos, concurrent, timeout)
290def subn(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
291 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
292 """Return a 2-tuple containing (new_string, number). new_string is the string
293 obtained by replacing the leftmost (or rightmost with a reverse pattern)
294 non-overlapping occurrences of the pattern in the source string by the
295 replacement repl. number is the number of substitutions that were made. repl
296 can be either a string or a callable; if a string, backslash escapes in it
297 are processed; if a callable, it's passed the match object and must return a
298 replacement string to be used."""
299 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
300 return pat.subn(repl, string, count, pos, endpos, concurrent, timeout)
302def subfn(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
303 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
304 """Return a 2-tuple containing (new_string, number). new_string is the string
305 obtained by replacing the leftmost (or rightmost with a reverse pattern)
306 non-overlapping occurrences of the pattern in the source string by the
307 replacement format. number is the number of substitutions that were made. format
308 can be either a string or a callable; if a string, it's treated as a format
309 string; if a callable, it's passed the match object and must return a
310 replacement string to be used."""
311 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
312 return pat.subfn(format, string, count, pos, endpos, concurrent, timeout)
314def split(pattern, string, maxsplit=0, flags=0, concurrent=None, timeout=None,
315 ignore_unused=False, **kwargs):
316 """Split the source string by the occurrences of the pattern, returning a
317 list containing the resulting substrings. If capturing parentheses are used
318 in pattern, then the text of all groups in the pattern are also returned as
319 part of the resulting list. If maxsplit is nonzero, at most maxsplit splits
320 occur, and the remainder of the string is returned as the final element of
321 the list."""
322 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
323 return pat.split(string, maxsplit, concurrent, timeout)
325def splititer(pattern, string, maxsplit=0, flags=0, concurrent=None,
326 timeout=None, ignore_unused=False, **kwargs):
327 "Return an iterator yielding the parts of a split string."
328 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
329 return pat.splititer(string, maxsplit, concurrent, timeout)
331def findall(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
332 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
333 """Return a list of all matches in the string. The matches may be overlapped
334 if overlapped is True. If one or more groups are present in the pattern,
335 return a list of groups; this will be a list of tuples if the pattern has
336 more than one group. Empty matches are included in the result."""
337 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
338 return pat.findall(string, pos, endpos, overlapped, concurrent, timeout)
340def finditer(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
341 partial=False, concurrent=None, timeout=None, ignore_unused=False, **kwargs):
342 """Return an iterator over all matches in the string. The matches may be
343 overlapped if overlapped is True. For each match, the iterator returns a
344 match object. Empty matches are included in the result."""
345 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
346 return pat.finditer(string, pos, endpos, overlapped, concurrent, partial,
347 timeout)
349def compile(pattern, flags=0, ignore_unused=False, cache_pattern=None, **kwargs):
350 "Compile a regular expression pattern, returning a pattern object."
351 if cache_pattern is None:
352 cache_pattern = _cache_all
353 return _compile(pattern, flags, ignore_unused, kwargs, cache_pattern)
355def purge():
356 "Clear the regular expression cache"
357 _cache.clear()
358 _locale_sensitive.clear()
360# Whether to cache all patterns.
361_cache_all = True
363def cache_all(value=True):
364 """Sets whether to cache all patterns, even those are compiled explicitly.
365 Passing None has no effect, but returns the current setting."""
366 global _cache_all
368 if value is None:
369 return _cache_all
371 _cache_all = value
373def template(pattern, flags=0):
374 "Compile a template pattern, returning a pattern object."
375 return _compile(pattern, flags | TEMPLATE, False, {}, False)
377def escape(pattern, special_only=True, literal_spaces=False):
378 """Escape a string for use as a literal in a pattern. If special_only is
379 True, escape only special characters, else escape all non-alphanumeric
380 characters. If literal_spaces is True, don't escape spaces."""
381 # Convert it to Unicode.
382 if isinstance(pattern, bytes):
383 p = pattern.decode("latin-1")
384 else:
385 p = pattern
387 s = []
388 if special_only:
389 for c in p:
390 if c == " " and literal_spaces:
391 s.append(c)
392 elif c in _METACHARS or c.isspace():
393 s.append("\\")
394 s.append(c)
395 else:
396 s.append(c)
397 else:
398 for c in p:
399 if c == " " and literal_spaces:
400 s.append(c)
401 elif c in _ALNUM:
402 s.append(c)
403 else:
404 s.append("\\")
405 s.append(c)
407 r = "".join(s)
408 # Convert it back to bytes if necessary.
409 if isinstance(pattern, bytes):
410 r = r.encode("latin-1")
412 return r
414# --------------------------------------------------------------------
415# Internals.
417import regex._regex_core as _regex_core
418import regex._regex as _regex
419from threading import RLock as _RLock
420from locale import getpreferredencoding as _getpreferredencoding
421from regex._regex_core import *
422from regex._regex_core import (_ALL_VERSIONS, _ALL_ENCODINGS, _FirstSetError,
423 _UnscopedFlagSet, _check_group_features, _compile_firstset,
424 _compile_replacement, _flatten_code, _fold_case, _get_required_string,
425 _parse_pattern, _shrink_cache)
426from regex._regex_core import (ALNUM as _ALNUM, Info as _Info, OP as _OP, Source
427 as _Source, Fuzzy as _Fuzzy)
429# Version 0 is the old behaviour, compatible with the original 're' module.
430# Version 1 is the new behaviour, which differs slightly.
432DEFAULT_VERSION = VERSION0
434_METACHARS = frozenset("()[]{}?*+|^$\\.-#&~")
436_regex_core.DEFAULT_VERSION = DEFAULT_VERSION
438# Caches for the patterns and replacements.
439_cache = {}
440_cache_lock = _RLock()
441_named_args = {}
442_replacement_cache = {}
443_locale_sensitive = {}
445# Maximum size of the cache.
446_MAXCACHE = 500
447_MAXREPCACHE = 500
449def _compile(pattern, flags, ignore_unused, kwargs, cache_it):
450 "Compiles a regular expression to a PatternObject."
452 global DEFAULT_VERSION
453 try:
454 from regex import DEFAULT_VERSION
455 except ImportError:
456 pass
458 # We won't bother to cache the pattern if we're debugging.
459 if (flags & DEBUG) != 0:
460 cache_it = False
462 # What locale is this pattern using?
463 locale_key = (type(pattern), pattern)
464 if _locale_sensitive.get(locale_key, True) or (flags & LOCALE) != 0:
465 # This pattern is, or might be, locale-sensitive.
466 pattern_locale = _getpreferredencoding()
467 else:
468 # This pattern is definitely not locale-sensitive.
469 pattern_locale = None
471 def complain_unused_args():
472 if ignore_unused:
473 return
475 # Complain about any unused keyword arguments, possibly resulting from a typo.
476 unused_kwargs = set(kwargs) - {k for k, v in args_needed}
477 if unused_kwargs:
478 any_one = next(iter(unused_kwargs))
479 raise ValueError('unused keyword argument {!a}'.format(any_one))
481 if cache_it:
482 try:
483 # Do we know what keyword arguments are needed?
484 args_key = pattern, type(pattern), flags
485 args_needed = _named_args[args_key]
487 # Are we being provided with its required keyword arguments?
488 args_supplied = set()
489 if args_needed:
490 for k, v in args_needed:
491 try:
492 args_supplied.add((k, frozenset(kwargs[k])))
493 except KeyError:
494 raise error("missing named list: {!r}".format(k))
496 complain_unused_args()
498 args_supplied = frozenset(args_supplied)
500 # Have we already seen this regular expression and named list?
501 pattern_key = (pattern, type(pattern), flags, args_supplied,
502 DEFAULT_VERSION, pattern_locale)
503 return _cache[pattern_key]
504 except KeyError:
505 # It's a new pattern, or new named list for a known pattern.
506 pass
508 # Guess the encoding from the class of the pattern string.
509 if isinstance(pattern, str):
510 guess_encoding = UNICODE
511 elif isinstance(pattern, bytes):
512 guess_encoding = ASCII
513 elif isinstance(pattern, Pattern):
514 if flags:
515 raise ValueError("cannot process flags argument with a compiled pattern")
517 return pattern
518 else:
519 raise TypeError("first argument must be a string or compiled pattern")
521 # Set the default version in the core code in case it has been changed.
522 _regex_core.DEFAULT_VERSION = DEFAULT_VERSION
524 global_flags = flags
526 while True:
527 caught_exception = None
528 try:
529 source = _Source(pattern)
530 info = _Info(global_flags, source.char_type, kwargs)
531 info.guess_encoding = guess_encoding
532 source.ignore_space = bool(info.flags & VERBOSE)
533 parsed = _parse_pattern(source, info)
534 break
535 except _UnscopedFlagSet:
536 # Remember the global flags for the next attempt.
537 global_flags = info.global_flags
538 except error as e:
539 caught_exception = e
541 if caught_exception:
542 raise error(caught_exception.msg, caught_exception.pattern,
543 caught_exception.pos)
545 if not source.at_end():
546 raise error("unbalanced parenthesis", pattern, source.pos)
548 # Check the global flags for conflicts.
549 version = (info.flags & _ALL_VERSIONS) or DEFAULT_VERSION
550 if version not in (0, VERSION0, VERSION1):
551 raise ValueError("VERSION0 and VERSION1 flags are mutually incompatible")
553 if (info.flags & _ALL_ENCODINGS) not in (0, ASCII, LOCALE, UNICODE):
554 raise ValueError("ASCII, LOCALE and UNICODE flags are mutually incompatible")
556 if isinstance(pattern, bytes) and (info.flags & UNICODE):
557 raise ValueError("cannot use UNICODE flag with a bytes pattern")
559 if not (info.flags & _ALL_ENCODINGS):
560 if isinstance(pattern, str):
561 info.flags |= UNICODE
562 else:
563 info.flags |= ASCII
565 reverse = bool(info.flags & REVERSE)
566 fuzzy = isinstance(parsed, _Fuzzy)
568 # Remember whether this pattern as an inline locale flag.
569 _locale_sensitive[locale_key] = info.inline_locale
571 # Fix the group references.
572 caught_exception = None
573 try:
574 parsed.fix_groups(pattern, reverse, False)
575 except error as e:
576 caught_exception = e
578 if caught_exception:
579 raise error(caught_exception.msg, caught_exception.pattern,
580 caught_exception.pos)
582 # Should we print the parsed pattern?
583 if flags & DEBUG:
584 parsed.dump(indent=0, reverse=reverse)
586 # Optimise the parsed pattern.
587 parsed = parsed.optimise(info, reverse)
588 parsed = parsed.pack_characters(info)
590 # Get the required string.
591 req_offset, req_chars, req_flags = _get_required_string(parsed, info.flags)
593 # Build the named lists.
594 named_lists = {}
595 named_list_indexes = [None] * len(info.named_lists_used)
596 args_needed = set()
597 for key, index in info.named_lists_used.items():
598 name, case_flags = key
599 values = frozenset(kwargs[name])
600 if case_flags:
601 items = frozenset(_fold_case(info, v) for v in values)
602 else:
603 items = values
604 named_lists[name] = values
605 named_list_indexes[index] = items
606 args_needed.add((name, values))
608 complain_unused_args()
610 # Check the features of the groups.
611 _check_group_features(info, parsed)
613 # Compile the parsed pattern. The result is a list of tuples.
614 code = parsed.compile(reverse)
616 # Is there a group call to the pattern as a whole?
617 key = (0, reverse, fuzzy)
618 ref = info.call_refs.get(key)
619 if ref is not None:
620 code = [(_OP.CALL_REF, ref)] + code + [(_OP.END, )]
622 # Add the final 'success' opcode.
623 code += [(_OP.SUCCESS, )]
625 # Compile the additional copies of the groups that we need.
626 for group, rev, fuz in info.additional_groups:
627 code += group.compile(rev, fuz)
629 # Flatten the code into a list of ints.
630 code = _flatten_code(code)
632 if not parsed.has_simple_start():
633 # Get the first set, if possible.
634 try:
635 fs_code = _compile_firstset(info, parsed.get_firstset(reverse))
636 fs_code = _flatten_code(fs_code)
637 code = fs_code + code
638 except _FirstSetError:
639 pass
641 # The named capture groups.
642 index_group = dict((v, n) for n, v in info.group_index.items())
644 # Create the PatternObject.
645 #
646 # Local flags like IGNORECASE affect the code generation, but aren't needed
647 # by the PatternObject itself. Conversely, global flags like LOCALE _don't_
648 # affect the code generation but _are_ needed by the PatternObject.
649 compiled_pattern = _regex.compile(pattern, info.flags | version, code,
650 info.group_index, index_group, named_lists, named_list_indexes,
651 req_offset, req_chars, req_flags, info.group_count)
653 # Do we need to reduce the size of the cache?
654 if len(_cache) >= _MAXCACHE:
655 with _cache_lock:
656 _shrink_cache(_cache, _named_args, _locale_sensitive, _MAXCACHE)
658 if cache_it:
659 if (info.flags & LOCALE) == 0:
660 pattern_locale = None
662 args_needed = frozenset(args_needed)
664 # Store this regular expression and named list.
665 pattern_key = (pattern, type(pattern), flags, args_needed,
666 DEFAULT_VERSION, pattern_locale)
667 _cache[pattern_key] = compiled_pattern
669 # Store what keyword arguments are needed.
670 _named_args[args_key] = args_needed
672 return compiled_pattern
674def _compile_replacement_helper(pattern, template):
675 "Compiles a replacement template."
676 # This function is called by the _regex module.
678 # Have we seen this before?
679 key = pattern.pattern, pattern.flags, template
680 compiled = _replacement_cache.get(key)
681 if compiled is not None:
682 return compiled
684 if len(_replacement_cache) >= _MAXREPCACHE:
685 _replacement_cache.clear()
687 is_unicode = isinstance(template, str)
688 source = _Source(template)
689 if is_unicode:
690 def make_string(char_codes):
691 return "".join(chr(c) for c in char_codes)
692 else:
693 def make_string(char_codes):
694 return bytes(char_codes)
696 compiled = []
697 literal = []
698 while True:
699 ch = source.get()
700 if not ch:
701 break
702 if ch == "\\":
703 # '_compile_replacement' will return either an int group reference
704 # or a string literal. It returns items (plural) in order to handle
705 # a 2-character literal (an invalid escape sequence).
706 is_group, items = _compile_replacement(source, pattern, is_unicode)
707 if is_group:
708 # It's a group, so first flush the literal.
709 if literal:
710 compiled.append(make_string(literal))
711 literal = []
712 compiled.extend(items)
713 else:
714 literal.extend(items)
715 else:
716 literal.append(ord(ch))
718 # Flush the literal.
719 if literal:
720 compiled.append(make_string(literal))
722 _replacement_cache[key] = compiled
724 return compiled
726# We define Pattern here after all the support objects have been defined.
727_pat = _compile('', 0, False, {}, False)
728Pattern = type(_pat)
729Match = type(_pat.match(''))
730del _pat
732# Make Pattern public for typing annotations.
733__all__.append("Pattern")
734__all__.append("Match")
736# We'll define an alias for the 'compile' function so that the repr of a
737# pattern object is eval-able.
738Regex = compile
740# Register myself for pickling.
741import copyreg as _copy_reg
743def _pickle(pattern):
744 return _regex.compile, pattern._pickled_data
746_copy_reg.pickle(Pattern, _pickle)