Coverage for /pythoncovmergedfiles/medio/medio/usr/local/lib/python3.8/site-packages/regex-2023.3.23-py3.8-linux-x86_64.egg/regex/regex.py: 68%
269 statements
« prev ^ index » next coverage.py v7.2.2, created at 2023-03-26 06:34 +0000
1#
2# Secret Labs' Regular Expression Engine
3#
4# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.
5#
6# This version of the SRE library can be redistributed under CNRI's
7# Python 1.6 license. For any other use, please contact Secret Labs
8# AB (info@pythonware.com).
9#
10# Portions of this engine have been developed in cooperation with
11# CNRI. Hewlett-Packard provided funding for 1.6 integration and
12# other compatibility work.
13#
14# 2010-01-16 mrab Python front-end re-written and extended
16r"""Support for regular expressions (RE).
18This module provides regular expression matching operations similar to those
19found in Perl. It supports both 8-bit and Unicode strings; both the pattern and
20the strings being processed can contain null bytes and characters outside the
21US ASCII range.
23Regular expressions can contain both special and ordinary characters. Most
24ordinary characters, like "A", "a", or "0", are the simplest regular
25expressions; they simply match themselves. You can concatenate ordinary
26characters, so last matches the string 'last'.
28There are a few differences between the old (legacy) behaviour and the new
29(enhanced) behaviour, which are indicated by VERSION0 or VERSION1.
31The special characters are:
32 "." Matches any character except a newline.
33 "^" Matches the start of the string.
34 "$" Matches the end of the string or just before the
35 newline at the end of the string.
36 "*" Matches 0 or more (greedy) repetitions of the preceding
37 RE. Greedy means that it will match as many repetitions
38 as possible.
39 "+" Matches 1 or more (greedy) repetitions of the preceding
40 RE.
41 "?" Matches 0 or 1 (greedy) of the preceding RE.
42 *?,+?,?? Non-greedy versions of the previous three special
43 characters.
44 *+,++,?+ Possessive versions of the previous three special
45 characters.
46 {m,n} Matches from m to n repetitions of the preceding RE.
47 {m,n}? Non-greedy version of the above.
48 {m,n}+ Possessive version of the above.
49 {...} Fuzzy matching constraints.
50 "\\" Either escapes special characters or signals a special
51 sequence.
52 [...] Indicates a set of characters. A "^" as the first
53 character indicates a complementing set.
54 "|" A|B, creates an RE that will match either A or B.
55 (...) Matches the RE inside the parentheses. The contents are
56 captured and can be retrieved or matched later in the
57 string.
58 (?flags-flags) VERSION1: Sets/clears the flags for the remainder of
59 the group or pattern; VERSION0: Sets the flags for the
60 entire pattern.
61 (?:...) Non-capturing version of regular parentheses.
62 (?>...) Atomic non-capturing version of regular parentheses.
63 (?flags-flags:...) Non-capturing version of regular parentheses with local
64 flags.
65 (?P<name>...) The substring matched by the group is accessible by
66 name.
67 (?<name>...) The substring matched by the group is accessible by
68 name.
69 (?P=name) Matches the text matched earlier by the group named
70 name.
71 (?#...) A comment; ignored.
72 (?=...) Matches if ... matches next, but doesn't consume the
73 string.
74 (?!...) Matches if ... doesn't match next.
75 (?<=...) Matches if preceded by ....
76 (?<!...) Matches if not preceded by ....
77 (?(id)yes|no) Matches yes pattern if group id matched, the (optional)
78 no pattern otherwise.
79 (?(DEFINE)...) If there's no group called "DEFINE", then ... will be
80 ignored, but any group definitions will be available.
81 (?|...|...) (?|A|B), creates an RE that will match either A or B,
82 but reuses capture group numbers across the
83 alternatives.
84 (*FAIL) Forces matching to fail, which means immediate
85 backtracking.
86 (*F) Abbreviation for (*FAIL).
87 (*PRUNE) Discards the current backtracking information. Its
88 effect doesn't extend outside an atomic group or a
89 lookaround.
90 (*SKIP) Similar to (*PRUNE), except that it also sets where in
91 the text the next attempt at matching the entire
92 pattern will start. Its effect doesn't extend outside
93 an atomic group or a lookaround.
95The fuzzy matching constraints are: "i" to permit insertions, "d" to permit
96deletions, "s" to permit substitutions, "e" to permit any of these. Limits are
97optional with "<=" and "<". If any type of error is provided then any type not
98provided is not permitted.
100A cost equation may be provided.
102Examples:
103 (?:fuzzy){i<=2}
104 (?:fuzzy){i<=1,s<=2,d<=1,1i+1s+1d<3}
106VERSION1: Set operators are supported, and a set can include nested sets. The
107set operators, in order of increasing precedence, are:
108 || Set union ("x||y" means "x or y").
109 ~~ (double tilde) Symmetric set difference ("x~~y" means "x or y, but not
110 both").
111 && Set intersection ("x&&y" means "x and y").
112 -- (double dash) Set difference ("x--y" means "x but not y").
114Implicit union, ie, simple juxtaposition like in [ab], has the highest
115precedence.
117VERSION0 and VERSION1:
118The special sequences consist of "\\" and a character from the list below. If
119the ordinary character is not on the list, then the resulting RE will match the
120second character.
121 \number Matches the contents of the group of the same number if
122 number is no more than 2 digits, otherwise the character
123 with the 3-digit octal code.
124 \a Matches the bell character.
125 \A Matches only at the start of the string.
126 \b Matches the empty string, but only at the start or end of a
127 word.
128 \B Matches the empty string, but not at the start or end of a
129 word.
130 \d Matches any decimal digit; equivalent to the set [0-9] when
131 matching a bytestring or a Unicode string with the ASCII
132 flag, or the whole range of Unicode digits when matching a
133 Unicode string.
134 \D Matches any non-digit character; equivalent to [^\d].
135 \f Matches the formfeed character.
136 \g<name> Matches the text matched by the group named name.
137 \G Matches the empty string, but only at the position where
138 the search started.
139 \h Matches horizontal whitespace.
140 \K Keeps only what follows for the entire match.
141 \L<name> Named list. The list is provided as a keyword argument.
142 \m Matches the empty string, but only at the start of a word.
143 \M Matches the empty string, but only at the end of a word.
144 \n Matches the newline character.
145 \N{name} Matches the named character.
146 \p{name=value} Matches the character if its property has the specified
147 value.
148 \P{name=value} Matches the character if its property hasn't the specified
149 value.
150 \r Matches the carriage-return character.
151 \s Matches any whitespace character; equivalent to
152 [ \t\n\r\f\v].
153 \S Matches any non-whitespace character; equivalent to [^\s].
154 \t Matches the tab character.
155 \uXXXX Matches the Unicode codepoint with 4-digit hex code XXXX.
156 \UXXXXXXXX Matches the Unicode codepoint with 8-digit hex code
157 XXXXXXXX.
158 \v Matches the vertical tab character.
159 \w Matches any alphanumeric character; equivalent to
160 [a-zA-Z0-9_] when matching a bytestring or a Unicode string
161 with the ASCII flag, or the whole range of Unicode
162 alphanumeric characters (letters plus digits plus
163 underscore) when matching a Unicode string. With LOCALE, it
164 will match the set [0-9_] plus characters defined as
165 letters for the current locale.
166 \W Matches the complement of \w; equivalent to [^\w].
167 \xXX Matches the character with 2-digit hex code XX.
168 \X Matches a grapheme.
169 \Z Matches only at the end of the string.
170 \\ Matches a literal backslash.
172This module exports the following functions:
173 match Match a regular expression pattern at the beginning of a string.
174 fullmatch Match a regular expression pattern against all of a string.
175 search Search a string for the presence of a pattern.
176 sub Substitute occurrences of a pattern found in a string using a
177 template string.
178 subf Substitute occurrences of a pattern found in a string using a
179 format string.
180 subn Same as sub, but also return the number of substitutions made.
181 subfn Same as subf, but also return the number of substitutions made.
182 split Split a string by the occurrences of a pattern. VERSION1: will
183 split at zero-width match; VERSION0: won't split at zero-width
184 match.
185 splititer Return an iterator yielding the parts of a split string.
186 findall Find all occurrences of a pattern in a string.
187 finditer Return an iterator yielding a match object for each match.
188 compile Compile a pattern into a Pattern object.
189 purge Clear the regular expression cache.
190 escape Backslash all non-alphanumerics or special characters in a
191 string.
193Most of the functions support a concurrent parameter: if True, the GIL will be
194released during matching, allowing other Python threads to run concurrently. If
195the string changes during matching, the behaviour is undefined. This parameter
196is not needed when working on the builtin (immutable) string classes.
198Some of the functions in this module take flags as optional parameters. Most of
199these flags can also be set within an RE:
200 A a ASCII Make \w, \W, \b, \B, \d, and \D match the
201 corresponding ASCII character categories. Default
202 when matching a bytestring.
203 B b BESTMATCH Find the best fuzzy match (default is first).
204 D DEBUG Print the parsed pattern.
205 E e ENHANCEMATCH Attempt to improve the fit after finding the first
206 fuzzy match.
207 F f FULLCASE Use full case-folding when performing
208 case-insensitive matching in Unicode.
209 I i IGNORECASE Perform case-insensitive matching.
210 L L LOCALE Make \w, \W, \b, \B, \d, and \D dependent on the
211 current locale. (One byte per character only.)
212 M m MULTILINE "^" matches the beginning of lines (after a newline)
213 as well as the string. "$" matches the end of lines
214 (before a newline) as well as the end of the string.
215 P p POSIX Perform POSIX-standard matching (leftmost longest).
216 R r REVERSE Searches backwards.
217 S s DOTALL "." matches any character at all, including the
218 newline.
219 U u UNICODE Make \w, \W, \b, \B, \d, and \D dependent on the
220 Unicode locale. Default when matching a Unicode
221 string.
222 V0 V0 VERSION0 Turn on the old legacy behaviour.
223 V1 V1 VERSION1 Turn on the new enhanced behaviour. This flag
224 includes the FULLCASE flag.
225 W w WORD Make \b and \B work with default Unicode word breaks
226 and make ".", "^" and "$" work with Unicode line
227 breaks.
228 X x VERBOSE Ignore whitespace and comments for nicer looking REs.
230This module also defines an exception 'error'.
232"""
234# Public symbols.
235__all__ = ["cache_all", "compile", "DEFAULT_VERSION", "escape", "findall",
236 "finditer", "fullmatch", "match", "purge", "search", "split", "splititer",
237 "sub", "subf", "subfn", "subn", "template", "Scanner", "A", "ASCII", "B",
238 "BESTMATCH", "D", "DEBUG", "E", "ENHANCEMATCH", "S", "DOTALL", "F",
239 "FULLCASE", "I", "IGNORECASE", "L", "LOCALE", "M", "MULTILINE", "P", "POSIX",
240 "R", "REVERSE", "T", "TEMPLATE", "U", "UNICODE", "V0", "VERSION0", "V1",
241 "VERSION1", "X", "VERBOSE", "W", "WORD", "error", "Regex", "__version__",
242 "__doc__", "RegexFlag"]
244__version__ = "2.5.125"
246# --------------------------------------------------------------------
247# Public interface.
249def match(pattern, string, flags=0, pos=None, endpos=None, partial=False,
250 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
251 """Try to apply the pattern at the start of the string, returning a match
252 object, or None if no match was found."""
253 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
254 return pat.match(string, pos, endpos, concurrent, partial, timeout)
256def fullmatch(pattern, string, flags=0, pos=None, endpos=None, partial=False,
257 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
258 """Try to apply the pattern against all of the string, returning a match
259 object, or None if no match was found."""
260 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
261 return pat.fullmatch(string, pos, endpos, concurrent, partial, timeout)
263def search(pattern, string, flags=0, pos=None, endpos=None, partial=False,
264 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
265 """Search through string looking for a match to the pattern, returning a
266 match object, or None if no match was found."""
267 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
268 return pat.search(string, pos, endpos, concurrent, partial, timeout)
270def sub(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
271 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
272 """Return the string obtained by replacing the leftmost (or rightmost with a
273 reverse pattern) non-overlapping occurrences of the pattern in string by the
274 replacement repl. repl can be either a string or a callable; if a string,
275 backslash escapes in it are processed; if a callable, it's passed the match
276 object and must return a replacement string to be used."""
277 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
278 return pat.sub(repl, string, count, pos, endpos, concurrent, timeout)
280def subf(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
281 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
282 """Return the string obtained by replacing the leftmost (or rightmost with a
283 reverse pattern) non-overlapping occurrences of the pattern in string by the
284 replacement format. format can be either a string or a callable; if a string,
285 it's treated as a format string; if a callable, it's passed the match object
286 and must return a replacement string to be used."""
287 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
288 return pat.subf(format, string, count, pos, endpos, concurrent, timeout)
290def subn(pattern, repl, string, count=0, flags=0, pos=None, endpos=None,
291 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
292 """Return a 2-tuple containing (new_string, number). new_string is the string
293 obtained by replacing the leftmost (or rightmost with a reverse pattern)
294 non-overlapping occurrences of the pattern in the source string by the
295 replacement repl. number is the number of substitutions that were made. repl
296 can be either a string or a callable; if a string, backslash escapes in it
297 are processed; if a callable, it's passed the match object and must return a
298 replacement string to be used."""
299 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
300 return pat.subn(repl, string, count, pos, endpos, concurrent, timeout)
302def subfn(pattern, format, string, count=0, flags=0, pos=None, endpos=None,
303 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
304 """Return a 2-tuple containing (new_string, number). new_string is the string
305 obtained by replacing the leftmost (or rightmost with a reverse pattern)
306 non-overlapping occurrences of the pattern in the source string by the
307 replacement format. number is the number of substitutions that were made. format
308 can be either a string or a callable; if a string, it's treated as a format
309 string; if a callable, it's passed the match object and must return a
310 replacement string to be used."""
311 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
312 return pat.subfn(format, string, count, pos, endpos, concurrent, timeout)
314def split(pattern, string, maxsplit=0, flags=0, concurrent=None, timeout=None,
315 ignore_unused=False, **kwargs):
316 """Split the source string by the occurrences of the pattern, returning a
317 list containing the resulting substrings. If capturing parentheses are used
318 in pattern, then the text of all groups in the pattern are also returned as
319 part of the resulting list. If maxsplit is nonzero, at most maxsplit splits
320 occur, and the remainder of the string is returned as the final element of
321 the list."""
322 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
323 return pat.split(string, maxsplit, concurrent, timeout)
325def splititer(pattern, string, maxsplit=0, flags=0, concurrent=None,
326 timeout=None, ignore_unused=False, **kwargs):
327 "Return an iterator yielding the parts of a split string."
328 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
329 return pat.splititer(string, maxsplit, concurrent, timeout)
331def findall(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
332 concurrent=None, timeout=None, ignore_unused=False, **kwargs):
333 """Return a list of all matches in the string. The matches may be overlapped
334 if overlapped is True. If one or more groups are present in the pattern,
335 return a list of groups; this will be a list of tuples if the pattern has
336 more than one group. Empty matches are included in the result."""
337 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
338 return pat.findall(string, pos, endpos, overlapped, concurrent, timeout)
340def finditer(pattern, string, flags=0, pos=None, endpos=None, overlapped=False,
341 partial=False, concurrent=None, timeout=None, ignore_unused=False, **kwargs):
342 """Return an iterator over all matches in the string. The matches may be
343 overlapped if overlapped is True. For each match, the iterator returns a
344 match object. Empty matches are included in the result."""
345 pat = _compile(pattern, flags, ignore_unused, kwargs, True)
346 return pat.finditer(string, pos, endpos, overlapped, concurrent, partial,
347 timeout)
349def compile(pattern, flags=0, ignore_unused=False, cache_pattern=None, **kwargs):
350 "Compile a regular expression pattern, returning a pattern object."
351 if cache_pattern is None:
352 cache_pattern = _cache_all
353 return _compile(pattern, flags, ignore_unused, kwargs, cache_pattern)
355def purge():
356 "Clear the regular expression cache"
357 _cache.clear()
358 _locale_sensitive.clear()
360# Whether to cache all patterns.
361_cache_all = True
363def cache_all(value=True):
364 """Sets whether to cache all patterns, even those are compiled explicitly.
365 Passing None has no effect, but returns the current setting."""
366 global _cache_all
368 if value is None:
369 return _cache_all
371 _cache_all = value
373def template(pattern, flags=0):
374 "Compile a template pattern, returning a pattern object."
375 return _compile(pattern, flags | TEMPLATE, False, {}, False)
377def escape(pattern, special_only=True, literal_spaces=False):
378 """Escape a string for use as a literal in a pattern. If special_only is
379 True, escape only special characters, else escape all non-alphanumeric
380 characters. If literal_spaces is True, don't escape spaces."""
381 # Convert it to Unicode.
382 if isinstance(pattern, bytes):
383 p = pattern.decode("latin-1")
384 else:
385 p = pattern
387 s = []
388 if special_only:
389 for c in p:
390 if c == " " and literal_spaces:
391 s.append(c)
392 elif c in _METACHARS or c.isspace():
393 s.append("\\")
394 s.append(c)
395 elif c == "\x00":
396 s.append("\\000")
397 else:
398 s.append(c)
399 else:
400 for c in p:
401 if c == " " and literal_spaces:
402 s.append(c)
403 elif c in _ALNUM:
404 s.append(c)
405 elif c == "\x00":
406 s.append("\\000")
407 else:
408 s.append("\\")
409 s.append(c)
411 r = "".join(s)
412 # Convert it back to bytes if necessary.
413 if isinstance(pattern, bytes):
414 r = r.encode("latin-1")
416 return r
418# --------------------------------------------------------------------
419# Internals.
421import regex._regex_core as _regex_core
422import regex._regex as _regex
423from threading import RLock as _RLock
424from locale import getpreferredencoding as _getpreferredencoding
425from regex._regex_core import *
426from regex._regex_core import (_ALL_VERSIONS, _ALL_ENCODINGS, _FirstSetError,
427 _UnscopedFlagSet, _check_group_features, _compile_firstset,
428 _compile_replacement, _flatten_code, _fold_case, _get_required_string,
429 _parse_pattern, _shrink_cache)
430from regex._regex_core import (ALNUM as _ALNUM, Info as _Info, OP as _OP, Source
431 as _Source, Fuzzy as _Fuzzy)
433# Version 0 is the old behaviour, compatible with the original 're' module.
434# Version 1 is the new behaviour, which differs slightly.
436DEFAULT_VERSION = VERSION0
438_METACHARS = frozenset("()[]{}?*+|^$\\.-#&~")
440_regex_core.DEFAULT_VERSION = DEFAULT_VERSION
442# Caches for the patterns and replacements.
443_cache = {}
444_cache_lock = _RLock()
445_named_args = {}
446_replacement_cache = {}
447_locale_sensitive = {}
449# Maximum size of the cache.
450_MAXCACHE = 500
451_MAXREPCACHE = 500
453def _compile(pattern, flags, ignore_unused, kwargs, cache_it):
454 "Compiles a regular expression to a PatternObject."
456 global DEFAULT_VERSION
457 try:
458 from regex import DEFAULT_VERSION
459 except ImportError:
460 pass
462 # We won't bother to cache the pattern if we're debugging.
463 if (flags & DEBUG) != 0:
464 cache_it = False
466 # What locale is this pattern using?
467 locale_key = (type(pattern), pattern)
468 if _locale_sensitive.get(locale_key, True) or (flags & LOCALE) != 0:
469 # This pattern is, or might be, locale-sensitive.
470 pattern_locale = _getpreferredencoding()
471 else:
472 # This pattern is definitely not locale-sensitive.
473 pattern_locale = None
475 def complain_unused_args():
476 if ignore_unused:
477 return
479 # Complain about any unused keyword arguments, possibly resulting from a typo.
480 unused_kwargs = set(kwargs) - {k for k, v in args_needed}
481 if unused_kwargs:
482 any_one = next(iter(unused_kwargs))
483 raise ValueError('unused keyword argument {!a}'.format(any_one))
485 if cache_it:
486 try:
487 # Do we know what keyword arguments are needed?
488 args_key = pattern, type(pattern), flags
489 args_needed = _named_args[args_key]
491 # Are we being provided with its required keyword arguments?
492 args_supplied = set()
493 if args_needed:
494 for k, v in args_needed:
495 try:
496 args_supplied.add((k, frozenset(kwargs[k])))
497 except KeyError:
498 raise error("missing named list: {!r}".format(k))
500 complain_unused_args()
502 args_supplied = frozenset(args_supplied)
504 # Have we already seen this regular expression and named list?
505 pattern_key = (pattern, type(pattern), flags, args_supplied,
506 DEFAULT_VERSION, pattern_locale)
507 return _cache[pattern_key]
508 except KeyError:
509 # It's a new pattern, or new named list for a known pattern.
510 pass
512 # Guess the encoding from the class of the pattern string.
513 if isinstance(pattern, str):
514 guess_encoding = UNICODE
515 elif isinstance(pattern, bytes):
516 guess_encoding = ASCII
517 elif isinstance(pattern, Pattern):
518 if flags:
519 raise ValueError("cannot process flags argument with a compiled pattern")
521 return pattern
522 else:
523 raise TypeError("first argument must be a string or compiled pattern")
525 # Set the default version in the core code in case it has been changed.
526 _regex_core.DEFAULT_VERSION = DEFAULT_VERSION
528 global_flags = flags
530 while True:
531 caught_exception = None
532 try:
533 source = _Source(pattern)
534 info = _Info(global_flags, source.char_type, kwargs)
535 info.guess_encoding = guess_encoding
536 source.ignore_space = bool(info.flags & VERBOSE)
537 parsed = _parse_pattern(source, info)
538 break
539 except _UnscopedFlagSet:
540 # Remember the global flags for the next attempt.
541 global_flags = info.global_flags
542 except error as e:
543 caught_exception = e
545 if caught_exception:
546 raise error(caught_exception.msg, caught_exception.pattern,
547 caught_exception.pos)
549 if not source.at_end():
550 raise error("unbalanced parenthesis", pattern, source.pos)
552 # Check the global flags for conflicts.
553 version = (info.flags & _ALL_VERSIONS) or DEFAULT_VERSION
554 if version not in (0, VERSION0, VERSION1):
555 raise ValueError("VERSION0 and VERSION1 flags are mutually incompatible")
557 if (info.flags & _ALL_ENCODINGS) not in (0, ASCII, LOCALE, UNICODE):
558 raise ValueError("ASCII, LOCALE and UNICODE flags are mutually incompatible")
560 if isinstance(pattern, bytes) and (info.flags & UNICODE):
561 raise ValueError("cannot use UNICODE flag with a bytes pattern")
563 if not (info.flags & _ALL_ENCODINGS):
564 if isinstance(pattern, str):
565 info.flags |= UNICODE
566 else:
567 info.flags |= ASCII
569 reverse = bool(info.flags & REVERSE)
570 fuzzy = isinstance(parsed, _Fuzzy)
572 # Remember whether this pattern as an inline locale flag.
573 _locale_sensitive[locale_key] = info.inline_locale
575 # Fix the group references.
576 caught_exception = None
577 try:
578 parsed.fix_groups(pattern, reverse, False)
579 except error as e:
580 caught_exception = e
582 if caught_exception:
583 raise error(caught_exception.msg, caught_exception.pattern,
584 caught_exception.pos)
586 # Should we print the parsed pattern?
587 if flags & DEBUG:
588 parsed.dump(indent=0, reverse=reverse)
590 # Optimise the parsed pattern.
591 parsed = parsed.optimise(info, reverse)
592 parsed = parsed.pack_characters(info)
594 # Get the required string.
595 req_offset, req_chars, req_flags = _get_required_string(parsed, info.flags)
597 # Build the named lists.
598 named_lists = {}
599 named_list_indexes = [None] * len(info.named_lists_used)
600 args_needed = set()
601 for key, index in info.named_lists_used.items():
602 name, case_flags = key
603 values = frozenset(kwargs[name])
604 if case_flags:
605 items = frozenset(_fold_case(info, v) for v in values)
606 else:
607 items = values
608 named_lists[name] = values
609 named_list_indexes[index] = items
610 args_needed.add((name, values))
612 complain_unused_args()
614 # Check the features of the groups.
615 _check_group_features(info, parsed)
617 # Compile the parsed pattern. The result is a list of tuples.
618 code = parsed.compile(reverse)
620 # Is there a group call to the pattern as a whole?
621 key = (0, reverse, fuzzy)
622 ref = info.call_refs.get(key)
623 if ref is not None:
624 code = [(_OP.CALL_REF, ref)] + code + [(_OP.END, )]
626 # Add the final 'success' opcode.
627 code += [(_OP.SUCCESS, )]
629 # Compile the additional copies of the groups that we need.
630 for group, rev, fuz in info.additional_groups:
631 code += group.compile(rev, fuz)
633 # Flatten the code into a list of ints.
634 code = _flatten_code(code)
636 if not parsed.has_simple_start():
637 # Get the first set, if possible.
638 try:
639 fs_code = _compile_firstset(info, parsed.get_firstset(reverse))
640 fs_code = _flatten_code(fs_code)
641 code = fs_code + code
642 except _FirstSetError:
643 pass
645 # The named capture groups.
646 index_group = dict((v, n) for n, v in info.group_index.items())
648 # Create the PatternObject.
649 #
650 # Local flags like IGNORECASE affect the code generation, but aren't needed
651 # by the PatternObject itself. Conversely, global flags like LOCALE _don't_
652 # affect the code generation but _are_ needed by the PatternObject.
653 compiled_pattern = _regex.compile(pattern, info.flags | version, code,
654 info.group_index, index_group, named_lists, named_list_indexes,
655 req_offset, req_chars, req_flags, info.group_count)
657 # Do we need to reduce the size of the cache?
658 if len(_cache) >= _MAXCACHE:
659 with _cache_lock:
660 _shrink_cache(_cache, _named_args, _locale_sensitive, _MAXCACHE)
662 if cache_it:
663 if (info.flags & LOCALE) == 0:
664 pattern_locale = None
666 args_needed = frozenset(args_needed)
668 # Store this regular expression and named list.
669 pattern_key = (pattern, type(pattern), flags, args_needed,
670 DEFAULT_VERSION, pattern_locale)
671 _cache[pattern_key] = compiled_pattern
673 # Store what keyword arguments are needed.
674 _named_args[args_key] = args_needed
676 return compiled_pattern
678def _compile_replacement_helper(pattern, template):
679 "Compiles a replacement template."
680 # This function is called by the _regex module.
682 # Have we seen this before?
683 key = pattern.pattern, pattern.flags, template
684 compiled = _replacement_cache.get(key)
685 if compiled is not None:
686 return compiled
688 if len(_replacement_cache) >= _MAXREPCACHE:
689 _replacement_cache.clear()
691 is_unicode = isinstance(template, str)
692 source = _Source(template)
693 if is_unicode:
694 def make_string(char_codes):
695 return "".join(chr(c) for c in char_codes)
696 else:
697 def make_string(char_codes):
698 return bytes(char_codes)
700 compiled = []
701 literal = []
702 while True:
703 ch = source.get()
704 if not ch:
705 break
706 if ch == "\\":
707 # '_compile_replacement' will return either an int group reference
708 # or a string literal. It returns items (plural) in order to handle
709 # a 2-character literal (an invalid escape sequence).
710 is_group, items = _compile_replacement(source, pattern, is_unicode)
711 if is_group:
712 # It's a group, so first flush the literal.
713 if literal:
714 compiled.append(make_string(literal))
715 literal = []
716 compiled.extend(items)
717 else:
718 literal.extend(items)
719 else:
720 literal.append(ord(ch))
722 # Flush the literal.
723 if literal:
724 compiled.append(make_string(literal))
726 _replacement_cache[key] = compiled
728 return compiled
730# We define Pattern here after all the support objects have been defined.
731_pat = _compile('', 0, False, {}, False)
732Pattern = type(_pat)
733Match = type(_pat.match(''))
734del _pat
736# Make Pattern public for typing annotations.
737__all__.append("Pattern")
738__all__.append("Match")
740# We'll define an alias for the 'compile' function so that the repr of a
741# pattern object is eval-able.
742Regex = compile
744# Register myself for pickling.
745import copyreg as _copy_reg
747def _pickle(pattern):
748 return _regex.compile, pattern._pickled_data
750_copy_reg.pickle(Pattern, _pickle)