Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1"""Configuration file parser.
3A configuration file consists of sections, lead by a "[section]" header,
4and followed by "name: value" entries, with continuations and such in
5the style of RFC 822.
7Intrinsic defaults can be specified by passing them into the
8ConfigParser constructor as a dictionary.
10class:
12ConfigParser -- responsible for parsing a list of
13 configuration files, and managing the parsed database.
15 methods:
17 __init__(defaults=None, dict_type=_default_dict, allow_no_value=False,
18 delimiters=('=', ':'), comment_prefixes=('#', ';'),
19 inline_comment_prefixes=None, strict=True,
20 empty_lines_in_values=True, default_section='DEFAULT',
21 interpolation=<unset>, converters=<unset>,
22 allow_unnamed_section=False):
23 Create the parser. When `defaults` is given, it is initialized into the
24 dictionary or intrinsic defaults. The keys must be strings, the values
25 must be appropriate for %()s string interpolation.
27 When `dict_type` is given, it will be used to create the dictionary
28 objects for the list of sections, for the options within a section, and
29 for the default values.
31 When `delimiters` is given, it will be used as the set of substrings
32 that divide keys from values.
34 When `comment_prefixes` is given, it will be used as the set of
35 substrings that prefix comments in empty lines. Comments can be
36 indented.
38 When `inline_comment_prefixes` is given, it will be used as the set of
39 substrings that prefix comments in non-empty lines.
41 When `strict` is True, the parser won't allow for any section or option
42 duplicates while reading from a single source (file, string or
43 dictionary). Default is True.
45 When `empty_lines_in_values` is False (default: True), each empty line
46 marks the end of an option. Otherwise, internal empty lines of
47 a multiline option are kept as part of the value.
49 When `allow_no_value` is True (default: False), options without
50 values are accepted; the value presented for these is None.
52 When `default_section` is given, the name of the special section is
53 named accordingly. By default it is called ``"DEFAULT"`` but this can
54 be customized to point to any other valid section name. Its current
55 value can be retrieved using the ``parser_instance.default_section``
56 attribute and may be modified at runtime.
58 When `interpolation` is given, it should be an Interpolation subclass
59 instance. It will be used as the handler for option value
60 pre-processing when using getters. RawConfigParser objects don't do
61 any sort of interpolation, whereas ConfigParser uses an instance of
62 BasicInterpolation. The library also provides a ``zc.buildout``
63 inspired ExtendedInterpolation implementation.
65 When `converters` is given, it should be a dictionary where each key
66 represents the name of a type converter and each value is a callable
67 implementing the conversion from string to the desired datatype. Every
68 converter gets its corresponding get*() method on the parser object and
69 section proxies.
71 When `allow_unnamed_section` is True (default: False), options
72 without section are accepted: the section for these is
73 ``configparser.UNNAMED_SECTION``.
75 sections()
76 Return all the configuration section names, sans DEFAULT.
78 has_section(section)
79 Return whether the given section exists.
81 has_option(section, option)
82 Return whether the given option exists in the given section.
84 options(section)
85 Return list of configuration options for the named section.
87 read(filenames, encoding=None)
88 Read and parse the iterable of named configuration files, given by
89 name. A single filename is also allowed. Non-existing files
90 are ignored. Return list of successfully read files.
92 read_file(f, filename=None)
93 Read and parse one configuration file, given as a file object.
94 The filename defaults to f.name; it is only used in error
95 messages (if f has no `name` attribute, the string `<???>` is used).
97 read_string(string)
98 Read configuration from a given string.
100 read_dict(dictionary)
101 Read configuration from a dictionary. Keys are section names,
102 values are dictionaries with keys and values that should be present
103 in the section. If the used dictionary type preserves order, sections
104 and their keys will be added in order. Values are automatically
105 converted to strings.
107 get(section, option, raw=False, vars=None, fallback=_UNSET)
108 Return a string value for the named option. All % interpolations are
109 expanded in the return values, based on the defaults passed into the
110 constructor and the DEFAULT section. Additional substitutions may be
111 provided using the `vars` argument, which must be a dictionary whose
112 contents override any pre-existing defaults. If `option` is a key in
113 `vars`, the value from `vars` is used.
115 getint(section, options, raw=False, vars=None, fallback=_UNSET)
116 Like get(), but convert value to an integer.
118 getfloat(section, options, raw=False, vars=None, fallback=_UNSET)
119 Like get(), but convert value to a float.
121 getboolean(section, options, raw=False, vars=None, fallback=_UNSET)
122 Like get(), but convert value to a boolean (currently case
123 insensitively defined as 0, false, no, off for False, and 1, true,
124 yes, on for True). Returns False or True.
126 items(section=_UNSET, raw=False, vars=None)
127 If section is given, return a list of tuples with (name, value) for
128 each option in the section. Otherwise, return a list of tuples with
129 (section_name, section_proxy) for each section, including DEFAULTSECT.
131 remove_section(section)
132 Remove the given file section and all its options.
134 remove_option(section, option)
135 Remove the given option from the given section.
137 set(section, option, value)
138 Set the given option.
140 write(fp, space_around_delimiters=True)
141 Write the configuration state in .ini format. If
142 `space_around_delimiters` is True (the default), delimiters
143 between keys and values are surrounded by spaces.
144"""
146from __future__ import annotations
148import contextlib
149import functools
150import itertools
151import os
152import re
153import sys
154from collections import ChainMap as _ChainMap
156# Do not import dataclasses; overhead is unacceptable (gh-117703)
157from collections.abc import Iterable, MutableMapping
159from .compat.py39 import io
161__all__ = (
162 "NoSectionError",
163 "DuplicateOptionError",
164 "DuplicateSectionError",
165 "NoOptionError",
166 "InterpolationError",
167 "InterpolationDepthError",
168 "InterpolationMissingOptionError",
169 "InterpolationSyntaxError",
170 "ParsingError",
171 "MissingSectionHeaderError",
172 "MultilineContinuationError",
173 "UnnamedSectionDisabledError",
174 "InvalidWriteError",
175 "ConfigParser",
176 "RawConfigParser",
177 "Interpolation",
178 "BasicInterpolation",
179 "ExtendedInterpolation",
180 "SectionProxy",
181 "ConverterMapping",
182 "DEFAULTSECT",
183 "MAX_INTERPOLATION_DEPTH",
184 "UNNAMED_SECTION",
185)
187_default_dict = dict
188DEFAULTSECT = "DEFAULT"
190MAX_INTERPOLATION_DEPTH = 10
193# exception classes
194class Error(Exception):
195 """Base class for ConfigParser exceptions."""
197 def __init__(self, msg=''):
198 self.message = msg
199 Exception.__init__(self, msg)
201 def __repr__(self):
202 return self.message
204 __str__ = __repr__
207class NoSectionError(Error):
208 """Raised when no section matches a requested option."""
210 def __init__(self, section):
211 Error.__init__(self, 'No section: %r' % (section,))
212 self.section = section
213 self.args = (section,)
216class DuplicateSectionError(Error):
217 """Raised when a section is repeated in an input source.
219 Possible repetitions that raise this exception are: multiple creation
220 using the API or in strict parsers when a section is found more than once
221 in a single input file, string or dictionary.
222 """
224 def __init__(self, section, source=None, lineno=None):
225 msg = [repr(section), " already exists"]
226 if source is not None:
227 message = ["While reading from ", repr(source)]
228 if lineno is not None:
229 message.append(" [line {0:2d}]".format(lineno))
230 message.append(": section ")
231 message.extend(msg)
232 msg = message
233 else:
234 msg.insert(0, "Section ")
235 Error.__init__(self, "".join(msg))
236 self.section = section
237 self.source = source
238 self.lineno = lineno
239 self.args = (section, source, lineno)
242class DuplicateOptionError(Error):
243 """Raised by strict parsers when an option is repeated in an input source.
245 Current implementation raises this exception only when an option is found
246 more than once in a single file, string or dictionary.
247 """
249 def __init__(self, section, option, source=None, lineno=None):
250 msg = [repr(option), " in section ", repr(section), " already exists"]
251 if source is not None:
252 message = ["While reading from ", repr(source)]
253 if lineno is not None:
254 message.append(" [line {0:2d}]".format(lineno))
255 message.append(": option ")
256 message.extend(msg)
257 msg = message
258 else:
259 msg.insert(0, "Option ")
260 Error.__init__(self, "".join(msg))
261 self.section = section
262 self.option = option
263 self.source = source
264 self.lineno = lineno
265 self.args = (section, option, source, lineno)
268class NoOptionError(Error):
269 """A requested option was not found."""
271 def __init__(self, option, section):
272 Error.__init__(self, "No option %r in section: %r" % (option, section))
273 self.option = option
274 self.section = section
275 self.args = (option, section)
278class InterpolationError(Error):
279 """Base class for interpolation-related exceptions."""
281 def __init__(self, option, section, msg):
282 Error.__init__(self, msg)
283 self.option = option
284 self.section = section
285 self.args = (option, section, msg)
288class InterpolationMissingOptionError(InterpolationError):
289 """A string substitution required a setting which was not available."""
291 def __init__(self, option, section, rawval, reference):
292 msg = (
293 "Bad value substitution: option {!r} in section {!r} contains "
294 "an interpolation key {!r} which is not a valid option name. "
295 "Raw value: {!r}".format(option, section, reference, rawval)
296 )
297 InterpolationError.__init__(self, option, section, msg)
298 self.reference = reference
299 self.args = (option, section, rawval, reference)
302class InterpolationSyntaxError(InterpolationError):
303 """Raised when the source text contains invalid syntax.
305 Current implementation raises this exception when the source text into
306 which substitutions are made does not conform to the required syntax.
307 """
310class InterpolationDepthError(InterpolationError):
311 """Raised when substitutions are nested too deeply."""
313 def __init__(self, option, section, rawval):
314 msg = (
315 "Recursion limit exceeded in value substitution: option {!r} "
316 "in section {!r} contains an interpolation key which "
317 "cannot be substituted in {} steps. Raw value: {!r}"
318 "".format(option, section, MAX_INTERPOLATION_DEPTH, rawval)
319 )
320 InterpolationError.__init__(self, option, section, msg)
321 self.args = (option, section, rawval)
324class ParsingError(Error):
325 """Raised when a configuration file does not follow legal syntax."""
327 def __init__(self, source, *args):
328 super().__init__(f'Source contains parsing errors: {source!r}')
329 self.source = source
330 self.errors = []
331 self.args = (source,)
332 if args:
333 self.append(*args)
335 def append(self, lineno, line):
336 self.errors.append((lineno, line))
337 self.message += '\n\t[line %2d]: %s' % (lineno, repr(line))
339 def combine(self, others):
340 for other in others:
341 for error in other.errors:
342 self.append(*error)
343 return self
345 @staticmethod
346 def _raise_all(exceptions: Iterable[ParsingError]):
347 """
348 Combine any number of ParsingErrors into one and raise it.
349 """
350 exceptions = iter(exceptions)
351 with contextlib.suppress(StopIteration):
352 raise next(exceptions).combine(exceptions)
355class MissingSectionHeaderError(ParsingError):
356 """Raised when a key-value pair is found before any section header."""
358 def __init__(self, filename, lineno, line):
359 Error.__init__(
360 self,
361 'File contains no section headers.\nfile: %r, line: %d\n%r'
362 % (filename, lineno, line),
363 )
364 self.source = filename
365 self.lineno = lineno
366 self.line = line
367 self.args = (filename, lineno, line)
370class MultilineContinuationError(ParsingError):
371 """Raised when a key without value is followed by continuation line"""
373 def __init__(self, filename, lineno, line):
374 Error.__init__(
375 self,
376 "Key without value continued with an indented line.\n"
377 "file: %r, line: %d\n%r" % (filename, lineno, line),
378 )
379 self.source = filename
380 self.lineno = lineno
381 self.line = line
382 self.args = (filename, lineno, line)
385class UnnamedSectionDisabledError(Error):
386 """Raised when an attempt to use UNNAMED_SECTION is made with the
387 feature disabled."""
389 def __init__(self):
390 Error.__init__(self, "Support for UNNAMED_SECTION is disabled.")
393class _UnnamedSection:
394 def __repr__(self):
395 return "<UNNAMED_SECTION>"
398class InvalidWriteError(Error):
399 """Raised when attempting to write data that the parser would read back differently.
400 ex: writing a key which begins with the section header pattern would read back as a
401 new section"""
403 def __init__(self, msg=''):
404 Error.__init__(self, msg)
407UNNAMED_SECTION = _UnnamedSection()
410# Used in parser getters to indicate the default behaviour when a specific
411# option is not found it to raise an exception. Created to enable `None` as
412# a valid fallback value.
413_UNSET = object()
416class Interpolation:
417 """Dummy interpolation that passes the value through with no changes."""
419 def before_get(self, parser, section, option, value, defaults):
420 return value
422 def before_set(self, parser, section, option, value):
423 return value
425 def before_read(self, parser, section, option, value):
426 return value
428 def before_write(self, parser, section, option, value):
429 return value
432class BasicInterpolation(Interpolation):
433 """Interpolation as implemented in the classic ConfigParser.
435 The option values can contain format strings which refer to other values in
436 the same section, or values in the special default section.
438 For example:
440 something: %(dir)s/whatever
442 would resolve the "%(dir)s" to the value of dir. All reference
443 expansions are done late, on demand. If a user needs to use a bare % in
444 a configuration file, she can escape it by writing %%. Other % usage
445 is considered a user error and raises `InterpolationSyntaxError`."""
447 _KEYCRE = re.compile(r"%\(([^)]+)\)s")
449 def before_get(self, parser, section, option, value, defaults):
450 L = []
451 self._interpolate_some(parser, option, L, value, section, defaults, 1)
452 return ''.join(L)
454 def before_set(self, parser, section, option, value):
455 tmp_value = value.replace('%%', '') # escaped percent signs
456 tmp_value = self._KEYCRE.sub('', tmp_value) # valid syntax
457 if '%' in tmp_value:
458 raise ValueError(
459 "invalid interpolation syntax in %r at "
460 "position %d" % (value, tmp_value.find('%'))
461 )
462 return value
464 def _interpolate_some( # noqa: C901
465 self, parser, option, accum, rest, section, map, depth
466 ):
467 rawval = parser.get(section, option, raw=True, fallback=rest)
468 if depth > MAX_INTERPOLATION_DEPTH:
469 raise InterpolationDepthError(option, section, rawval)
470 while rest:
471 p = rest.find("%")
472 if p < 0:
473 accum.append(rest)
474 return
475 if p > 0:
476 accum.append(rest[:p])
477 rest = rest[p:]
478 # p is no longer used
479 c = rest[1:2]
480 if c == "%":
481 accum.append("%")
482 rest = rest[2:]
483 elif c == "(":
484 m = self._KEYCRE.match(rest)
485 if m is None:
486 raise InterpolationSyntaxError(
487 option,
488 section,
489 "bad interpolation variable reference %r" % rest,
490 )
491 var = parser.optionxform(m.group(1))
492 rest = rest[m.end() :]
493 try:
494 v = map[var]
495 except KeyError:
496 raise InterpolationMissingOptionError(
497 option, section, rawval, var
498 ) from None
499 if "%" in v:
500 self._interpolate_some(
501 parser, option, accum, v, section, map, depth + 1
502 )
503 else:
504 accum.append(v)
505 else:
506 raise InterpolationSyntaxError(
507 option,
508 section,
509 "'%%' must be followed by '%%' or '(', found: %r" % (rest,),
510 )
513class ExtendedInterpolation(Interpolation):
514 """Advanced variant of interpolation, supports the syntax used by
515 `zc.buildout`. Enables interpolation between sections."""
517 _KEYCRE = re.compile(r"\$\{([^}]+)\}")
519 def before_get(self, parser, section, option, value, defaults):
520 L = []
521 self._interpolate_some(parser, option, L, value, section, defaults, 1)
522 return ''.join(L)
524 def before_set(self, parser, section, option, value):
525 tmp_value = value.replace('$$', '') # escaped dollar signs
526 tmp_value = self._KEYCRE.sub('', tmp_value) # valid syntax
527 if '$' in tmp_value:
528 raise ValueError(
529 "invalid interpolation syntax in %r at "
530 "position %d" % (value, tmp_value.find('$'))
531 )
532 return value
534 def _interpolate_some( # noqa: C901
535 self, parser, option, accum, rest, section, map, depth
536 ):
537 rawval = parser.get(section, option, raw=True, fallback=rest)
538 if depth > MAX_INTERPOLATION_DEPTH:
539 raise InterpolationDepthError(option, section, rawval)
540 while rest:
541 p = rest.find("$")
542 if p < 0:
543 accum.append(rest)
544 return
545 if p > 0:
546 accum.append(rest[:p])
547 rest = rest[p:]
548 # p is no longer used
549 c = rest[1:2]
550 if c == "$":
551 accum.append("$")
552 rest = rest[2:]
553 elif c == "{":
554 m = self._KEYCRE.match(rest)
555 if m is None:
556 raise InterpolationSyntaxError(
557 option,
558 section,
559 "bad interpolation variable reference %r" % rest,
560 )
561 path = m.group(1).split(':')
562 rest = rest[m.end() :]
563 sect = section
564 opt = option
565 try:
566 if len(path) == 1:
567 opt = parser.optionxform(path[0])
568 v = map[opt]
569 elif len(path) == 2:
570 sect = path[0]
571 opt = parser.optionxform(path[1])
572 v = parser.get(sect, opt, raw=True)
573 else:
574 raise InterpolationSyntaxError(
575 option, section, "More than one ':' found: %r" % (rest,)
576 )
577 except (KeyError, NoSectionError, NoOptionError):
578 raise InterpolationMissingOptionError(
579 option, section, rawval, ":".join(path)
580 ) from None
581 if "$" in v:
582 self._interpolate_some(
583 parser,
584 opt,
585 accum,
586 v,
587 sect,
588 dict(parser.items(sect, raw=True)),
589 depth + 1,
590 )
591 else:
592 accum.append(v)
593 else:
594 raise InterpolationSyntaxError(
595 option,
596 section,
597 "'$' must be followed by '$' or '{', found: %r" % (rest,),
598 )
601class _ReadState:
602 elements_added: set[str]
603 cursect: dict[str, str] | None = None
604 sectname: str | None = None
605 optname: str | None = None
606 lineno: int = 0
607 indent_level: int = 0
608 errors: list[ParsingError]
610 def __init__(self):
611 self.elements_added = set()
612 self.errors = list()
615class _Line(str):
616 __slots__ = 'clean', 'has_comments'
618 def __new__(cls, val, *args, **kwargs):
619 return super().__new__(cls, val)
621 def __init__(self, val, comments):
622 trimmed = val.strip()
623 self.clean = comments.strip(trimmed)
624 self.has_comments = trimmed != self.clean
627class _CommentSpec:
628 def __init__(self, full_prefixes, inline_prefixes):
629 full_patterns = (
630 # prefix at the beginning of a line
631 rf'^({re.escape(prefix)}).*'
632 for prefix in full_prefixes
633 )
634 inline_patterns = (
635 # prefix at the beginning of the line or following a space
636 rf'(^|\s)({re.escape(prefix)}.*)'
637 for prefix in inline_prefixes
638 )
639 self.pattern = re.compile(
640 '|'.join(itertools.chain(full_patterns, inline_patterns))
641 )
643 def strip(self, text):
644 return self.pattern.sub('', text).rstrip()
646 def wrap(self, text):
647 return _Line(text, self)
650class RawConfigParser(MutableMapping):
651 """ConfigParser that does not do interpolation."""
653 # Regular expressions for parsing section headers and options
654 _SECT_TMPL = r"""
655 \[ # [
656 (?P<header>.+) # very permissive!
657 \] # ]
658 """
659 _OPT_TMPL = r"""
660 (?P<option>.*?) # very permissive!
661 \s*(?P<vi>{delim})\s* # any number of space/tab,
662 # followed by any of the
663 # allowed delimiters,
664 # followed by any space/tab
665 (?P<value>.*)$ # everything up to eol
666 """
667 _OPT_NV_TMPL = r"""
668 (?P<option>.*?) # very permissive!
669 \s*(?: # any number of space/tab,
670 (?P<vi>{delim})\s* # optionally followed by
671 # any of the allowed
672 # delimiters, followed by any
673 # space/tab
674 (?P<value>.*))?$ # everything up to eol
675 """
676 # Interpolation algorithm to be used if the user does not specify another
677 _DEFAULT_INTERPOLATION = Interpolation()
678 # Compiled regular expression for matching sections
679 SECTCRE = re.compile(_SECT_TMPL, re.VERBOSE)
680 # Compiled regular expression for matching options with typical separators
681 OPTCRE = re.compile(_OPT_TMPL.format(delim="=|:"), re.VERBOSE)
682 # Compiled regular expression for matching options with optional values
683 # delimited using typical separators
684 OPTCRE_NV = re.compile(_OPT_NV_TMPL.format(delim="=|:"), re.VERBOSE)
685 # Compiled regular expression for matching leading whitespace in a line
686 NONSPACECRE = re.compile(r"\S")
687 # Possible boolean values in the configuration.
688 BOOLEAN_STATES = {
689 '1': True,
690 'yes': True,
691 'true': True,
692 'on': True,
693 '0': False,
694 'no': False,
695 'false': False,
696 'off': False,
697 }
699 def __init__(
700 self,
701 defaults=None,
702 dict_type=_default_dict,
703 allow_no_value=False,
704 *,
705 delimiters=('=', ':'),
706 comment_prefixes=('#', ';'),
707 inline_comment_prefixes=None,
708 strict=True,
709 empty_lines_in_values=True,
710 default_section=DEFAULTSECT,
711 interpolation=_UNSET,
712 converters=_UNSET,
713 allow_unnamed_section=False,
714 ):
715 self._dict = dict_type
716 self._sections = self._dict()
717 self._defaults = self._dict()
718 self._converters = ConverterMapping(self)
719 self._proxies = self._dict()
720 self._proxies[default_section] = SectionProxy(self, default_section)
721 self._delimiters = tuple(delimiters)
722 if delimiters == ('=', ':'):
723 self._optcre = self.OPTCRE_NV if allow_no_value else self.OPTCRE
724 else:
725 d = "|".join(re.escape(d) for d in delimiters)
726 if allow_no_value:
727 self._optcre = re.compile(self._OPT_NV_TMPL.format(delim=d), re.VERBOSE)
728 else:
729 self._optcre = re.compile(self._OPT_TMPL.format(delim=d), re.VERBOSE)
730 self._comments = _CommentSpec(
731 comment_prefixes or (), inline_comment_prefixes or ()
732 )
733 self._strict = strict
734 self._allow_no_value = allow_no_value
735 self._empty_lines_in_values = empty_lines_in_values
736 self.default_section = default_section
737 self._interpolation = interpolation
738 if self._interpolation is _UNSET:
739 self._interpolation = self._DEFAULT_INTERPOLATION
740 if self._interpolation is None:
741 self._interpolation = Interpolation()
742 if not isinstance(self._interpolation, Interpolation):
743 raise TypeError(
744 f"interpolation= must be None or an instance of Interpolation;"
745 f" got an object of type {type(self._interpolation)}"
746 )
747 if converters is not _UNSET:
748 self._converters.update(converters)
749 if defaults:
750 self._read_defaults(defaults)
751 self._allow_unnamed_section = allow_unnamed_section
753 def defaults(self):
754 return self._defaults
756 def sections(self):
757 """Return a list of section names, excluding [DEFAULT]"""
758 # self._sections will never have [DEFAULT] in it
759 return list(self._sections.keys())
761 def add_section(self, section):
762 """Create a new section in the configuration.
764 Raise DuplicateSectionError if a section by the specified name
765 already exists. Raise ValueError if name is DEFAULT.
766 """
767 if section == self.default_section:
768 raise ValueError('Invalid section name: %r' % section)
770 if section is UNNAMED_SECTION:
771 if not self._allow_unnamed_section:
772 raise UnnamedSectionDisabledError
774 if section in self._sections:
775 raise DuplicateSectionError(section)
776 self._sections[section] = self._dict()
777 self._proxies[section] = SectionProxy(self, section)
779 def has_section(self, section):
780 """Indicate whether the named section is present in the configuration.
782 The DEFAULT section is not acknowledged.
783 """
784 return section in self._sections
786 def options(self, section):
787 """Return a list of option names for the given section name."""
788 try:
789 opts = self._sections[section].copy()
790 except KeyError:
791 raise NoSectionError(section) from None
792 opts.update(self._defaults)
793 return list(opts.keys())
795 def read(self, filenames, encoding=None):
796 """Read and parse a filename or an iterable of filenames.
798 Files that cannot be opened are silently ignored; this is
799 designed so that you can specify an iterable of potential
800 configuration file locations (e.g. current directory, user's
801 home directory, systemwide directory), and all existing
802 configuration files in the iterable will be read. A single
803 filename may also be given.
805 Return list of successfully read files.
806 """
807 if isinstance(filenames, (str, bytes, os.PathLike)):
808 filenames = [filenames]
809 encoding = io.text_encoding(encoding)
810 read_ok = []
811 for filename in filenames:
812 try:
813 with open(filename, encoding=encoding) as fp:
814 self._read(fp, filename)
815 except OSError:
816 continue
817 if isinstance(filename, os.PathLike):
818 filename = os.fspath(filename)
819 read_ok.append(filename)
820 return read_ok
822 def read_file(self, f, source=None):
823 """Like read() but the argument must be a file-like object.
825 The `f` argument must be iterable, returning one line at a time.
826 Optional second argument is the `source` specifying the name of the
827 file being read. If not given, it is taken from f.name. If `f` has no
828 `name` attribute, `<???>` is used.
829 """
830 if source is None:
831 try:
832 source = f.name
833 except AttributeError:
834 source = '<???>'
835 self._read(f, source)
837 def read_string(self, string, source='<string>'):
838 """Read configuration from a given string."""
839 sfile = io.StringIO(string)
840 self.read_file(sfile, source)
842 def read_dict(self, dictionary, source='<dict>'):
843 """Read configuration from a dictionary.
845 Keys are section names, values are dictionaries with keys and values
846 that should be present in the section. If the used dictionary type
847 preserves order, sections and their keys will be added in order.
849 All types held in the dictionary are converted to strings during
850 reading, including section names, option names and keys.
852 Optional second argument is the `source` specifying the name of the
853 dictionary being read.
854 """
855 elements_added = set()
856 for section, keys in dictionary.items():
857 section = str(section)
858 try:
859 self.add_section(section)
860 except (DuplicateSectionError, ValueError):
861 if self._strict and section in elements_added:
862 raise
863 elements_added.add(section)
864 for key, value in keys.items():
865 key = self.optionxform(str(key))
866 if value is not None:
867 value = str(value)
868 if self._strict and (section, key) in elements_added:
869 raise DuplicateOptionError(section, key, source)
870 elements_added.add((section, key))
871 self.set(section, key, value)
873 def get(self, section, option, *, raw=False, vars=None, fallback=_UNSET):
874 """Get an option value for a given section.
876 If `vars` is provided, it must be a dictionary. The option is looked up
877 in `vars` (if provided), `section`, and in `DEFAULTSECT` in that order.
878 If the key is not found and `fallback` is provided, it is used as
879 a fallback value. `None` can be provided as a `fallback` value.
881 If interpolation is enabled and the optional argument `raw` is False,
882 all interpolations are expanded in the return values.
884 Arguments `raw`, `vars`, and `fallback` are keyword only.
886 The section DEFAULT is special.
887 """
888 try:
889 d = self._unify_values(section, vars)
890 except NoSectionError:
891 if fallback is _UNSET:
892 raise
893 else:
894 return fallback
895 option = self.optionxform(option)
896 try:
897 value = d[option]
898 except KeyError:
899 if fallback is _UNSET:
900 raise NoOptionError(option, section)
901 else:
902 return fallback
904 if raw or value is None:
905 return value
906 else:
907 return self._interpolation.before_get(self, section, option, value, d)
909 def _get(self, section, conv, option, **kwargs):
910 return conv(self.get(section, option, **kwargs))
912 def _get_conv(
913 self, section, option, conv, *, raw=False, vars=None, fallback=_UNSET, **kwargs
914 ):
915 try:
916 return self._get(section, conv, option, raw=raw, vars=vars, **kwargs)
917 except (NoSectionError, NoOptionError):
918 if fallback is _UNSET:
919 raise
920 return fallback
922 # getint, getfloat and getboolean provided directly for backwards compat
923 def getint(
924 self, section, option, *, raw=False, vars=None, fallback=_UNSET, **kwargs
925 ):
926 return self._get_conv(
927 section, option, int, raw=raw, vars=vars, fallback=fallback, **kwargs
928 )
930 def getfloat(
931 self, section, option, *, raw=False, vars=None, fallback=_UNSET, **kwargs
932 ):
933 return self._get_conv(
934 section, option, float, raw=raw, vars=vars, fallback=fallback, **kwargs
935 )
937 def getboolean(
938 self, section, option, *, raw=False, vars=None, fallback=_UNSET, **kwargs
939 ):
940 return self._get_conv(
941 section,
942 option,
943 self._convert_to_boolean,
944 raw=raw,
945 vars=vars,
946 fallback=fallback,
947 **kwargs,
948 )
950 def items(self, section=_UNSET, raw=False, vars=None):
951 """Return a list of (name, value) tuples for each option in a section.
953 All % interpolations are expanded in the return values, based on the
954 defaults passed into the constructor, unless the optional argument
955 `raw` is true. Additional substitutions may be provided using the
956 `vars` argument, which must be a dictionary whose contents overrides
957 any pre-existing defaults.
959 The section DEFAULT is special.
960 """
961 if section is _UNSET:
962 return super(RawConfigParser, self).items()
963 d = self._defaults.copy()
964 try:
965 d.update(self._sections[section])
966 except KeyError:
967 if section != self.default_section:
968 raise NoSectionError(section)
969 orig_keys = list(d.keys())
970 # Update with the entry specific variables
971 if vars:
972 for key, value in vars.items():
973 d[self.optionxform(key)] = value
975 def value_getter_interp(option):
976 return self._interpolation.before_get(self, section, option, d[option], d)
978 def value_getter_raw(option):
979 return d[option]
981 value_getter = value_getter_raw if raw else value_getter_interp
983 return [(option, value_getter(option)) for option in orig_keys]
985 def popitem(self):
986 """Remove a section from the parser and return it as
987 a (section_name, section_proxy) tuple. If no section is present, raise
988 KeyError.
990 The section DEFAULT is never returned because it cannot be removed.
991 """
992 for key in self.sections():
993 value = self[key]
994 del self[key]
995 return key, value
996 raise KeyError
998 def optionxform(self, optionstr):
999 return optionstr.lower()
1001 def has_option(self, section, option):
1002 """Check for the existence of a given option in a given section.
1003 If the specified `section` is None or an empty string, DEFAULT is
1004 assumed. If the specified `section` does not exist, returns False."""
1005 if not section or section == self.default_section:
1006 option = self.optionxform(option)
1007 return option in self._defaults
1008 elif section not in self._sections:
1009 return False
1010 else:
1011 option = self.optionxform(option)
1012 return option in self._sections[section] or option in self._defaults
1014 def set(self, section, option, value=None):
1015 """Set an option."""
1016 if value:
1017 value = self._interpolation.before_set(self, section, option, value)
1018 if not section or section == self.default_section:
1019 sectdict = self._defaults
1020 else:
1021 try:
1022 sectdict = self._sections[section]
1023 except KeyError:
1024 raise NoSectionError(section) from None
1025 sectdict[self.optionxform(option)] = value
1027 def write(self, fp, space_around_delimiters=True):
1028 """Write an .ini-format representation of the configuration state.
1030 If `space_around_delimiters` is True (the default), delimiters
1031 between keys and values are surrounded by spaces.
1033 Please note that comments in the original configuration file are not
1034 preserved when writing the configuration back.
1035 """
1036 if space_around_delimiters:
1037 d = " {} ".format(self._delimiters[0])
1038 else:
1039 d = self._delimiters[0]
1040 if self._defaults:
1041 self._write_section(fp, self.default_section, self._defaults.items(), d)
1042 if UNNAMED_SECTION in self._sections and self._sections[UNNAMED_SECTION]:
1043 self._write_section(
1044 fp,
1045 UNNAMED_SECTION,
1046 self._sections[UNNAMED_SECTION].items(),
1047 d,
1048 unnamed=True,
1049 )
1051 for section in self._sections:
1052 if section is UNNAMED_SECTION:
1053 continue
1054 self._write_section(fp, section, self._sections[section].items(), d)
1056 def _write_section(self, fp, section_name, section_items, delimiter, unnamed=False):
1057 """Write a single section to the specified 'fp'."""
1058 if not unnamed:
1059 fp.write("[{}]\n".format(section_name))
1060 for key, value in section_items:
1061 self._validate_key_contents(key)
1062 value = self._interpolation.before_write(self, section_name, key, value)
1063 if value is not None or not self._allow_no_value:
1064 value = delimiter + str(value).replace('\n', '\n\t')
1065 else:
1066 value = ""
1067 fp.write("{}{}\n".format(key, value))
1068 fp.write("\n")
1070 def remove_option(self, section, option):
1071 """Remove an option."""
1072 if not section or section == self.default_section:
1073 sectdict = self._defaults
1074 else:
1075 try:
1076 sectdict = self._sections[section]
1077 except KeyError:
1078 raise NoSectionError(section) from None
1079 option = self.optionxform(option)
1080 existed = option in sectdict
1081 if existed:
1082 del sectdict[option]
1083 return existed
1085 def remove_section(self, section):
1086 """Remove a file section."""
1087 existed = section in self._sections
1088 if existed:
1089 del self._sections[section]
1090 del self._proxies[section]
1091 return existed
1093 def __getitem__(self, key):
1094 if key != self.default_section and not self.has_section(key):
1095 raise KeyError(key)
1096 return self._proxies[key]
1098 def __setitem__(self, key, value):
1099 # To conform with the mapping protocol, overwrites existing values in
1100 # the section.
1101 if key in self and self[key] is value:
1102 return
1103 # XXX this is not atomic if read_dict fails at any point. Then again,
1104 # no update method in configparser is atomic in this implementation.
1105 if key == self.default_section:
1106 self._defaults.clear()
1107 elif key in self._sections:
1108 self._sections[key].clear()
1109 self.read_dict({key: value})
1111 def __delitem__(self, key):
1112 if key == self.default_section:
1113 raise ValueError("Cannot remove the default section.")
1114 if not self.has_section(key):
1115 raise KeyError(key)
1116 self.remove_section(key)
1118 def __contains__(self, key):
1119 return key == self.default_section or self.has_section(key)
1121 def __len__(self):
1122 return len(self._sections) + 1 # the default section
1124 def __iter__(self):
1125 # XXX does it break when underlying container state changed?
1126 return itertools.chain((self.default_section,), self._sections.keys())
1128 def _read(self, fp, fpname): # noqa: C901
1129 """Parse a sectioned configuration file.
1131 Each section in a configuration file contains a header, indicated by
1132 a name in square brackets (`[]`), plus key/value options, indicated by
1133 `name` and `value` delimited with a specific substring (`=` or `:` by
1134 default).
1136 Values can span multiple lines, as long as they are indented deeper
1137 than the first line of the value. Depending on the parser's mode, blank
1138 lines may be treated as parts of multiline values or ignored.
1140 Configuration files may include comments, prefixed by specific
1141 characters (`#` and `;` by default). Comments may appear on their own
1142 in an otherwise empty line or may be entered in lines holding values or
1143 section names. Please note that comments get stripped off when reading
1144 configuration files.
1145 """
1146 try:
1147 ParsingError._raise_all(self._read_inner(fp, fpname))
1148 finally:
1149 self._join_multiline_values()
1151 def _read_inner(self, fp, fpname):
1152 st = _ReadState()
1154 for st.lineno, line in enumerate(map(self._comments.wrap, fp), start=1):
1155 if not line.clean:
1156 if self._empty_lines_in_values:
1157 # add empty line to the value, but only if there was no
1158 # comment on the line
1159 if (
1160 not line.has_comments
1161 and st.cursect is not None
1162 and st.optname
1163 and st.cursect[st.optname] is not None
1164 ):
1165 st.cursect[st.optname].append('') # newlines added at join
1166 else:
1167 # empty line marks end of value
1168 st.indent_level = sys.maxsize
1169 continue
1171 first_nonspace = self.NONSPACECRE.search(line)
1172 st.cur_indent_level = first_nonspace.start() if first_nonspace else 0
1174 if self._handle_continuation_line(st, line, fpname):
1175 continue
1177 self._handle_rest(st, line, fpname)
1179 return st.errors
1181 def _handle_continuation_line(self, st, line, fpname):
1182 # continuation line?
1183 is_continue = (
1184 st.cursect is not None
1185 and st.optname
1186 and st.cur_indent_level > st.indent_level
1187 )
1188 if is_continue:
1189 if st.cursect[st.optname] is None:
1190 raise MultilineContinuationError(fpname, st.lineno, line)
1191 st.cursect[st.optname].append(line.clean)
1192 return is_continue
1194 def _handle_rest(self, st, line, fpname):
1195 # a section header or option header?
1196 if self._allow_unnamed_section and st.cursect is None:
1197 self._handle_header(st, UNNAMED_SECTION, fpname)
1199 st.indent_level = st.cur_indent_level
1200 # is it a section header?
1201 mo = self.SECTCRE.match(line.clean)
1203 if not mo and st.cursect is None:
1204 raise MissingSectionHeaderError(fpname, st.lineno, line)
1206 self._handle_header(
1207 st, mo.group('header'), fpname
1208 ) if mo else self._handle_option(st, line, fpname)
1210 def _handle_header(self, st, sectname, fpname):
1211 st.sectname = sectname
1212 if st.sectname in self._sections:
1213 if self._strict and st.sectname in st.elements_added:
1214 raise DuplicateSectionError(st.sectname, fpname, st.lineno)
1215 st.cursect = self._sections[st.sectname]
1216 st.elements_added.add(st.sectname)
1217 elif st.sectname == self.default_section:
1218 st.cursect = self._defaults
1219 else:
1220 st.cursect = self._dict()
1221 self._sections[st.sectname] = st.cursect
1222 self._proxies[st.sectname] = SectionProxy(self, st.sectname)
1223 st.elements_added.add(st.sectname)
1224 # So sections can't start with a continuation line
1225 st.optname = None
1227 def _handle_option(self, st, line, fpname):
1228 # an option line?
1229 st.indent_level = st.cur_indent_level
1231 mo = self._optcre.match(line.clean)
1232 if not mo:
1233 # a non-fatal parsing error occurred. set up the
1234 # exception but keep going. the exception will be
1235 # raised at the end of the file and will contain a
1236 # list of all bogus lines
1237 st.errors.append(ParsingError(fpname, st.lineno, line))
1238 return
1240 st.optname, vi, optval = mo.group('option', 'vi', 'value')
1241 if not st.optname:
1242 st.errors.append(ParsingError(fpname, st.lineno, line))
1243 st.optname = self.optionxform(st.optname.rstrip())
1244 if self._strict and (st.sectname, st.optname) in st.elements_added:
1245 raise DuplicateOptionError(st.sectname, st.optname, fpname, st.lineno)
1246 st.elements_added.add((st.sectname, st.optname))
1247 # This check is fine because the OPTCRE cannot
1248 # match if it would set optval to None
1249 if optval is not None:
1250 optval = optval.strip()
1251 st.cursect[st.optname] = [optval]
1252 else:
1253 # valueless option handling
1254 st.cursect[st.optname] = None
1256 def _join_multiline_values(self):
1257 defaults = self.default_section, self._defaults
1258 all_sections = itertools.chain((defaults,), self._sections.items())
1259 for section, options in all_sections:
1260 for name, val in options.items():
1261 if isinstance(val, list):
1262 val = '\n'.join(val).rstrip()
1263 options[name] = self._interpolation.before_read(
1264 self, section, name, val
1265 )
1267 def _read_defaults(self, defaults):
1268 """Read the defaults passed in the initializer.
1269 Note: values can be non-string."""
1270 for key, value in defaults.items():
1271 self._defaults[self.optionxform(key)] = value
1273 def _unify_values(self, section, vars):
1274 """Create a sequence of lookups with 'vars' taking priority over
1275 the 'section' which takes priority over the DEFAULTSECT.
1277 """
1278 sectiondict = {}
1279 try:
1280 sectiondict = self._sections[section]
1281 except KeyError:
1282 if section != self.default_section:
1283 raise NoSectionError(section)
1284 # Update with the entry specific variables
1285 vardict = {}
1286 if vars:
1287 for key, value in vars.items():
1288 if value is not None:
1289 value = str(value)
1290 vardict[self.optionxform(key)] = value
1291 return _ChainMap(vardict, sectiondict, self._defaults)
1293 def _convert_to_boolean(self, value):
1294 """Return a boolean value translating from other types if necessary."""
1295 if value.lower() not in self.BOOLEAN_STATES:
1296 raise ValueError('Not a boolean: %s' % value)
1297 return self.BOOLEAN_STATES[value.lower()]
1299 def _validate_key_contents(self, key):
1300 """Raises an InvalidWriteError for any keys containing
1301 delimiters or that match the section header pattern"""
1302 if re.match(self.SECTCRE, key):
1303 raise InvalidWriteError("Cannot write keys matching section pattern")
1304 if any(delim in key for delim in self._delimiters):
1305 raise InvalidWriteError("Cannot write key that contains delimiters")
1307 def _validate_value_types(self, *, section="", option="", value=""):
1308 """Raises a TypeError for illegal non-string values.
1310 Legal non-string values are UNNAMED_SECTION and falsey values if
1311 they are allowed.
1313 For compatibility reasons this method is not used in classic set()
1314 for RawConfigParsers. It is invoked in every case for mapping protocol
1315 access and in ConfigParser.set().
1316 """
1317 if section is UNNAMED_SECTION:
1318 if not self._allow_unnamed_section:
1319 raise UnnamedSectionDisabledError
1320 elif not isinstance(section, str):
1321 raise TypeError("section names must be strings or UNNAMED_SECTION")
1322 if not isinstance(option, str):
1323 raise TypeError("option keys must be strings")
1324 if not self._allow_no_value or value:
1325 if not isinstance(value, str):
1326 raise TypeError("option values must be strings")
1328 @property
1329 def converters(self):
1330 return self._converters
1333class ConfigParser(RawConfigParser):
1334 """ConfigParser implementing interpolation."""
1336 _DEFAULT_INTERPOLATION = BasicInterpolation()
1338 def set(self, section, option, value=None):
1339 """Set an option. Extends RawConfigParser.set by validating type and
1340 interpolation syntax on the value."""
1341 self._validate_value_types(option=option, value=value)
1342 super().set(section, option, value)
1344 def add_section(self, section):
1345 """Create a new section in the configuration. Extends
1346 RawConfigParser.add_section by validating if the section name is
1347 a string."""
1348 self._validate_value_types(section=section)
1349 super().add_section(section)
1351 def _read_defaults(self, defaults):
1352 """Reads the defaults passed in the initializer, implicitly converting
1353 values to strings like the rest of the API.
1355 Does not perform interpolation for backwards compatibility.
1356 """
1357 try:
1358 hold_interpolation = self._interpolation
1359 self._interpolation = Interpolation()
1360 self.read_dict({self.default_section: defaults})
1361 finally:
1362 self._interpolation = hold_interpolation
1365class SectionProxy(MutableMapping):
1366 """A proxy for a single section from a parser."""
1368 def __init__(self, parser, name):
1369 """Creates a view on a section of the specified `name` in `parser`."""
1370 self._parser = parser
1371 self._name = name
1372 for conv in parser.converters:
1373 key = 'get' + conv
1374 getter = functools.partial(self.get, _impl=getattr(parser, key))
1375 setattr(self, key, getter)
1377 def __repr__(self):
1378 return '<Section: {}>'.format(self._name)
1380 def __getitem__(self, key):
1381 if not self._parser.has_option(self._name, key):
1382 raise KeyError(key)
1383 return self._parser.get(self._name, key)
1385 def __setitem__(self, key, value):
1386 self._parser._validate_value_types(option=key, value=value)
1387 return self._parser.set(self._name, key, value)
1389 def __delitem__(self, key):
1390 if not (
1391 self._parser.has_option(self._name, key)
1392 and self._parser.remove_option(self._name, key)
1393 ):
1394 raise KeyError(key)
1396 def __contains__(self, key):
1397 return self._parser.has_option(self._name, key)
1399 def __len__(self):
1400 return len(self._options())
1402 def __iter__(self):
1403 return self._options().__iter__()
1405 def _options(self):
1406 if self._name != self._parser.default_section:
1407 return self._parser.options(self._name)
1408 else:
1409 return self._parser.defaults()
1411 @property
1412 def parser(self):
1413 # The parser object of the proxy is read-only.
1414 return self._parser
1416 @property
1417 def name(self):
1418 # The name of the section on a proxy is read-only.
1419 return self._name
1421 def get(self, option, fallback=None, *, raw=False, vars=None, _impl=None, **kwargs):
1422 """Get an option value.
1424 Unless `fallback` is provided, `None` will be returned if the option
1425 is not found.
1427 """
1428 # If `_impl` is provided, it should be a getter method on the parser
1429 # object that provides the desired type conversion.
1430 if not _impl:
1431 _impl = self._parser.get
1432 return _impl(
1433 self._name, option, raw=raw, vars=vars, fallback=fallback, **kwargs
1434 )
1437class ConverterMapping(MutableMapping):
1438 """Enables reuse of get*() methods between the parser and section proxies.
1440 If a parser class implements a getter directly, the value for the given
1441 key will be ``None``. The presence of the converter name here enables
1442 section proxies to find and use the implementation on the parser class.
1443 """
1445 GETTERCRE = re.compile(r"^get(?P<name>.+)$")
1447 def __init__(self, parser):
1448 self._parser = parser
1449 self._data = {}
1450 for getter in dir(self._parser):
1451 m = self.GETTERCRE.match(getter)
1452 if not m or not callable(getattr(self._parser, getter)):
1453 continue
1454 self._data[m.group('name')] = None # See class docstring.
1456 def __getitem__(self, key):
1457 return self._data[key]
1459 def __setitem__(self, key, value):
1460 try:
1461 k = 'get' + key
1462 except TypeError:
1463 raise ValueError('Incompatible key: {} (type: {})'.format(key, type(key)))
1464 if k == 'get':
1465 raise ValueError('Incompatible key: cannot use "" as a name')
1466 self._data[key] = value
1467 func = functools.partial(self._parser._get_conv, conv=value)
1468 func.converter = value
1469 setattr(self._parser, k, func)
1470 for proxy in self._parser.values():
1471 getter = functools.partial(proxy.get, _impl=func)
1472 setattr(proxy, k, getter)
1474 def __delitem__(self, key):
1475 try:
1476 k = 'get' + (key or None)
1477 except TypeError:
1478 raise KeyError(key)
1479 del self._data[key]
1480 for inst in itertools.chain((self._parser,), self._parser.values()):
1481 try:
1482 delattr(inst, k)
1483 except AttributeError:
1484 # don't raise since the entry was present in _data, silently
1485 # clean up
1486 continue
1488 def __iter__(self):
1489 return iter(self._data)
1491 def __len__(self):
1492 return len(self._data)