Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1from __future__ import annotations
3_shared_docs: dict[str, str] = {}
5_shared_docs["aggregate"] = """
6Aggregate using one or more operations over the specified axis.
8Parameters
9----------
10func : function, str, list or dict
11 Function to use for aggregating the data. If a function, must either
12 work when passed a {klass} or when passed to {klass}.apply.
14 Accepted combinations are:
16 - function
17 - string function name
18 - list of functions and/or function names, e.g. ``[np.sum, 'mean']``
19 - dict of axis labels -> functions, function names or list of such.
20{axis}
21*args
22 Positional arguments to pass to `func`.
23**kwargs
24 Keyword arguments to pass to `func`.
26Returns
27-------
28scalar, Series or DataFrame
30 The return can be:
32 * scalar : when Series.agg is called with single function
33 * Series : when DataFrame.agg is called with a single function
34 * DataFrame : when DataFrame.agg is called with several functions
35{see_also}
36Notes
37-----
38The aggregation operations are always performed over an axis, either the
39index (default) or the column axis. This behavior is different from
40`numpy` aggregation functions (`mean`, `median`, `prod`, `sum`, `std`,
41`var`), where the default is to compute the aggregation of the flattened
42array, e.g., ``numpy.mean(arr_2d)`` as opposed to
43``numpy.mean(arr_2d, axis=0)``.
45`agg` is an alias for `aggregate`. Use the alias.
47Functions that mutate the passed object can produce unexpected
48behavior or errors and are not supported. See :ref:`gotchas.udf-mutation`
49for more details.
51A passed user-defined-function will be passed a Series for evaluation.
53If ``func`` defines an index relabeling, ``axis`` must be ``0`` or ``index``.
54{examples}"""
56_shared_docs["compare"] = """
57Compare to another {klass} and show the differences.
59Parameters
60----------
61other : {klass}
62 Object to compare with.
64align_axis : {{0 or 'index', 1 or 'columns'}}, default 1
65 Determine which axis to align the comparison on.
67 * 0, or 'index' : Resulting differences are stacked vertically
68 with rows drawn alternately from self and other.
69 * 1, or 'columns' : Resulting differences are aligned horizontally
70 with columns drawn alternately from self and other.
72keep_shape : bool, default False
73 If true, all rows and columns are kept.
74 Otherwise, only the ones with different values are kept.
76keep_equal : bool, default False
77 If true, the result keeps values that are equal.
78 Otherwise, equal values are shown as NaNs.
80result_names : tuple, default ('self', 'other')
81 Set the dataframes names in the comparison.
82"""
84_shared_docs["groupby"] = """
85Group %(klass)s using a mapper or by a Series of columns.
87A groupby operation involves some combination of splitting the
88object, applying a function, and combining the results. This can be
89used to group large amounts of data and compute operations on these
90groups.
92Parameters
93----------
94by : mapping, function, label, pd.Grouper or list of such
95 Used to determine the groups for the groupby.
96 If ``by`` is a function, it's called on each value of the object's
97 index. If a dict or Series is passed, the Series or dict VALUES
98 will be used to determine the groups (the Series' values are first
99 aligned; see ``.align()`` method). If a list or ndarray of length
100 equal to the selected axis is passed (see the `groupby user guide
101 <https://pandas.pydata.org/pandas-docs/stable/user_guide/groupby.html#splitting-an-object-into-groups>`_),
102 the values are used as-is to determine the groups. A label or list
103 of labels may be passed to group by the columns in ``self``.
104 Notice that a tuple is interpreted as a (single) key.
105level : int, level name, or sequence of such, default None
106 If the axis is a MultiIndex (hierarchical), group by a particular
107 level or levels. Do not specify both ``by`` and ``level``.
108as_index : bool, default True
109 Return object with group labels as the
110 index. Only relevant for DataFrame input. as_index=False is
111 effectively "SQL-style" grouped output. This argument has no effect
112 on filtrations (see the `filtrations in the user guide
113 <https://pandas.pydata.org/docs/dev/user_guide/groupby.html#filtration>`_),
114 such as ``head()``, ``tail()``, ``nth()`` and in transformations
115 (see the `transformations in the user guide
116 <https://pandas.pydata.org/docs/dev/user_guide/groupby.html#transformation>`_).
117sort : bool, default True
118 Sort group keys. Get better performance by turning this off.
119 Note this does not influence the order of observations within each
120 group. Groupby preserves the order of rows within each group. If False,
121 the groups will appear in the same order as they did in the original DataFrame.
122 This argument has no effect on filtrations (see the `filtrations in the user guide
123 <https://pandas.pydata.org/docs/dev/user_guide/groupby.html#filtration>`_),
124 such as ``head()``, ``tail()``, ``nth()`` and in transformations
125 (see the `transformations in the user guide
126 <https://pandas.pydata.org/docs/dev/user_guide/groupby.html#transformation>`_).
128 .. versionchanged:: 2.0.0
130 Specifying ``sort=False`` with an ordered categorical grouper will no
131 longer sort the values.
133group_keys : bool, default True
134 When calling apply and the ``by`` argument produces a like-indexed
135 (i.e. :ref:`a transform <groupby.transform>`) result, add group keys to
136 index to identify pieces. By default group keys are not included
137 when the result's index (and column) labels match the inputs, and
138 are included otherwise.
140 .. versionchanged:: 2.0.0
142 ``group_keys`` now defaults to ``True``.
144observed : bool, default True
145 This only applies if any of the groupers are Categoricals.
146 If True: only show observed values for categorical groupers.
147 If False: show all values for categorical groupers.
149 .. versionchanged:: 3.0.0
151 The default value is now ``True``.
153dropna : bool, default True
154 If True, and if group keys contain NA values, NA values together
155 with row/column will be dropped.
156 If False, NA values will also be treated as the key in groups.
158Returns
159-------
160pandas.api.typing.%(klass)sGroupBy
161 Returns a groupby object that contains information about the groups.
163See Also
164--------
165resample : Convenience method for frequency conversion and resampling
166 of time series.
168Notes
169-----
170See the `user guide
171<https://pandas.pydata.org/pandas-docs/stable/groupby.html>`__ for more
172detailed usage and examples, including splitting an object into groups,
173iterating through groups, selecting a group, aggregation, and more.
175The implementation of groupby is hash-based, meaning in particular that
176objects that compare as equal will be considered to be in the same group.
177An exception to this is that pandas has special handling of NA values:
178any NA values will be collapsed to a single group, regardless of how
179they compare. See the user guide linked above for more details.
180"""
182_shared_docs["transform"] = """
183Call ``func`` on self producing a {klass} with the same axis shape as self.
185Parameters
186----------
187func : function, str, list-like or dict-like
188 Function to use for transforming the data. If a function, must either
189 work when passed a {klass} or when passed to {klass}.apply. If func
190 is both list-like and dict-like, dict-like behavior takes precedence.
192 Accepted combinations are:
194 - function
195 - string function name
196 - list-like of functions and/or function names, e.g. ``[np.exp, 'sqrt']``
197 - dict-like of axis labels -> functions, function names or list-like of such.
198{axis}
199*args
200 Positional arguments to pass to `func`.
201**kwargs
202 Keyword arguments to pass to `func`.
204Returns
205-------
206{klass}
207 A {klass} that must have the same length as self.
209Raises
210------
211ValueError : If the returned {klass} has a different length than self.
213See Also
214--------
215{klass}.agg : Only perform aggregating type operations.
216{klass}.apply : Invoke function on a {klass}.
218Notes
219-----
220Functions that mutate the passed object can produce unexpected
221behavior or errors and are not supported. See :ref:`gotchas.udf-mutation`
222for more details.
224Examples
225--------
226>>> df = pd.DataFrame({{'A': range(3), 'B': range(1, 4)}})
227>>> df
228 A B
2290 0 1
2301 1 2
2312 2 3
232>>> df.transform(lambda x: x + 1)
233 A B
2340 1 2
2351 2 3
2362 3 4
238Even though the resulting {klass} must have the same length as the
239input {klass}, it is possible to provide several input functions:
241>>> s = pd.Series(range(3))
242>>> s
2430 0
2441 1
2452 2
246dtype: int64
247>>> s.transform([np.sqrt, np.exp])
248 sqrt exp
2490 0.000000 1.000000
2501 1.000000 2.718282
2512 1.414214 7.389056
253You can call transform on a GroupBy object:
255>>> df = pd.DataFrame({{
256... "Date": [
257... "2015-05-08", "2015-05-07", "2015-05-06", "2015-05-05",
258... "2015-05-08", "2015-05-07", "2015-05-06", "2015-05-05"],
259... "Data": [5, 8, 6, 1, 50, 100, 60, 120],
260... }})
261>>> df
262 Date Data
2630 2015-05-08 5
2641 2015-05-07 8
2652 2015-05-06 6
2663 2015-05-05 1
2674 2015-05-08 50
2685 2015-05-07 100
2696 2015-05-06 60
2707 2015-05-05 120
271>>> df.groupby('Date')['Data'].transform('sum')
2720 55
2731 108
2742 66
2753 121
2764 55
2775 108
2786 66
2797 121
280Name: Data, dtype: int64
282>>> df = pd.DataFrame({{
283... "c": [1, 1, 1, 2, 2, 2, 2],
284... "type": ["m", "n", "o", "m", "m", "n", "n"]
285... }})
286>>> df
287 c type
2880 1 m
2891 1 n
2902 1 o
2913 2 m
2924 2 m
2935 2 n
2946 2 n
295>>> df['size'] = df.groupby('c')['type'].transform(len)
296>>> df
297 c type size
2980 1 m 3
2991 1 n 3
3002 1 o 3
3013 2 m 4
3024 2 m 4
3035 2 n 4
3046 2 n 4
305"""
307_shared_docs["storage_options"] = """storage_options : dict, optional
308 Extra options that make sense for a particular storage connection, e.g.
309 host, port, username, password, etc. For HTTP(S) URLs the key-value pairs
310 are forwarded to ``urllib.request.Request`` as header options. For other
311 URLs (e.g. starting with "s3://", and "gcs://") the key-value pairs are
312 forwarded to ``fsspec.open``. Please see ``fsspec`` and ``urllib`` for more
313 details, and for more examples on storage options refer `here
314 <https://pandas.pydata.org/docs/user_guide/io.html?
315 highlight=storage_options#reading-writing-remote-files>`_."""
317_shared_docs["compression_options"] = """compression : str or dict, default 'infer'
318 For on-the-fly compression of the output data. If 'infer' and '%s' is
319 path-like, then detect compression from the following extensions: '.gz',
320 '.bz2', '.zip', '.xz', '.zst', '.tar', '.tar.gz', '.tar.xz' or '.tar.bz2'
321 (otherwise no compression).
322 Set to ``None`` for no compression.
323 Can also be a dict with key ``'method'`` set
324 to one of {``'zip'``, ``'gzip'``, ``'bz2'``, ``'zstd'``, ``'xz'``, ``'tar'``} and
325 other key-value pairs are forwarded to
326 ``zipfile.ZipFile``, ``gzip.GzipFile``,
327 ``bz2.BZ2File``, ``zstandard.ZstdCompressor``, ``lzma.LZMAFile`` or
328 ``tarfile.TarFile``, respectively.
329 As an example, the following could be passed for faster compression and to create
330 a reproducible gzip archive:
331 ``compression={'method': 'gzip', 'compresslevel': 1, 'mtime': 1}``.
333 .. versionadded:: 1.5.0
334 Added support for `.tar` files."""
336_shared_docs["decompression_options"] = """compression : str or dict, default 'infer'
337 For on-the-fly decompression of on-disk data. If 'infer' and '%s' is
338 path-like, then detect compression from the following extensions: '.gz',
339 '.bz2', '.zip', '.xz', '.zst', '.tar', '.tar.gz', '.tar.xz' or '.tar.bz2'
340 (otherwise no compression).
341 If using 'zip' or 'tar', the ZIP file must contain only one data file to be read in.
342 Set to ``None`` for no decompression.
343 Can also be a dict with key ``'method'`` set
344 to one of {``'zip'``, ``'gzip'``, ``'bz2'``, ``'zstd'``, ``'xz'``, ``'tar'``} and
345 other key-value pairs are forwarded to
346 ``zipfile.ZipFile``, ``gzip.GzipFile``,
347 ``bz2.BZ2File``, ``zstandard.ZstdDecompressor``, ``lzma.LZMAFile`` or
348 ``tarfile.TarFile``, respectively.
349 As an example, the following could be passed for Zstandard decompression using a
350 custom compression dictionary:
351 ``compression={'method': 'zstd', 'dict_data': my_compression_dict}``.
353 .. versionadded:: 1.5.0
354 Added support for `.tar` files."""
356_shared_docs["replace"] = """
357 Replace values given in `to_replace` with `value`.
359 Values of the {klass} are replaced with other values dynamically.
360 This differs from updating with ``.loc`` or ``.iloc``, which require
361 you to specify a location to update with some value.
363 Parameters
364 ----------
365 to_replace : str, regex, list, dict, Series, int, float, or None
366 How to find the values that will be replaced.
368 * numeric, str or regex:
370 - numeric: numeric values equal to `to_replace` will be
371 replaced with `value`
372 - str: string exactly matching `to_replace` will be replaced
373 with `value`
374 - regex: regexes matching `to_replace` will be replaced with
375 `value`
377 * list of str, regex, or numeric:
379 - First, if `to_replace` and `value` are both lists, they
380 **must** be the same length.
381 - Second, if ``regex=True`` then all of the strings in **both**
382 lists will be interpreted as regexes otherwise they will match
383 directly. This doesn't matter much for `value` since there
384 are only a few possible substitution regexes you can use.
385 - str, regex and numeric rules apply as above.
387 * dict:
389 - Dicts can be used to specify different replacement values
390 for different existing values. For example,
391 ``{{'a': 'b', 'y': 'z'}}`` replaces the value 'a' with 'b' and
392 'y' with 'z'. To use a dict in this way, the optional `value`
393 parameter should not be given.
394 - For a DataFrame a dict can specify that different values
395 should be replaced in different columns. For example,
396 ``{{'a': 1, 'b': 'z'}}`` looks for the value 1 in column 'a'
397 and the value 'z' in column 'b' and replaces these values
398 with whatever is specified in `value`. The `value` parameter
399 should not be ``None`` in this case. You can treat this as a
400 special case of passing two lists except that you are
401 specifying the column to search in.
402 - For a DataFrame nested dictionaries, e.g.,
403 ``{{'a': {{'b': np.nan}}}}``, are read as follows: look in column
404 'a' for the value 'b' and replace it with NaN. The optional `value`
405 parameter should not be specified to use a nested dict in this
406 way. You can nest regular expressions as well. Note that
407 column names (the top-level dictionary keys in a nested
408 dictionary) **cannot** be regular expressions.
410 * None:
412 - This means that the `regex` argument must be a string,
413 compiled regular expression, or list, dict, ndarray or
414 Series of such elements. If `value` is also ``None`` then
415 this **must** be a nested dictionary or Series.
417 See the examples section for examples of each of these.
418 value : scalar, dict, list, str, regex, default None
419 Value to replace any values matching `to_replace` with.
420 For a DataFrame a dict of values can be used to specify which
421 value to use for each column (columns not in the dict will not be
422 filled). Regular expressions, strings and lists or dicts of such
423 objects are also allowed.
424 {inplace}
425 regex : bool or same types as `to_replace`, default False
426 Whether to interpret `to_replace` and/or `value` as regular
427 expressions. Alternatively, this could be a regular expression or a
428 list, dict, or array of regular expressions in which case
429 `to_replace` must be ``None``.
431 Returns
432 -------
433 {klass}
434 Object after replacement.
436 Raises
437 ------
438 AssertionError
439 * If `regex` is not a ``bool`` and `to_replace` is not
440 ``None``.
442 TypeError
443 * If `to_replace` is not a scalar, array-like, ``dict``, or ``None``
444 * If `to_replace` is a ``dict`` and `value` is not a ``list``,
445 ``dict``, ``ndarray``, or ``Series``
446 * If `to_replace` is ``None`` and `regex` is not compilable
447 into a regular expression or is a list, dict, ndarray, or
448 Series.
449 * When replacing multiple ``bool`` or ``datetime64`` objects and
450 the arguments to `to_replace` does not match the type of the
451 value being replaced
453 ValueError
454 * If a ``list`` or an ``ndarray`` is passed to `to_replace` and
455 `value` but they are not the same length.
457 See Also
458 --------
459 Series.fillna : Fill NA values.
460 DataFrame.fillna : Fill NA values.
461 Series.where : Replace values based on boolean condition.
462 DataFrame.where : Replace values based on boolean condition.
463 DataFrame.map: Apply a function to a Dataframe elementwise.
464 Series.map: Map values of Series according to an input mapping or function.
465 Series.str.replace : Simple string replacement.
467 Notes
468 -----
469 * Regex substitution is performed under the hood with ``re.sub``. The
470 rules for substitution for ``re.sub`` are the same.
471 * Regular expressions will only substitute on strings, meaning you
472 cannot provide, for example, a regular expression matching floating
473 point numbers and expect the columns in your frame that have a
474 numeric dtype to be matched. However, if those floating point
475 numbers *are* strings, then you can do this.
476 * This method has *a lot* of options. You are encouraged to experiment
477 and play with this method to gain intuition about how it works.
478 * When dict is used as the `to_replace` value, it is like
479 key(s) in the dict are the to_replace part and
480 value(s) in the dict are the value parameter.
482 Examples
483 --------
485 **Scalar `to_replace` and `value`**
487 >>> s = pd.Series([1, 2, 3, 4, 5])
488 >>> s.replace(1, 5)
489 0 5
490 1 2
491 2 3
492 3 4
493 4 5
494 dtype: int64
496 >>> df = pd.DataFrame({{'A': [0, 1, 2, 3, 4],
497 ... 'B': [5, 6, 7, 8, 9],
498 ... 'C': ['a', 'b', 'c', 'd', 'e']}})
499 >>> df.replace(0, 5)
500 A B C
501 0 5 5 a
502 1 1 6 b
503 2 2 7 c
504 3 3 8 d
505 4 4 9 e
507 **List-like `to_replace`**
509 >>> df.replace([0, 1, 2, 3], 4)
510 A B C
511 0 4 5 a
512 1 4 6 b
513 2 4 7 c
514 3 4 8 d
515 4 4 9 e
517 >>> df.replace([0, 1, 2, 3], [4, 3, 2, 1])
518 A B C
519 0 4 5 a
520 1 3 6 b
521 2 2 7 c
522 3 1 8 d
523 4 4 9 e
525 **dict-like `to_replace`**
527 >>> df.replace({{0: 10, 1: 100}})
528 A B C
529 0 10 5 a
530 1 100 6 b
531 2 2 7 c
532 3 3 8 d
533 4 4 9 e
535 >>> df.replace({{'A': 0, 'B': 5}}, 100)
536 A B C
537 0 100 100 a
538 1 1 6 b
539 2 2 7 c
540 3 3 8 d
541 4 4 9 e
543 >>> df.replace({{'A': {{0: 100, 4: 400}}}})
544 A B C
545 0 100 5 a
546 1 1 6 b
547 2 2 7 c
548 3 3 8 d
549 4 400 9 e
551 **Regular expression `to_replace`**
553 >>> df = pd.DataFrame({{'A': ['bat', 'foo', 'bait'],
554 ... 'B': ['abc', 'bar', 'xyz']}})
555 >>> df.replace(to_replace=r'^ba.$', value='new', regex=True)
556 A B
557 0 new abc
558 1 foo new
559 2 bait xyz
561 >>> df.replace({{'A': r'^ba.$'}}, {{'A': 'new'}}, regex=True)
562 A B
563 0 new abc
564 1 foo bar
565 2 bait xyz
567 >>> df.replace(regex=r'^ba.$', value='new')
568 A B
569 0 new abc
570 1 foo new
571 2 bait xyz
573 >>> df.replace(regex={{r'^ba.$': 'new', 'foo': 'xyz'}})
574 A B
575 0 new abc
576 1 xyz new
577 2 bait xyz
579 >>> df.replace(regex=[r'^ba.$', 'foo'], value='new')
580 A B
581 0 new abc
582 1 new new
583 2 bait xyz
585 Compare the behavior of ``s.replace({{'a': None}})`` and
586 ``s.replace('a', None)`` to understand the peculiarities
587 of the `to_replace` parameter:
589 >>> s = pd.Series([10, 'a', 'a', 'b', 'a'])
591 When one uses a dict as the `to_replace` value, it is like the
592 value(s) in the dict are equal to the `value` parameter.
593 ``s.replace({{'a': None}})`` is equivalent to
594 ``s.replace(to_replace={{'a': None}}, value=None)``:
596 >>> s.replace({{'a': None}})
597 0 10
598 1 None
599 2 None
600 3 b
601 4 None
602 dtype: object
604 If ``None`` is explicitly passed for ``value``, it will be respected:
606 >>> s.replace('a', None)
607 0 10
608 1 None
609 2 None
610 3 b
611 4 None
612 dtype: object
614 When ``regex=True``, ``value`` is not ``None`` and `to_replace` is a string,
615 the replacement will be applied in all columns of the DataFrame.
617 >>> df = pd.DataFrame({{'A': [0, 1, 2, 3, 4],
618 ... 'B': ['a', 'b', 'c', 'd', 'e'],
619 ... 'C': ['f', 'g', 'h', 'i', 'j']}})
621 >>> df.replace(to_replace='^[a-g]', value='e', regex=True)
622 A B C
623 0 0 e e
624 1 1 e e
625 2 2 e h
626 3 3 e i
627 4 4 e j
629 If ``value`` is not ``None`` and `to_replace` is a dictionary, the dictionary
630 keys will be the DataFrame columns that the replacement will be applied.
632 >>> df.replace(to_replace={{'B': '^[a-c]', 'C': '^[h-j]'}}, value='e', regex=True)
633 A B C
634 0 0 e f
635 1 1 e g
636 2 2 e e
637 3 3 d e
638 4 4 e e
639"""