Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# ext/mutable.py
2# Copyright (C) 2005-2025 the SQLAlchemy authors and contributors
3# <see AUTHORS file>
4#
5# This module is part of SQLAlchemy and is released under
6# the MIT License: https://www.opensource.org/licenses/mit-license.php
8r"""Provide support for tracking of in-place changes to scalar values,
9which are propagated into ORM change events on owning parent objects.
11.. _mutable_scalars:
13Establishing Mutability on Scalar Column Values
14===============================================
16A typical example of a "mutable" structure is a Python dictionary.
17Following the example introduced in :ref:`types_toplevel`, we
18begin with a custom type that marshals Python dictionaries into
19JSON strings before being persisted::
21 from sqlalchemy.types import TypeDecorator, VARCHAR
22 import json
25 class JSONEncodedDict(TypeDecorator):
26 "Represents an immutable structure as a json-encoded string."
28 impl = VARCHAR
30 def process_bind_param(self, value, dialect):
31 if value is not None:
32 value = json.dumps(value)
33 return value
35 def process_result_value(self, value, dialect):
36 if value is not None:
37 value = json.loads(value)
38 return value
40The usage of ``json`` is only for the purposes of example. The
41:mod:`sqlalchemy.ext.mutable` extension can be used
42with any type whose target Python type may be mutable, including
43:class:`.PickleType`, :class:`_postgresql.ARRAY`, etc.
45When using the :mod:`sqlalchemy.ext.mutable` extension, the value itself
46tracks all parents which reference it. Below, we illustrate a simple
47version of the :class:`.MutableDict` dictionary object, which applies
48the :class:`.Mutable` mixin to a plain Python dictionary::
50 from sqlalchemy.ext.mutable import Mutable
53 class MutableDict(Mutable, dict):
54 @classmethod
55 def coerce(cls, key, value):
56 "Convert plain dictionaries to MutableDict."
58 if not isinstance(value, MutableDict):
59 if isinstance(value, dict):
60 return MutableDict(value)
62 # this call will raise ValueError
63 return Mutable.coerce(key, value)
64 else:
65 return value
67 def __setitem__(self, key, value):
68 "Detect dictionary set events and emit change events."
70 dict.__setitem__(self, key, value)
71 self.changed()
73 def __delitem__(self, key):
74 "Detect dictionary del events and emit change events."
76 dict.__delitem__(self, key)
77 self.changed()
79The above dictionary class takes the approach of subclassing the Python
80built-in ``dict`` to produce a dict
81subclass which routes all mutation events through ``__setitem__``. There are
82variants on this approach, such as subclassing ``UserDict.UserDict`` or
83``collections.MutableMapping``; the part that's important to this example is
84that the :meth:`.Mutable.changed` method is called whenever an in-place
85change to the datastructure takes place.
87We also redefine the :meth:`.Mutable.coerce` method which will be used to
88convert any values that are not instances of ``MutableDict``, such
89as the plain dictionaries returned by the ``json`` module, into the
90appropriate type. Defining this method is optional; we could just as well
91created our ``JSONEncodedDict`` such that it always returns an instance
92of ``MutableDict``, and additionally ensured that all calling code
93uses ``MutableDict`` explicitly. When :meth:`.Mutable.coerce` is not
94overridden, any values applied to a parent object which are not instances
95of the mutable type will raise a ``ValueError``.
97Our new ``MutableDict`` type offers a class method
98:meth:`~.Mutable.as_mutable` which we can use within column metadata
99to associate with types. This method grabs the given type object or
100class and associates a listener that will detect all future mappings
101of this type, applying event listening instrumentation to the mapped
102attribute. Such as, with classical table metadata::
104 from sqlalchemy import Table, Column, Integer
106 my_data = Table(
107 "my_data",
108 metadata,
109 Column("id", Integer, primary_key=True),
110 Column("data", MutableDict.as_mutable(JSONEncodedDict)),
111 )
113Above, :meth:`~.Mutable.as_mutable` returns an instance of ``JSONEncodedDict``
114(if the type object was not an instance already), which will intercept any
115attributes which are mapped against this type. Below we establish a simple
116mapping against the ``my_data`` table::
118 from sqlalchemy.orm import DeclarativeBase
119 from sqlalchemy.orm import Mapped
120 from sqlalchemy.orm import mapped_column
123 class Base(DeclarativeBase):
124 pass
127 class MyDataClass(Base):
128 __tablename__ = "my_data"
129 id: Mapped[int] = mapped_column(primary_key=True)
130 data: Mapped[dict[str, str]] = mapped_column(
131 MutableDict.as_mutable(JSONEncodedDict)
132 )
134The ``MyDataClass.data`` member will now be notified of in place changes
135to its value.
137Any in-place changes to the ``MyDataClass.data`` member
138will flag the attribute as "dirty" on the parent object::
140 >>> from sqlalchemy.orm import Session
142 >>> sess = Session(some_engine)
143 >>> m1 = MyDataClass(data={"value1": "foo"})
144 >>> sess.add(m1)
145 >>> sess.commit()
147 >>> m1.data["value1"] = "bar"
148 >>> assert m1 in sess.dirty
149 True
151The ``MutableDict`` can be associated with all future instances
152of ``JSONEncodedDict`` in one step, using
153:meth:`~.Mutable.associate_with`. This is similar to
154:meth:`~.Mutable.as_mutable` except it will intercept all occurrences
155of ``MutableDict`` in all mappings unconditionally, without
156the need to declare it individually::
158 from sqlalchemy.orm import DeclarativeBase
159 from sqlalchemy.orm import Mapped
160 from sqlalchemy.orm import mapped_column
162 MutableDict.associate_with(JSONEncodedDict)
165 class Base(DeclarativeBase):
166 pass
169 class MyDataClass(Base):
170 __tablename__ = "my_data"
171 id: Mapped[int] = mapped_column(primary_key=True)
172 data: Mapped[dict[str, str]] = mapped_column(JSONEncodedDict)
174Supporting Pickling
175--------------------
177The key to the :mod:`sqlalchemy.ext.mutable` extension relies upon the
178placement of a ``weakref.WeakKeyDictionary`` upon the value object, which
179stores a mapping of parent mapped objects keyed to the attribute name under
180which they are associated with this value. ``WeakKeyDictionary`` objects are
181not picklable, due to the fact that they contain weakrefs and function
182callbacks. In our case, this is a good thing, since if this dictionary were
183picklable, it could lead to an excessively large pickle size for our value
184objects that are pickled by themselves outside of the context of the parent.
185The developer responsibility here is only to provide a ``__getstate__`` method
186that excludes the :meth:`~MutableBase._parents` collection from the pickle
187stream::
189 class MyMutableType(Mutable):
190 def __getstate__(self):
191 d = self.__dict__.copy()
192 d.pop("_parents", None)
193 return d
195With our dictionary example, we need to return the contents of the dict itself
196(and also restore them on __setstate__)::
198 class MutableDict(Mutable, dict):
199 # ....
201 def __getstate__(self):
202 return dict(self)
204 def __setstate__(self, state):
205 self.update(state)
207In the case that our mutable value object is pickled as it is attached to one
208or more parent objects that are also part of the pickle, the :class:`.Mutable`
209mixin will re-establish the :attr:`.Mutable._parents` collection on each value
210object as the owning parents themselves are unpickled.
212Receiving Events
213----------------
215The :meth:`.AttributeEvents.modified` event handler may be used to receive
216an event when a mutable scalar emits a change event. This event handler
217is called when the :func:`.attributes.flag_modified` function is called
218from within the mutable extension::
220 from sqlalchemy.orm import DeclarativeBase
221 from sqlalchemy.orm import Mapped
222 from sqlalchemy.orm import mapped_column
223 from sqlalchemy import event
226 class Base(DeclarativeBase):
227 pass
230 class MyDataClass(Base):
231 __tablename__ = "my_data"
232 id: Mapped[int] = mapped_column(primary_key=True)
233 data: Mapped[dict[str, str]] = mapped_column(
234 MutableDict.as_mutable(JSONEncodedDict)
235 )
238 @event.listens_for(MyDataClass.data, "modified")
239 def modified_json(instance, initiator):
240 print("json value modified:", instance.data)
242.. _mutable_composites:
244Establishing Mutability on Composites
245=====================================
247Composites are a special ORM feature which allow a single scalar attribute to
248be assigned an object value which represents information "composed" from one
249or more columns from the underlying mapped table. The usual example is that of
250a geometric "point", and is introduced in :ref:`mapper_composite`.
252As is the case with :class:`.Mutable`, the user-defined composite class
253subclasses :class:`.MutableComposite` as a mixin, and detects and delivers
254change events to its parents via the :meth:`.MutableComposite.changed` method.
255In the case of a composite class, the detection is usually via the usage of the
256special Python method ``__setattr__()``. In the example below, we expand upon the ``Point``
257class introduced in :ref:`mapper_composite` to include
258:class:`.MutableComposite` in its bases and to route attribute set events via
259``__setattr__`` to the :meth:`.MutableComposite.changed` method::
261 import dataclasses
262 from sqlalchemy.ext.mutable import MutableComposite
265 @dataclasses.dataclass
266 class Point(MutableComposite):
267 x: int
268 y: int
270 def __setattr__(self, key, value):
271 "Intercept set events"
273 # set the attribute
274 object.__setattr__(self, key, value)
276 # alert all parents to the change
277 self.changed()
279The :class:`.MutableComposite` class makes use of class mapping events to
280automatically establish listeners for any usage of :func:`_orm.composite` that
281specifies our ``Point`` type. Below, when ``Point`` is mapped to the ``Vertex``
282class, listeners are established which will route change events from ``Point``
283objects to each of the ``Vertex.start`` and ``Vertex.end`` attributes::
285 from sqlalchemy.orm import DeclarativeBase, Mapped
286 from sqlalchemy.orm import composite, mapped_column
289 class Base(DeclarativeBase):
290 pass
293 class Vertex(Base):
294 __tablename__ = "vertices"
296 id: Mapped[int] = mapped_column(primary_key=True)
298 start: Mapped[Point] = composite(
299 mapped_column("x1"), mapped_column("y1")
300 )
301 end: Mapped[Point] = composite(
302 mapped_column("x2"), mapped_column("y2")
303 )
305 def __repr__(self):
306 return f"Vertex(start={self.start}, end={self.end})"
308Any in-place changes to the ``Vertex.start`` or ``Vertex.end`` members
309will flag the attribute as "dirty" on the parent object:
311.. sourcecode:: python+sql
313 >>> from sqlalchemy.orm import Session
314 >>> sess = Session(engine)
315 >>> v1 = Vertex(start=Point(3, 4), end=Point(12, 15))
316 >>> sess.add(v1)
317 {sql}>>> sess.flush()
318 BEGIN (implicit)
319 INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?)
320 [...] (3, 4, 12, 15)
322 {stop}>>> v1.end.x = 8
323 >>> assert v1 in sess.dirty
324 True
325 {sql}>>> sess.commit()
326 UPDATE vertices SET x2=? WHERE vertices.id = ?
327 [...] (8, 1)
328 COMMIT
330Coercing Mutable Composites
331---------------------------
333The :meth:`.MutableBase.coerce` method is also supported on composite types.
334In the case of :class:`.MutableComposite`, the :meth:`.MutableBase.coerce`
335method is only called for attribute set operations, not load operations.
336Overriding the :meth:`.MutableBase.coerce` method is essentially equivalent
337to using a :func:`.validates` validation routine for all attributes which
338make use of the custom composite type::
340 @dataclasses.dataclass
341 class Point(MutableComposite):
342 # other Point methods
343 # ...
345 def coerce(cls, key, value):
346 if isinstance(value, tuple):
347 value = Point(*value)
348 elif not isinstance(value, Point):
349 raise ValueError("tuple or Point expected")
350 return value
352Supporting Pickling
353--------------------
355As is the case with :class:`.Mutable`, the :class:`.MutableComposite` helper
356class uses a ``weakref.WeakKeyDictionary`` available via the
357:meth:`MutableBase._parents` attribute which isn't picklable. If we need to
358pickle instances of ``Point`` or its owning class ``Vertex``, we at least need
359to define a ``__getstate__`` that doesn't include the ``_parents`` dictionary.
360Below we define both a ``__getstate__`` and a ``__setstate__`` that package up
361the minimal form of our ``Point`` class::
363 @dataclasses.dataclass
364 class Point(MutableComposite):
365 # ...
367 def __getstate__(self):
368 return self.x, self.y
370 def __setstate__(self, state):
371 self.x, self.y = state
373As with :class:`.Mutable`, the :class:`.MutableComposite` augments the
374pickling process of the parent's object-relational state so that the
375:meth:`MutableBase._parents` collection is restored to all ``Point`` objects.
377""" # noqa: E501
379from __future__ import annotations
381from collections import defaultdict
382from typing import AbstractSet
383from typing import Any
384from typing import Dict
385from typing import Iterable
386from typing import List
387from typing import Optional
388from typing import overload
389from typing import Set
390from typing import SupportsIndex
391from typing import Tuple
392from typing import TYPE_CHECKING
393from typing import TypeVar
394from typing import Union
395import weakref
396from weakref import WeakKeyDictionary
398from .. import event
399from .. import inspect
400from .. import types
401from .. import util
402from ..orm import Mapper
403from ..orm._typing import _ExternalEntityType
404from ..orm._typing import _O
405from ..orm._typing import _T
406from ..orm.attributes import AttributeEventToken
407from ..orm.attributes import flag_modified
408from ..orm.attributes import InstrumentedAttribute
409from ..orm.attributes import QueryableAttribute
410from ..orm.context import QueryContext
411from ..orm.decl_api import DeclarativeAttributeIntercept
412from ..orm.state import InstanceState
413from ..orm.unitofwork import UOWTransaction
414from ..sql._typing import _TypeEngineArgument
415from ..sql.base import SchemaEventTarget
416from ..sql.schema import Column
417from ..sql.type_api import TypeEngine
418from ..util import memoized_property
419from ..util.typing import TypeGuard
421_KT = TypeVar("_KT") # Key type.
422_VT = TypeVar("_VT") # Value type.
425class MutableBase:
426 """Common base class to :class:`.Mutable`
427 and :class:`.MutableComposite`.
429 """
431 @memoized_property
432 def _parents(self) -> WeakKeyDictionary[Any, Any]:
433 """Dictionary of parent object's :class:`.InstanceState`->attribute
434 name on the parent.
436 This attribute is a so-called "memoized" property. It initializes
437 itself with a new ``weakref.WeakKeyDictionary`` the first time
438 it is accessed, returning the same object upon subsequent access.
440 .. versionchanged:: 1.4 the :class:`.InstanceState` is now used
441 as the key in the weak dictionary rather than the instance
442 itself.
444 """
446 return weakref.WeakKeyDictionary()
448 @classmethod
449 def coerce(cls, key: str, value: Any) -> Optional[Any]:
450 """Given a value, coerce it into the target type.
452 Can be overridden by custom subclasses to coerce incoming
453 data into a particular type.
455 By default, raises ``ValueError``.
457 This method is called in different scenarios depending on if
458 the parent class is of type :class:`.Mutable` or of type
459 :class:`.MutableComposite`. In the case of the former, it is called
460 for both attribute-set operations as well as during ORM loading
461 operations. For the latter, it is only called during attribute-set
462 operations; the mechanics of the :func:`.composite` construct
463 handle coercion during load operations.
466 :param key: string name of the ORM-mapped attribute being set.
467 :param value: the incoming value.
468 :return: the method should return the coerced value, or raise
469 ``ValueError`` if the coercion cannot be completed.
471 """
472 if value is None:
473 return None
474 msg = "Attribute '%s' does not accept objects of type %s"
475 raise ValueError(msg % (key, type(value)))
477 @classmethod
478 def _get_listen_keys(cls, attribute: QueryableAttribute[Any]) -> Set[str]:
479 """Given a descriptor attribute, return a ``set()`` of the attribute
480 keys which indicate a change in the state of this attribute.
482 This is normally just ``set([attribute.key])``, but can be overridden
483 to provide for additional keys. E.g. a :class:`.MutableComposite`
484 augments this set with the attribute keys associated with the columns
485 that comprise the composite value.
487 This collection is consulted in the case of intercepting the
488 :meth:`.InstanceEvents.refresh` and
489 :meth:`.InstanceEvents.refresh_flush` events, which pass along a list
490 of attribute names that have been refreshed; the list is compared
491 against this set to determine if action needs to be taken.
493 """
494 return {attribute.key}
496 @classmethod
497 def _listen_on_attribute(
498 cls,
499 attribute: QueryableAttribute[Any],
500 coerce: bool,
501 parent_cls: _ExternalEntityType[Any],
502 ) -> None:
503 """Establish this type as a mutation listener for the given
504 mapped descriptor.
506 """
507 key = attribute.key
508 if parent_cls is not attribute.class_:
509 return
511 # rely on "propagate" here
512 parent_cls = attribute.class_
514 listen_keys = cls._get_listen_keys(attribute)
516 def load(state: InstanceState[_O], *args: Any) -> None:
517 """Listen for objects loaded or refreshed.
519 Wrap the target data member's value with
520 ``Mutable``.
522 """
523 val = state.dict.get(key, None)
524 if val is not None:
525 if coerce:
526 val = cls.coerce(key, val)
527 assert val is not None
528 state.dict[key] = val
529 val._parents[state] = key
531 def load_attrs(
532 state: InstanceState[_O],
533 ctx: Union[object, QueryContext, UOWTransaction],
534 attrs: Iterable[Any],
535 ) -> None:
536 if not attrs or listen_keys.intersection(attrs):
537 load(state)
539 def set_(
540 target: InstanceState[_O],
541 value: MutableBase | None,
542 oldvalue: MutableBase | None,
543 initiator: AttributeEventToken,
544 ) -> MutableBase | None:
545 """Listen for set/replace events on the target
546 data member.
548 Establish a weak reference to the parent object
549 on the incoming value, remove it for the one
550 outgoing.
552 """
553 if value is oldvalue:
554 return value
556 if not isinstance(value, cls):
557 value = cls.coerce(key, value)
558 if value is not None:
559 value._parents[target] = key
560 if isinstance(oldvalue, cls):
561 oldvalue._parents.pop(inspect(target), None)
562 return value
564 def pickle(
565 state: InstanceState[_O], state_dict: Dict[str, Any]
566 ) -> None:
567 val = state.dict.get(key, None)
568 if val is not None:
569 if "ext.mutable.values" not in state_dict:
570 state_dict["ext.mutable.values"] = defaultdict(list)
571 state_dict["ext.mutable.values"][key].append(val)
573 def unpickle(
574 state: InstanceState[_O], state_dict: Dict[str, Any]
575 ) -> None:
576 if "ext.mutable.values" in state_dict:
577 collection = state_dict["ext.mutable.values"]
578 if isinstance(collection, list):
579 # legacy format
580 for val in collection:
581 val._parents[state] = key
582 else:
583 for val in state_dict["ext.mutable.values"][key]:
584 val._parents[state] = key
586 event.listen(
587 parent_cls,
588 "_sa_event_merge_wo_load",
589 load,
590 raw=True,
591 propagate=True,
592 )
594 event.listen(parent_cls, "load", load, raw=True, propagate=True)
595 event.listen(
596 parent_cls, "refresh", load_attrs, raw=True, propagate=True
597 )
598 event.listen(
599 parent_cls, "refresh_flush", load_attrs, raw=True, propagate=True
600 )
601 event.listen(
602 attribute, "set", set_, raw=True, retval=True, propagate=True
603 )
604 event.listen(parent_cls, "pickle", pickle, raw=True, propagate=True)
605 event.listen(
606 parent_cls, "unpickle", unpickle, raw=True, propagate=True
607 )
610class Mutable(MutableBase):
611 """Mixin that defines transparent propagation of change
612 events to a parent object.
614 See the example in :ref:`mutable_scalars` for usage information.
616 """
618 def changed(self) -> None:
619 """Subclasses should call this method whenever change events occur."""
621 for parent, key in self._parents.items():
622 flag_modified(parent.obj(), key)
624 @classmethod
625 def associate_with_attribute(
626 cls, attribute: InstrumentedAttribute[_O]
627 ) -> None:
628 """Establish this type as a mutation listener for the given
629 mapped descriptor.
631 """
632 cls._listen_on_attribute(attribute, True, attribute.class_)
634 @classmethod
635 def associate_with(cls, sqltype: type) -> None:
636 """Associate this wrapper with all future mapped columns
637 of the given type.
639 This is a convenience method that calls
640 ``associate_with_attribute`` automatically.
642 .. warning::
644 The listeners established by this method are *global*
645 to all mappers, and are *not* garbage collected. Only use
646 :meth:`.associate_with` for types that are permanent to an
647 application, not with ad-hoc types else this will cause unbounded
648 growth in memory usage.
650 """
652 def listen_for_type(mapper: Mapper[_O], class_: type) -> None:
653 for prop in mapper.column_attrs:
654 if isinstance(prop.columns[0].type, sqltype):
655 cls.associate_with_attribute(getattr(class_, prop.key))
657 event.listen(Mapper, "mapper_configured", listen_for_type)
659 @classmethod
660 def as_mutable(cls, sqltype: _TypeEngineArgument[_T]) -> TypeEngine[_T]:
661 """Associate a SQL type with this mutable Python type.
663 This establishes listeners that will detect ORM mappings against
664 the given type, adding mutation event trackers to those mappings.
666 The type is returned, unconditionally as an instance, so that
667 :meth:`.as_mutable` can be used inline::
669 Table(
670 "mytable",
671 metadata,
672 Column("id", Integer, primary_key=True),
673 Column("data", MyMutableType.as_mutable(PickleType)),
674 )
676 Note that the returned type is always an instance, even if a class
677 is given, and that only columns which are declared specifically with
678 that type instance receive additional instrumentation.
680 To associate a particular mutable type with all occurrences of a
681 particular type, use the :meth:`.Mutable.associate_with` classmethod
682 of the particular :class:`.Mutable` subclass to establish a global
683 association.
685 .. warning::
687 The listeners established by this method are *global*
688 to all mappers, and are *not* garbage collected. Only use
689 :meth:`.as_mutable` for types that are permanent to an application,
690 not with ad-hoc types else this will cause unbounded growth
691 in memory usage.
693 """
694 sqltype = types.to_instance(sqltype)
696 # a SchemaType will be copied when the Column is copied,
697 # and we'll lose our ability to link that type back to the original.
698 # so track our original type w/ columns
699 if isinstance(sqltype, SchemaEventTarget):
701 @event.listens_for(sqltype, "before_parent_attach")
702 def _add_column_memo(
703 sqltyp: TypeEngine[Any],
704 parent: Column[_T],
705 ) -> None:
706 parent.info["_ext_mutable_orig_type"] = sqltyp
708 schema_event_check = True
709 else:
710 schema_event_check = False
712 def listen_for_type(
713 mapper: Mapper[_T],
714 class_: Union[DeclarativeAttributeIntercept, type],
715 ) -> None:
716 _APPLIED_KEY = "_ext_mutable_listener_applied"
718 for prop in mapper.column_attrs:
719 if (
720 # all Mutable types refer to a Column that's mapped,
721 # since this is the only kind of Core target the ORM can
722 # "mutate"
723 isinstance(prop.expression, Column)
724 and (
725 (
726 schema_event_check
727 and prop.expression.info.get(
728 "_ext_mutable_orig_type"
729 )
730 is sqltype
731 )
732 or prop.expression.type is sqltype
733 )
734 ):
735 if not prop.expression.info.get(_APPLIED_KEY, False):
736 prop.expression.info[_APPLIED_KEY] = True
737 cls.associate_with_attribute(getattr(class_, prop.key))
739 event.listen(Mapper, "mapper_configured", listen_for_type)
741 return sqltype
744class MutableComposite(MutableBase):
745 """Mixin that defines transparent propagation of change
746 events on a SQLAlchemy "composite" object to its
747 owning parent or parents.
749 See the example in :ref:`mutable_composites` for usage information.
751 """
753 @classmethod
754 def _get_listen_keys(cls, attribute: QueryableAttribute[_O]) -> Set[str]:
755 return {attribute.key}.union(attribute.property._attribute_keys)
757 def changed(self) -> None:
758 """Subclasses should call this method whenever change events occur."""
760 for parent, key in self._parents.items():
761 prop = parent.mapper.get_property(key)
762 for value, attr_name in zip(
763 prop._composite_values_from_instance(self),
764 prop._attribute_keys,
765 ):
766 setattr(parent.obj(), attr_name, value)
769def _setup_composite_listener() -> None:
770 def _listen_for_type(mapper: Mapper[_T], class_: type) -> None:
771 for prop in mapper.iterate_properties:
772 if (
773 hasattr(prop, "composite_class")
774 and isinstance(prop.composite_class, type)
775 and issubclass(prop.composite_class, MutableComposite)
776 ):
777 prop.composite_class._listen_on_attribute(
778 getattr(class_, prop.key), False, class_
779 )
781 if not event.contains(Mapper, "mapper_configured", _listen_for_type):
782 event.listen(Mapper, "mapper_configured", _listen_for_type)
785_setup_composite_listener()
788class MutableDict(Mutable, Dict[_KT, _VT]):
789 """A dictionary type that implements :class:`.Mutable`.
791 The :class:`.MutableDict` object implements a dictionary that will
792 emit change events to the underlying mapping when the contents of
793 the dictionary are altered, including when values are added or removed.
795 Note that :class:`.MutableDict` does **not** apply mutable tracking to the
796 *values themselves* inside the dictionary. Therefore it is not a sufficient
797 solution for the use case of tracking deep changes to a *recursive*
798 dictionary structure, such as a JSON structure. To support this use case,
799 build a subclass of :class:`.MutableDict` that provides appropriate
800 coercion to the values placed in the dictionary so that they too are
801 "mutable", and emit events up to their parent structure.
803 .. seealso::
805 :class:`.MutableList`
807 :class:`.MutableSet`
809 """
811 def __setitem__(self, key: _KT, value: _VT) -> None:
812 """Detect dictionary set events and emit change events."""
813 super().__setitem__(key, value)
814 self.changed()
816 if TYPE_CHECKING:
817 # from https://github.com/python/mypy/issues/14858
819 @overload
820 def setdefault(
821 self: MutableDict[_KT, Optional[_T]], key: _KT, value: None = None
822 ) -> Optional[_T]: ...
824 @overload
825 def setdefault(self, key: _KT, value: _VT) -> _VT: ...
827 def setdefault(self, key: _KT, value: object = None) -> object: ...
829 else:
831 def setdefault(self, *arg): # noqa: F811
832 result = super().setdefault(*arg)
833 self.changed()
834 return result
836 def __delitem__(self, key: _KT) -> None:
837 """Detect dictionary del events and emit change events."""
838 super().__delitem__(key)
839 self.changed()
841 def update(self, *a: Any, **kw: _VT) -> None:
842 super().update(*a, **kw)
843 self.changed()
845 if TYPE_CHECKING:
847 @overload
848 def pop(self, __key: _KT, /) -> _VT: ...
850 @overload
851 def pop(self, __key: _KT, default: _VT | _T, /) -> _VT | _T: ...
853 def pop(
854 self, __key: _KT, __default: _VT | _T | None = None, /
855 ) -> _VT | _T: ...
857 else:
859 def pop(self, *arg): # noqa: F811
860 result = super().pop(*arg)
861 self.changed()
862 return result
864 def popitem(self) -> Tuple[_KT, _VT]:
865 result = super().popitem()
866 self.changed()
867 return result
869 def clear(self) -> None:
870 super().clear()
871 self.changed()
873 @classmethod
874 def coerce(cls, key: str, value: Any) -> MutableDict[_KT, _VT] | None:
875 """Convert plain dictionary to instance of this class."""
876 if not isinstance(value, cls):
877 if isinstance(value, dict):
878 return cls(value)
879 return Mutable.coerce(key, value)
880 else:
881 return value
883 def __getstate__(self) -> Dict[_KT, _VT]:
884 return dict(self)
886 def __setstate__(
887 self, state: Union[Dict[str, int], Dict[str, str]]
888 ) -> None:
889 self.update(state)
892class MutableList(Mutable, List[_T]):
893 """A list type that implements :class:`.Mutable`.
895 The :class:`.MutableList` object implements a list that will
896 emit change events to the underlying mapping when the contents of
897 the list are altered, including when values are added or removed.
899 Note that :class:`.MutableList` does **not** apply mutable tracking to the
900 *values themselves* inside the list. Therefore it is not a sufficient
901 solution for the use case of tracking deep changes to a *recursive*
902 mutable structure, such as a JSON structure. To support this use case,
903 build a subclass of :class:`.MutableList` that provides appropriate
904 coercion to the values placed in the dictionary so that they too are
905 "mutable", and emit events up to their parent structure.
907 .. seealso::
909 :class:`.MutableDict`
911 :class:`.MutableSet`
913 """
915 def __reduce_ex__(
916 self, proto: SupportsIndex
917 ) -> Tuple[type, Tuple[List[int]]]:
918 return (self.__class__, (list(self),))
920 # needed for backwards compatibility with
921 # older pickles
922 def __setstate__(self, state: Iterable[_T]) -> None:
923 self[:] = state
925 def is_scalar(self, value: _T | Iterable[_T]) -> TypeGuard[_T]:
926 return not util.is_non_string_iterable(value)
928 def is_iterable(self, value: _T | Iterable[_T]) -> TypeGuard[Iterable[_T]]:
929 return util.is_non_string_iterable(value)
931 def __setitem__(
932 self, index: SupportsIndex | slice, value: _T | Iterable[_T]
933 ) -> None:
934 """Detect list set events and emit change events."""
935 if isinstance(index, SupportsIndex) and self.is_scalar(value):
936 super().__setitem__(index, value)
937 elif isinstance(index, slice) and self.is_iterable(value):
938 super().__setitem__(index, value)
939 self.changed()
941 def __delitem__(self, index: SupportsIndex | slice) -> None:
942 """Detect list del events and emit change events."""
943 super().__delitem__(index)
944 self.changed()
946 def pop(self, *arg: SupportsIndex) -> _T:
947 result = super().pop(*arg)
948 self.changed()
949 return result
951 def append(self, x: _T) -> None:
952 super().append(x)
953 self.changed()
955 def extend(self, x: Iterable[_T]) -> None:
956 super().extend(x)
957 self.changed()
959 def __iadd__(self, x: Iterable[_T]) -> MutableList[_T]: # type: ignore[override,misc] # noqa: E501
960 self.extend(x)
961 return self
963 def insert(self, i: SupportsIndex, x: _T) -> None:
964 super().insert(i, x)
965 self.changed()
967 def remove(self, i: _T) -> None:
968 super().remove(i)
969 self.changed()
971 def clear(self) -> None:
972 super().clear()
973 self.changed()
975 def sort(self, **kw: Any) -> None:
976 super().sort(**kw)
977 self.changed()
979 def reverse(self) -> None:
980 super().reverse()
981 self.changed()
983 @classmethod
984 def coerce(
985 cls, key: str, value: MutableList[_T] | _T
986 ) -> Optional[MutableList[_T]]:
987 """Convert plain list to instance of this class."""
988 if not isinstance(value, cls):
989 if isinstance(value, list):
990 return cls(value)
991 return Mutable.coerce(key, value)
992 else:
993 return value
996class MutableSet(Mutable, Set[_T]):
997 """A set type that implements :class:`.Mutable`.
999 The :class:`.MutableSet` object implements a set that will
1000 emit change events to the underlying mapping when the contents of
1001 the set are altered, including when values are added or removed.
1003 Note that :class:`.MutableSet` does **not** apply mutable tracking to the
1004 *values themselves* inside the set. Therefore it is not a sufficient
1005 solution for the use case of tracking deep changes to a *recursive*
1006 mutable structure. To support this use case,
1007 build a subclass of :class:`.MutableSet` that provides appropriate
1008 coercion to the values placed in the dictionary so that they too are
1009 "mutable", and emit events up to their parent structure.
1011 .. seealso::
1013 :class:`.MutableDict`
1015 :class:`.MutableList`
1018 """
1020 def update(self, *arg: Iterable[_T]) -> None:
1021 super().update(*arg)
1022 self.changed()
1024 def intersection_update(self, *arg: Iterable[Any]) -> None:
1025 super().intersection_update(*arg)
1026 self.changed()
1028 def difference_update(self, *arg: Iterable[Any]) -> None:
1029 super().difference_update(*arg)
1030 self.changed()
1032 def symmetric_difference_update(self, *arg: Iterable[_T]) -> None:
1033 super().symmetric_difference_update(*arg)
1034 self.changed()
1036 def __ior__(self, other: AbstractSet[_T]) -> MutableSet[_T]: # type: ignore[override,misc] # noqa: E501
1037 self.update(other)
1038 return self
1040 def __iand__(self, other: AbstractSet[object]) -> MutableSet[_T]:
1041 self.intersection_update(other)
1042 return self
1044 def __ixor__(self, other: AbstractSet[_T]) -> MutableSet[_T]: # type: ignore[override,misc] # noqa: E501
1045 self.symmetric_difference_update(other)
1046 return self
1048 def __isub__(self, other: AbstractSet[object]) -> MutableSet[_T]: # type: ignore[misc] # noqa: E501
1049 self.difference_update(other)
1050 return self
1052 def add(self, elem: _T) -> None:
1053 super().add(elem)
1054 self.changed()
1056 def remove(self, elem: _T) -> None:
1057 super().remove(elem)
1058 self.changed()
1060 def discard(self, elem: _T) -> None:
1061 super().discard(elem)
1062 self.changed()
1064 def pop(self, *arg: Any) -> _T:
1065 result = super().pop(*arg)
1066 self.changed()
1067 return result
1069 def clear(self) -> None:
1070 super().clear()
1071 self.changed()
1073 @classmethod
1074 def coerce(cls, index: str, value: Any) -> Optional[MutableSet[_T]]:
1075 """Convert plain set to instance of this class."""
1076 if not isinstance(value, cls):
1077 if isinstance(value, set):
1078 return cls(value)
1079 return Mutable.coerce(index, value)
1080 else:
1081 return value
1083 def __getstate__(self) -> Set[_T]:
1084 return set(self)
1086 def __setstate__(self, state: Iterable[_T]) -> None:
1087 self.update(state)
1089 def __reduce_ex__(
1090 self, proto: SupportsIndex
1091 ) -> Tuple[type, Tuple[List[int]]]:
1092 return (self.__class__, (list(self),))