Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# ext/mutable.py
2# Copyright (C) 2005-2025 the SQLAlchemy authors and contributors
3# <see AUTHORS file>
4#
5# This module is part of SQLAlchemy and is released under
6# the MIT License: https://www.opensource.org/licenses/mit-license.php
8r"""Provide support for tracking of in-place changes to scalar values,
9which are propagated into ORM change events on owning parent objects.
11.. _mutable_scalars:
13Establishing Mutability on Scalar Column Values
14===============================================
16A typical example of a "mutable" structure is a Python dictionary.
17Following the example introduced in :ref:`types_toplevel`, we
18begin with a custom type that marshals Python dictionaries into
19JSON strings before being persisted::
21 from sqlalchemy.types import TypeDecorator, VARCHAR
22 import json
25 class JSONEncodedDict(TypeDecorator):
26 "Represents an immutable structure as a json-encoded string."
28 impl = VARCHAR
30 def process_bind_param(self, value, dialect):
31 if value is not None:
32 value = json.dumps(value)
33 return value
35 def process_result_value(self, value, dialect):
36 if value is not None:
37 value = json.loads(value)
38 return value
40The usage of ``json`` is only for the purposes of example. The
41:mod:`sqlalchemy.ext.mutable` extension can be used
42with any type whose target Python type may be mutable, including
43:class:`.PickleType`, :class:`_postgresql.ARRAY`, etc.
45When using the :mod:`sqlalchemy.ext.mutable` extension, the value itself
46tracks all parents which reference it. Below, we illustrate a simple
47version of the :class:`.MutableDict` dictionary object, which applies
48the :class:`.Mutable` mixin to a plain Python dictionary::
50 from sqlalchemy.ext.mutable import Mutable
53 class MutableDict(Mutable, dict):
54 @classmethod
55 def coerce(cls, key, value):
56 "Convert plain dictionaries to MutableDict."
58 if not isinstance(value, MutableDict):
59 if isinstance(value, dict):
60 return MutableDict(value)
62 # this call will raise ValueError
63 return Mutable.coerce(key, value)
64 else:
65 return value
67 def __setitem__(self, key, value):
68 "Detect dictionary set events and emit change events."
70 dict.__setitem__(self, key, value)
71 self.changed()
73 def __delitem__(self, key):
74 "Detect dictionary del events and emit change events."
76 dict.__delitem__(self, key)
77 self.changed()
79The above dictionary class takes the approach of subclassing the Python
80built-in ``dict`` to produce a dict
81subclass which routes all mutation events through ``__setitem__``. There are
82variants on this approach, such as subclassing ``UserDict.UserDict`` or
83``collections.MutableMapping``; the part that's important to this example is
84that the :meth:`.Mutable.changed` method is called whenever an in-place
85change to the datastructure takes place.
87We also redefine the :meth:`.Mutable.coerce` method which will be used to
88convert any values that are not instances of ``MutableDict``, such
89as the plain dictionaries returned by the ``json`` module, into the
90appropriate type. Defining this method is optional; we could just as well
91created our ``JSONEncodedDict`` such that it always returns an instance
92of ``MutableDict``, and additionally ensured that all calling code
93uses ``MutableDict`` explicitly. When :meth:`.Mutable.coerce` is not
94overridden, any values applied to a parent object which are not instances
95of the mutable type will raise a ``ValueError``.
97Our new ``MutableDict`` type offers a class method
98:meth:`~.Mutable.as_mutable` which we can use within column metadata
99to associate with types. This method grabs the given type object or
100class and associates a listener that will detect all future mappings
101of this type, applying event listening instrumentation to the mapped
102attribute. Such as, with classical table metadata::
104 from sqlalchemy import Table, Column, Integer
106 my_data = Table(
107 "my_data",
108 metadata,
109 Column("id", Integer, primary_key=True),
110 Column("data", MutableDict.as_mutable(JSONEncodedDict)),
111 )
113Above, :meth:`~.Mutable.as_mutable` returns an instance of ``JSONEncodedDict``
114(if the type object was not an instance already), which will intercept any
115attributes which are mapped against this type. Below we establish a simple
116mapping against the ``my_data`` table::
118 from sqlalchemy.orm import DeclarativeBase
119 from sqlalchemy.orm import Mapped
120 from sqlalchemy.orm import mapped_column
123 class Base(DeclarativeBase):
124 pass
127 class MyDataClass(Base):
128 __tablename__ = "my_data"
129 id: Mapped[int] = mapped_column(primary_key=True)
130 data: Mapped[dict[str, str]] = mapped_column(
131 MutableDict.as_mutable(JSONEncodedDict)
132 )
134The ``MyDataClass.data`` member will now be notified of in place changes
135to its value.
137Any in-place changes to the ``MyDataClass.data`` member
138will flag the attribute as "dirty" on the parent object::
140 >>> from sqlalchemy.orm import Session
142 >>> sess = Session(some_engine)
143 >>> m1 = MyDataClass(data={"value1": "foo"})
144 >>> sess.add(m1)
145 >>> sess.commit()
147 >>> m1.data["value1"] = "bar"
148 >>> assert m1 in sess.dirty
149 True
151The ``MutableDict`` can be associated with all future instances
152of ``JSONEncodedDict`` in one step, using
153:meth:`~.Mutable.associate_with`. This is similar to
154:meth:`~.Mutable.as_mutable` except it will intercept all occurrences
155of ``MutableDict`` in all mappings unconditionally, without
156the need to declare it individually::
158 from sqlalchemy.orm import DeclarativeBase
159 from sqlalchemy.orm import Mapped
160 from sqlalchemy.orm import mapped_column
162 MutableDict.associate_with(JSONEncodedDict)
165 class Base(DeclarativeBase):
166 pass
169 class MyDataClass(Base):
170 __tablename__ = "my_data"
171 id: Mapped[int] = mapped_column(primary_key=True)
172 data: Mapped[dict[str, str]] = mapped_column(JSONEncodedDict)
174Supporting Pickling
175--------------------
177The key to the :mod:`sqlalchemy.ext.mutable` extension relies upon the
178placement of a ``weakref.WeakKeyDictionary`` upon the value object, which
179stores a mapping of parent mapped objects keyed to the attribute name under
180which they are associated with this value. ``WeakKeyDictionary`` objects are
181not picklable, due to the fact that they contain weakrefs and function
182callbacks. In our case, this is a good thing, since if this dictionary were
183picklable, it could lead to an excessively large pickle size for our value
184objects that are pickled by themselves outside of the context of the parent.
185The developer responsibility here is only to provide a ``__getstate__`` method
186that excludes the :meth:`~MutableBase._parents` collection from the pickle
187stream::
189 class MyMutableType(Mutable):
190 def __getstate__(self):
191 d = self.__dict__.copy()
192 d.pop("_parents", None)
193 return d
195With our dictionary example, we need to return the contents of the dict itself
196(and also restore them on __setstate__)::
198 class MutableDict(Mutable, dict):
199 # ....
201 def __getstate__(self):
202 return dict(self)
204 def __setstate__(self, state):
205 self.update(state)
207In the case that our mutable value object is pickled as it is attached to one
208or more parent objects that are also part of the pickle, the :class:`.Mutable`
209mixin will re-establish the :attr:`.Mutable._parents` collection on each value
210object as the owning parents themselves are unpickled.
212Receiving Events
213----------------
215The :meth:`.AttributeEvents.modified` event handler may be used to receive
216an event when a mutable scalar emits a change event. This event handler
217is called when the :func:`.attributes.flag_modified` function is called
218from within the mutable extension::
220 from sqlalchemy.orm import DeclarativeBase
221 from sqlalchemy.orm import Mapped
222 from sqlalchemy.orm import mapped_column
223 from sqlalchemy import event
226 class Base(DeclarativeBase):
227 pass
230 class MyDataClass(Base):
231 __tablename__ = "my_data"
232 id: Mapped[int] = mapped_column(primary_key=True)
233 data: Mapped[dict[str, str]] = mapped_column(
234 MutableDict.as_mutable(JSONEncodedDict)
235 )
238 @event.listens_for(MyDataClass.data, "modified")
239 def modified_json(instance, initiator):
240 print("json value modified:", instance.data)
242.. _mutable_composites:
244Establishing Mutability on Composites
245=====================================
247Composites are a special ORM feature which allow a single scalar attribute to
248be assigned an object value which represents information "composed" from one
249or more columns from the underlying mapped table. The usual example is that of
250a geometric "point", and is introduced in :ref:`mapper_composite`.
252As is the case with :class:`.Mutable`, the user-defined composite class
253subclasses :class:`.MutableComposite` as a mixin, and detects and delivers
254change events to its parents via the :meth:`.MutableComposite.changed` method.
255In the case of a composite class, the detection is usually via the usage of the
256special Python method ``__setattr__()``. In the example below, we expand upon the ``Point``
257class introduced in :ref:`mapper_composite` to include
258:class:`.MutableComposite` in its bases and to route attribute set events via
259``__setattr__`` to the :meth:`.MutableComposite.changed` method::
261 import dataclasses
262 from sqlalchemy.ext.mutable import MutableComposite
265 @dataclasses.dataclass
266 class Point(MutableComposite):
267 x: int
268 y: int
270 def __setattr__(self, key, value):
271 "Intercept set events"
273 # set the attribute
274 object.__setattr__(self, key, value)
276 # alert all parents to the change
277 self.changed()
279The :class:`.MutableComposite` class makes use of class mapping events to
280automatically establish listeners for any usage of :func:`_orm.composite` that
281specifies our ``Point`` type. Below, when ``Point`` is mapped to the ``Vertex``
282class, listeners are established which will route change events from ``Point``
283objects to each of the ``Vertex.start`` and ``Vertex.end`` attributes::
285 from sqlalchemy.orm import DeclarativeBase, Mapped
286 from sqlalchemy.orm import composite, mapped_column
289 class Base(DeclarativeBase):
290 pass
293 class Vertex(Base):
294 __tablename__ = "vertices"
296 id: Mapped[int] = mapped_column(primary_key=True)
298 start: Mapped[Point] = composite(
299 mapped_column("x1"), mapped_column("y1")
300 )
301 end: Mapped[Point] = composite(
302 mapped_column("x2"), mapped_column("y2")
303 )
305 def __repr__(self):
306 return f"Vertex(start={self.start}, end={self.end})"
308Any in-place changes to the ``Vertex.start`` or ``Vertex.end`` members
309will flag the attribute as "dirty" on the parent object:
311.. sourcecode:: python+sql
313 >>> from sqlalchemy.orm import Session
314 >>> sess = Session(engine)
315 >>> v1 = Vertex(start=Point(3, 4), end=Point(12, 15))
316 >>> sess.add(v1)
317 {sql}>>> sess.flush()
318 BEGIN (implicit)
319 INSERT INTO vertices (x1, y1, x2, y2) VALUES (?, ?, ?, ?)
320 [...] (3, 4, 12, 15)
322 {stop}>>> v1.end.x = 8
323 >>> assert v1 in sess.dirty
324 True
325 {sql}>>> sess.commit()
326 UPDATE vertices SET x2=? WHERE vertices.id = ?
327 [...] (8, 1)
328 COMMIT
330Coercing Mutable Composites
331---------------------------
333The :meth:`.MutableBase.coerce` method is also supported on composite types.
334In the case of :class:`.MutableComposite`, the :meth:`.MutableBase.coerce`
335method is only called for attribute set operations, not load operations.
336Overriding the :meth:`.MutableBase.coerce` method is essentially equivalent
337to using a :func:`.validates` validation routine for all attributes which
338make use of the custom composite type::
340 @dataclasses.dataclass
341 class Point(MutableComposite):
342 # other Point methods
343 # ...
345 def coerce(cls, key, value):
346 if isinstance(value, tuple):
347 value = Point(*value)
348 elif not isinstance(value, Point):
349 raise ValueError("tuple or Point expected")
350 return value
352Supporting Pickling
353--------------------
355As is the case with :class:`.Mutable`, the :class:`.MutableComposite` helper
356class uses a ``weakref.WeakKeyDictionary`` available via the
357:meth:`MutableBase._parents` attribute which isn't picklable. If we need to
358pickle instances of ``Point`` or its owning class ``Vertex``, we at least need
359to define a ``__getstate__`` that doesn't include the ``_parents`` dictionary.
360Below we define both a ``__getstate__`` and a ``__setstate__`` that package up
361the minimal form of our ``Point`` class::
363 @dataclasses.dataclass
364 class Point(MutableComposite):
365 # ...
367 def __getstate__(self):
368 return self.x, self.y
370 def __setstate__(self, state):
371 self.x, self.y = state
373As with :class:`.Mutable`, the :class:`.MutableComposite` augments the
374pickling process of the parent's object-relational state so that the
375:meth:`MutableBase._parents` collection is restored to all ``Point`` objects.
377""" # noqa: E501
379from __future__ import annotations
381from collections import defaultdict
382from typing import AbstractSet
383from typing import Any
384from typing import Dict
385from typing import Iterable
386from typing import List
387from typing import Optional
388from typing import overload
389from typing import Set
390from typing import SupportsIndex
391from typing import Tuple
392from typing import TYPE_CHECKING
393from typing import TypeVar
394from typing import Union
395import weakref
396from weakref import WeakKeyDictionary
398from .. import event
399from .. import inspect
400from .. import types
401from ..orm import Mapper
402from ..orm._typing import _ExternalEntityType
403from ..orm._typing import _O
404from ..orm._typing import _T
405from ..orm.attributes import AttributeEventToken
406from ..orm.attributes import flag_modified
407from ..orm.attributes import InstrumentedAttribute
408from ..orm.attributes import QueryableAttribute
409from ..orm.context import QueryContext
410from ..orm.decl_api import DeclarativeAttributeIntercept
411from ..orm.state import InstanceState
412from ..orm.unitofwork import UOWTransaction
413from ..sql._typing import _TypeEngineArgument
414from ..sql.base import SchemaEventTarget
415from ..sql.schema import Column
416from ..sql.type_api import TypeEngine
417from ..util import memoized_property
419_KT = TypeVar("_KT") # Key type.
420_VT = TypeVar("_VT") # Value type.
423class MutableBase:
424 """Common base class to :class:`.Mutable`
425 and :class:`.MutableComposite`.
427 """
429 @memoized_property
430 def _parents(self) -> WeakKeyDictionary[Any, Any]:
431 """Dictionary of parent object's :class:`.InstanceState`->attribute
432 name on the parent.
434 This attribute is a so-called "memoized" property. It initializes
435 itself with a new ``weakref.WeakKeyDictionary`` the first time
436 it is accessed, returning the same object upon subsequent access.
438 .. versionchanged:: 1.4 the :class:`.InstanceState` is now used
439 as the key in the weak dictionary rather than the instance
440 itself.
442 """
444 return weakref.WeakKeyDictionary()
446 @classmethod
447 def coerce(cls, key: str, value: Any) -> Optional[Any]:
448 """Given a value, coerce it into the target type.
450 Can be overridden by custom subclasses to coerce incoming
451 data into a particular type.
453 By default, raises ``ValueError``.
455 This method is called in different scenarios depending on if
456 the parent class is of type :class:`.Mutable` or of type
457 :class:`.MutableComposite`. In the case of the former, it is called
458 for both attribute-set operations as well as during ORM loading
459 operations. For the latter, it is only called during attribute-set
460 operations; the mechanics of the :func:`.composite` construct
461 handle coercion during load operations.
464 :param key: string name of the ORM-mapped attribute being set.
465 :param value: the incoming value.
466 :return: the method should return the coerced value, or raise
467 ``ValueError`` if the coercion cannot be completed.
469 """
470 if value is None:
471 return None
472 msg = "Attribute '%s' does not accept objects of type %s"
473 raise ValueError(msg % (key, type(value)))
475 @classmethod
476 def _get_listen_keys(cls, attribute: QueryableAttribute[Any]) -> Set[str]:
477 """Given a descriptor attribute, return a ``set()`` of the attribute
478 keys which indicate a change in the state of this attribute.
480 This is normally just ``set([attribute.key])``, but can be overridden
481 to provide for additional keys. E.g. a :class:`.MutableComposite`
482 augments this set with the attribute keys associated with the columns
483 that comprise the composite value.
485 This collection is consulted in the case of intercepting the
486 :meth:`.InstanceEvents.refresh` and
487 :meth:`.InstanceEvents.refresh_flush` events, which pass along a list
488 of attribute names that have been refreshed; the list is compared
489 against this set to determine if action needs to be taken.
491 """
492 return {attribute.key}
494 @classmethod
495 def _listen_on_attribute(
496 cls,
497 attribute: QueryableAttribute[Any],
498 coerce: bool,
499 parent_cls: _ExternalEntityType[Any],
500 ) -> None:
501 """Establish this type as a mutation listener for the given
502 mapped descriptor.
504 """
505 key = attribute.key
506 if parent_cls is not attribute.class_:
507 return
509 # rely on "propagate" here
510 parent_cls = attribute.class_
512 listen_keys = cls._get_listen_keys(attribute)
514 def load(state: InstanceState[_O], *args: Any) -> None:
515 """Listen for objects loaded or refreshed.
517 Wrap the target data member's value with
518 ``Mutable``.
520 """
521 val = state.dict.get(key, None)
522 if val is not None:
523 if coerce:
524 val = cls.coerce(key, val)
525 assert val is not None
526 state.dict[key] = val
527 val._parents[state] = key
529 def load_attrs(
530 state: InstanceState[_O],
531 ctx: Union[object, QueryContext, UOWTransaction],
532 attrs: Iterable[Any],
533 ) -> None:
534 if not attrs or listen_keys.intersection(attrs):
535 load(state)
537 def set_(
538 target: InstanceState[_O],
539 value: MutableBase | None,
540 oldvalue: MutableBase | None,
541 initiator: AttributeEventToken,
542 ) -> MutableBase | None:
543 """Listen for set/replace events on the target
544 data member.
546 Establish a weak reference to the parent object
547 on the incoming value, remove it for the one
548 outgoing.
550 """
551 if value is oldvalue:
552 return value
554 if not isinstance(value, cls):
555 value = cls.coerce(key, value)
556 if value is not None:
557 value._parents[target] = key
558 if isinstance(oldvalue, cls):
559 oldvalue._parents.pop(inspect(target), None)
560 return value
562 def pickle(
563 state: InstanceState[_O], state_dict: Dict[str, Any]
564 ) -> None:
565 val = state.dict.get(key, None)
566 if val is not None:
567 if "ext.mutable.values" not in state_dict:
568 state_dict["ext.mutable.values"] = defaultdict(list)
569 state_dict["ext.mutable.values"][key].append(val)
571 def unpickle(
572 state: InstanceState[_O], state_dict: Dict[str, Any]
573 ) -> None:
574 if "ext.mutable.values" in state_dict:
575 collection = state_dict["ext.mutable.values"]
576 if isinstance(collection, list):
577 # legacy format
578 for val in collection:
579 val._parents[state] = key
580 else:
581 for val in state_dict["ext.mutable.values"][key]:
582 val._parents[state] = key
584 event.listen(
585 parent_cls,
586 "_sa_event_merge_wo_load",
587 load,
588 raw=True,
589 propagate=True,
590 )
592 event.listen(parent_cls, "load", load, raw=True, propagate=True)
593 event.listen(
594 parent_cls, "refresh", load_attrs, raw=True, propagate=True
595 )
596 event.listen(
597 parent_cls, "refresh_flush", load_attrs, raw=True, propagate=True
598 )
599 event.listen(
600 attribute, "set", set_, raw=True, retval=True, propagate=True
601 )
602 event.listen(parent_cls, "pickle", pickle, raw=True, propagate=True)
603 event.listen(
604 parent_cls, "unpickle", unpickle, raw=True, propagate=True
605 )
608class Mutable(MutableBase):
609 """Mixin that defines transparent propagation of change
610 events to a parent object.
612 See the example in :ref:`mutable_scalars` for usage information.
614 """
616 def changed(self) -> None:
617 """Subclasses should call this method whenever change events occur."""
619 for parent, key in self._parents.items():
620 flag_modified(parent.obj(), key)
622 @classmethod
623 def associate_with_attribute(
624 cls, attribute: InstrumentedAttribute[_O]
625 ) -> None:
626 """Establish this type as a mutation listener for the given
627 mapped descriptor.
629 """
630 cls._listen_on_attribute(attribute, True, attribute.class_)
632 @classmethod
633 def associate_with(cls, sqltype: type) -> None:
634 """Associate this wrapper with all future mapped columns
635 of the given type.
637 This is a convenience method that calls
638 ``associate_with_attribute`` automatically.
640 .. warning::
642 The listeners established by this method are *global*
643 to all mappers, and are *not* garbage collected. Only use
644 :meth:`.associate_with` for types that are permanent to an
645 application, not with ad-hoc types else this will cause unbounded
646 growth in memory usage.
648 """
650 def listen_for_type(mapper: Mapper[_O], class_: type) -> None:
651 for prop in mapper.column_attrs:
652 if isinstance(prop.columns[0].type, sqltype):
653 cls.associate_with_attribute(getattr(class_, prop.key))
655 event.listen(Mapper, "mapper_configured", listen_for_type)
657 @classmethod
658 def as_mutable(cls, sqltype: _TypeEngineArgument[_T]) -> TypeEngine[_T]:
659 """Associate a SQL type with this mutable Python type.
661 This establishes listeners that will detect ORM mappings against
662 the given type, adding mutation event trackers to those mappings.
664 The type is returned, unconditionally as an instance, so that
665 :meth:`.as_mutable` can be used inline::
667 Table(
668 "mytable",
669 metadata,
670 Column("id", Integer, primary_key=True),
671 Column("data", MyMutableType.as_mutable(PickleType)),
672 )
674 Note that the returned type is always an instance, even if a class
675 is given, and that only columns which are declared specifically with
676 that type instance receive additional instrumentation.
678 To associate a particular mutable type with all occurrences of a
679 particular type, use the :meth:`.Mutable.associate_with` classmethod
680 of the particular :class:`.Mutable` subclass to establish a global
681 association.
683 .. warning::
685 The listeners established by this method are *global*
686 to all mappers, and are *not* garbage collected. Only use
687 :meth:`.as_mutable` for types that are permanent to an application,
688 not with ad-hoc types else this will cause unbounded growth
689 in memory usage.
691 """
692 sqltype = types.to_instance(sqltype)
694 # a SchemaType will be copied when the Column is copied,
695 # and we'll lose our ability to link that type back to the original.
696 # so track our original type w/ columns
697 if isinstance(sqltype, SchemaEventTarget):
699 @event.listens_for(sqltype, "before_parent_attach")
700 def _add_column_memo(
701 sqltyp: TypeEngine[Any],
702 parent: Column[_T],
703 ) -> None:
704 parent.info["_ext_mutable_orig_type"] = sqltyp
706 schema_event_check = True
707 else:
708 schema_event_check = False
710 def listen_for_type(
711 mapper: Mapper[_T],
712 class_: Union[DeclarativeAttributeIntercept, type],
713 ) -> None:
714 _APPLIED_KEY = "_ext_mutable_listener_applied"
716 for prop in mapper.column_attrs:
717 if (
718 # all Mutable types refer to a Column that's mapped,
719 # since this is the only kind of Core target the ORM can
720 # "mutate"
721 isinstance(prop.expression, Column)
722 and (
723 (
724 schema_event_check
725 and prop.expression.info.get(
726 "_ext_mutable_orig_type"
727 )
728 is sqltype
729 )
730 or prop.expression.type is sqltype
731 )
732 ):
733 if not prop.expression.info.get(_APPLIED_KEY, False):
734 prop.expression.info[_APPLIED_KEY] = True
735 cls.associate_with_attribute(getattr(class_, prop.key))
737 event.listen(Mapper, "mapper_configured", listen_for_type)
739 return sqltype
742class MutableComposite(MutableBase):
743 """Mixin that defines transparent propagation of change
744 events on a SQLAlchemy "composite" object to its
745 owning parent or parents.
747 See the example in :ref:`mutable_composites` for usage information.
749 """
751 @classmethod
752 def _get_listen_keys(cls, attribute: QueryableAttribute[_O]) -> Set[str]:
753 return {attribute.key}.union(attribute.property._attribute_keys)
755 def changed(self) -> None:
756 """Subclasses should call this method whenever change events occur."""
758 for parent, key in self._parents.items():
759 prop = parent.mapper.get_property(key)
760 for value, attr_name in zip(
761 prop._composite_values_from_instance(self),
762 prop._attribute_keys,
763 ):
764 setattr(parent.obj(), attr_name, value)
767def _setup_composite_listener() -> None:
768 def _listen_for_type(mapper: Mapper[_T], class_: type) -> None:
769 for prop in mapper.iterate_properties:
770 if (
771 hasattr(prop, "composite_class")
772 and isinstance(prop.composite_class, type)
773 and issubclass(prop.composite_class, MutableComposite)
774 ):
775 prop.composite_class._listen_on_attribute(
776 getattr(class_, prop.key), False, class_
777 )
779 if not event.contains(Mapper, "mapper_configured", _listen_for_type):
780 event.listen(Mapper, "mapper_configured", _listen_for_type)
783_setup_composite_listener()
786class MutableDict(Mutable, Dict[_KT, _VT]):
787 """A dictionary type that implements :class:`.Mutable`.
789 The :class:`.MutableDict` object implements a dictionary that will
790 emit change events to the underlying mapping when the contents of
791 the dictionary are altered, including when values are added or removed.
793 Note that :class:`.MutableDict` does **not** apply mutable tracking to the
794 *values themselves* inside the dictionary. Therefore it is not a sufficient
795 solution for the use case of tracking deep changes to a *recursive*
796 dictionary structure, such as a JSON structure. To support this use case,
797 build a subclass of :class:`.MutableDict` that provides appropriate
798 coercion to the values placed in the dictionary so that they too are
799 "mutable", and emit events up to their parent structure.
801 .. seealso::
803 :class:`.MutableList`
805 :class:`.MutableSet`
807 """
809 def __setitem__(self, key: _KT, value: _VT) -> None:
810 """Detect dictionary set events and emit change events."""
811 dict.__setitem__(self, key, value)
812 self.changed()
814 if TYPE_CHECKING:
815 # from https://github.com/python/mypy/issues/14858
817 @overload
818 def setdefault(
819 self: MutableDict[_KT, Optional[_T]], key: _KT, value: None = None
820 ) -> Optional[_T]: ...
822 @overload
823 def setdefault(self, key: _KT, value: _VT) -> _VT: ...
825 def setdefault(self, key: _KT, value: object = None) -> object: ...
827 else:
829 def setdefault(self, *arg): # noqa: F811
830 result = dict.setdefault(self, *arg)
831 self.changed()
832 return result
834 def __delitem__(self, key: _KT) -> None:
835 """Detect dictionary del events and emit change events."""
836 dict.__delitem__(self, key)
837 self.changed()
839 def update(self, *a: Any, **kw: _VT) -> None:
840 dict.update(self, *a, **kw)
841 self.changed()
843 if TYPE_CHECKING:
845 @overload
846 def pop(self, __key: _KT, /) -> _VT: ...
848 @overload
849 def pop(self, __key: _KT, default: _VT | _T, /) -> _VT | _T: ...
851 def pop(
852 self, __key: _KT, __default: _VT | _T | None = None, /
853 ) -> _VT | _T: ...
855 else:
857 def pop(self, *arg): # noqa: F811
858 result = dict.pop(self, *arg)
859 self.changed()
860 return result
862 def popitem(self) -> Tuple[_KT, _VT]:
863 result = dict.popitem(self)
864 self.changed()
865 return result
867 def clear(self) -> None:
868 dict.clear(self)
869 self.changed()
871 @classmethod
872 def coerce(cls, key: str, value: Any) -> MutableDict[_KT, _VT] | None:
873 """Convert plain dictionary to instance of this class."""
874 if not isinstance(value, cls):
875 if isinstance(value, dict):
876 return cls(value)
877 return Mutable.coerce(key, value)
878 else:
879 return value
881 def __getstate__(self) -> Dict[_KT, _VT]:
882 return dict(self)
884 def __setstate__(
885 self, state: Union[Dict[str, int], Dict[str, str]]
886 ) -> None:
887 self.update(state)
890class MutableList(Mutable, List[_T]):
891 """A list type that implements :class:`.Mutable`.
893 The :class:`.MutableList` object implements a list that will
894 emit change events to the underlying mapping when the contents of
895 the list are altered, including when values are added or removed.
897 Note that :class:`.MutableList` does **not** apply mutable tracking to the
898 *values themselves* inside the list. Therefore it is not a sufficient
899 solution for the use case of tracking deep changes to a *recursive*
900 mutable structure, such as a JSON structure. To support this use case,
901 build a subclass of :class:`.MutableList` that provides appropriate
902 coercion to the values placed in the dictionary so that they too are
903 "mutable", and emit events up to their parent structure.
905 .. seealso::
907 :class:`.MutableDict`
909 :class:`.MutableSet`
911 """
913 def __reduce_ex__(
914 self, proto: SupportsIndex
915 ) -> Tuple[type, Tuple[List[int]]]:
916 return (self.__class__, (list(self),))
918 # needed for backwards compatibility with
919 # older pickles
920 def __setstate__(self, state: Iterable[_T]) -> None:
921 self[:] = state
923 def __setitem__(
924 self, index: SupportsIndex | slice, value: _T | Iterable[_T]
925 ) -> None:
926 """Detect list set events and emit change events."""
927 list.__setitem__(self, index, value)
928 self.changed()
930 def __delitem__(self, index: SupportsIndex | slice) -> None:
931 """Detect list del events and emit change events."""
932 list.__delitem__(self, index)
933 self.changed()
935 def pop(self, *arg: SupportsIndex) -> _T:
936 result = list.pop(self, *arg)
937 self.changed()
938 return result
940 def append(self, x: _T) -> None:
941 list.append(self, x)
942 self.changed()
944 def extend(self, x: Iterable[_T]) -> None:
945 list.extend(self, x)
946 self.changed()
948 def __iadd__(self, x: Iterable[_T]) -> MutableList[_T]: # type: ignore[override,misc] # noqa: E501
949 self.extend(x)
950 return self
952 def insert(self, i: SupportsIndex, x: _T) -> None:
953 list.insert(self, i, x)
954 self.changed()
956 def remove(self, i: _T) -> None:
957 list.remove(self, i)
958 self.changed()
960 def clear(self) -> None:
961 list.clear(self)
962 self.changed()
964 def sort(self, **kw: Any) -> None:
965 list.sort(self, **kw)
966 self.changed()
968 def reverse(self) -> None:
969 list.reverse(self)
970 self.changed()
972 @classmethod
973 def coerce(
974 cls, key: str, value: MutableList[_T] | _T
975 ) -> Optional[MutableList[_T]]:
976 """Convert plain list to instance of this class."""
977 if not isinstance(value, cls):
978 if isinstance(value, list):
979 return cls(value)
980 return Mutable.coerce(key, value)
981 else:
982 return value
985class MutableSet(Mutable, Set[_T]):
986 """A set type that implements :class:`.Mutable`.
988 The :class:`.MutableSet` object implements a set that will
989 emit change events to the underlying mapping when the contents of
990 the set are altered, including when values are added or removed.
992 Note that :class:`.MutableSet` does **not** apply mutable tracking to the
993 *values themselves* inside the set. Therefore it is not a sufficient
994 solution for the use case of tracking deep changes to a *recursive*
995 mutable structure. To support this use case,
996 build a subclass of :class:`.MutableSet` that provides appropriate
997 coercion to the values placed in the dictionary so that they too are
998 "mutable", and emit events up to their parent structure.
1000 .. seealso::
1002 :class:`.MutableDict`
1004 :class:`.MutableList`
1007 """
1009 def update(self, *arg: Iterable[_T]) -> None:
1010 set.update(self, *arg)
1011 self.changed()
1013 def intersection_update(self, *arg: Iterable[Any]) -> None:
1014 set.intersection_update(self, *arg)
1015 self.changed()
1017 def difference_update(self, *arg: Iterable[Any]) -> None:
1018 set.difference_update(self, *arg)
1019 self.changed()
1021 def symmetric_difference_update(self, *arg: Iterable[_T]) -> None:
1022 set.symmetric_difference_update(self, *arg)
1023 self.changed()
1025 def __ior__(self, other: AbstractSet[_T]) -> MutableSet[_T]: # type: ignore[override,misc] # noqa: E501
1026 self.update(other)
1027 return self
1029 def __iand__(self, other: AbstractSet[object]) -> MutableSet[_T]:
1030 self.intersection_update(other)
1031 return self
1033 def __ixor__(self, other: AbstractSet[_T]) -> MutableSet[_T]: # type: ignore[override,misc] # noqa: E501
1034 self.symmetric_difference_update(other)
1035 return self
1037 def __isub__(self, other: AbstractSet[object]) -> MutableSet[_T]: # type: ignore[misc] # noqa: E501
1038 self.difference_update(other)
1039 return self
1041 def add(self, elem: _T) -> None:
1042 set.add(self, elem)
1043 self.changed()
1045 def remove(self, elem: _T) -> None:
1046 set.remove(self, elem)
1047 self.changed()
1049 def discard(self, elem: _T) -> None:
1050 set.discard(self, elem)
1051 self.changed()
1053 def pop(self, *arg: Any) -> _T:
1054 result = set.pop(self, *arg)
1055 self.changed()
1056 return result
1058 def clear(self) -> None:
1059 set.clear(self)
1060 self.changed()
1062 @classmethod
1063 def coerce(cls, index: str, value: Any) -> Optional[MutableSet[_T]]:
1064 """Convert plain set to instance of this class."""
1065 if not isinstance(value, cls):
1066 if isinstance(value, set):
1067 return cls(value)
1068 return Mutable.coerce(index, value)
1069 else:
1070 return value
1072 def __getstate__(self) -> Set[_T]:
1073 return set(self)
1075 def __setstate__(self, state: Iterable[_T]) -> None:
1076 self.update(state)
1078 def __reduce_ex__(
1079 self, proto: SupportsIndex
1080 ) -> Tuple[type, Tuple[List[int]]]:
1081 return (self.__class__, (list(self),))