Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# ___
2# \./ DANGER: This project implements some code generation
3# .--.O.--. techniques involving string concatenation.
4# \/ \/ If you look at it, you might die.
5#
7r"""
8Installation
9************
11.. code-block:: bash
13 pip install fastjsonschema
15Support only for Python 3.3 and higher.
17About
18*****
20``fastjsonschema`` implements validation of JSON documents by JSON schema.
21The library implements JSON schema drafts 04, 06, and 07. The main purpose is
22to have a really fast implementation. See some numbers:
24 * Probably the most popular, ``jsonschema``, can take up to 5 seconds for valid
25 inputs and 1.2 seconds for invalid inputs.
26 * Second most popular, ``json-spec``, is even worse with up to 7.2 and 1.7 seconds.
27 * Last ``validictory``, now deprecated, is much better with 370 or 23 milliseconds,
28 but it does not follow all standards, and it can be still slow for some purposes.
30With this library you can gain big improvements as ``fastjsonschema`` takes
31only about 25 milliseconds for valid inputs and 2 milliseconds for invalid ones.
32Pretty amazing, right? :-)
34Technically it works by generating the most stupid code on the fly, which is fast but
35is hard to write by hand. The best efficiency is achieved when a validator is compiled
36once and used many times, of course. It works similarly like regular expressions. But
37you can also generate the code to a file, which is even slightly faster.
39You can run the performance benchmarks on your computer or server with the included
40script:
42.. code-block:: bash
44 $ make performance
45 fast_compiled valid ==> 0.0993900
46 fast_compiled invalid ==> 0.0041089
47 fast_compiled_without_exc valid ==> 0.0465258
48 fast_compiled_without_exc invalid ==> 0.0023688
49 fast_file valid ==> 0.0989483
50 fast_file invalid ==> 0.0041104
51 fast_not_compiled valid ==> 11.9572681
52 fast_not_compiled invalid ==> 2.9512092
53 jsonschema valid ==> 5.2233240
54 jsonschema invalid ==> 1.3227916
55 jsonschema_compiled valid ==> 0.4447982
56 jsonschema_compiled invalid ==> 0.0231333
57 jsonspec valid ==> 4.1450569
58 jsonspec invalid ==> 1.0485777
59 validictory valid ==> 0.2730411
60 validictory invalid ==> 0.0183669
62This library follows and implements `JSON schema draft-04, draft-06, and draft-07
63<http://json-schema.org>`_. Sometimes it's not perfectly clear, so I recommend also
64check out this `understanding JSON schema <https://spacetelescope.github.io/understanding-json-schema>`_.
66Note that there are some differences compared to JSON schema standard:
68 * Regular expressions are full Python ones, not only what JSON schema allows. It's easier
69 to allow everything, and also it's faster to compile without limits. So keep in mind that when
70 you will use a more advanced regular expression, it may not work with other libraries or in
71 other languages.
72 * Because Python matches new line for a dollar in regular expressions (``a$`` matches ``a`` and ``a\\n``),
73 instead of ``$`` is used ``\Z`` and all dollars in your regular expression are changed to ``\\Z``
74 as well. When you want to use dollar as regular character, you have to escape it (``\$``).
75 * JSON schema says you can use keyword ``default`` for providing default values. This implementation
76 uses that and always returns transformed input data.
78Usage
79*****
81.. code-block:: python
83 import fastjsonschema
85 point_schema = {
86 "type": "object",
87 "properties": {
88 "x": {
89 "type": "number",
90 },
91 "y": {
92 "type": "number",
93 },
94 },
95 "required": ["x", "y"],
96 "additionalProperties": False,
97 }
99 point_validator = fastjsonschema.compile(point_schema)
100 try:
101 point_validator({"x": 1.0, "y": 2.0})
102 except fastjsonschema.JsonSchemaException as e:
103 print(f"Data failed validation: {e}")
105API
106***
107"""
108from collections.abc import Callable, Mapping
109from functools import partial, update_wrapper
110from typing import Any
112from .draft04 import CodeGeneratorDraft04
113from .draft06 import CodeGeneratorDraft06
114from .draft07 import CodeGeneratorDraft07
115from .draft2019 import CodeGeneratorDraft2019
116from .exceptions import (
117 JsonSchemaException,
118 JsonSchemaValueException,
119 JsonSchemaValuesException,
120 JsonSchemaDefinitionException,
121)
122from .ref_resolver import RefResolver
123from .version import VERSION
125__all__ = (
126 'VERSION',
127 'JsonSchemaException',
128 'JsonSchemaValueException',
129 'JsonSchemaValuesException',
130 'JsonSchemaDefinitionException',
131 'validate',
132 'compile',
133 'compile_to_code',
134)
137Definition = dict[str, Any] | bool
138Handlers = Mapping[str, Callable[[str], Any]]
139Formats = Mapping[str, str | Callable[[Any], bool]]
140Validator = Callable[..., Any]
143def validate(
144 definition: Definition,
145 data: Any,
146 handlers: Handlers = {},
147 formats: Formats = {},
148 use_default: bool = True,
149 use_formats: bool = True,
150 detailed_exceptions: bool = True,
151 fast_fail: bool = True,
152) -> Any:
153 """
154 Validation function for lazy programmers or for use cases when you need
155 to call validation only once, so you do not have to compile it first.
156 Use it only when you do not care about performance (even though it will
157 be still faster than alternative implementations).
159 .. code-block:: python
161 import fastjsonschema
163 fastjsonschema.validate({'type': 'string'}, 'hello')
164 # same as: compile({'type': 'string'})('hello')
166 Preferred is to use :any:`compile` function.
168 The ``handlers`` parameter controls resolution of remote ``$ref`` URIs; see
169 :any:`compile` for details and security considerations when schemas are not
170 fully trusted.
171 """
172 return compile(definition, handlers, formats, use_default, use_formats, detailed_exceptions, fast_fail)(data)
175#TODO: Change use_default to False when upgrading to version 3.
176# pylint: disable=redefined-builtin,dangerous-default-value,exec-used
177def compile(
178 definition: Definition,
179 handlers: Handlers = {},
180 formats: Formats = {},
181 use_default: bool = True,
182 use_formats: bool = True,
183 detailed_exceptions: bool = True,
184 fast_fail: bool = True,
185) -> Validator:
186 """
187 Generates validation function for validating JSON schema passed in ``definition``.
188 Example:
190 .. code-block:: python
192 import fastjsonschema
194 validate = fastjsonschema.compile({'type': 'string'})
195 validate('hello')
197 This implementation supports keyword ``default`` (can be turned off
198 by passing `use_default=False`):
200 .. code-block:: python
202 validate = fastjsonschema.compile({
203 'type': 'object',
204 'properties': {
205 'a': {'type': 'number', 'default': 42},
206 },
207 })
209 data = validate({})
210 assert data == {'a': 42}
212 Supported implementations are draft-04, draft-06 and draft-07. Which version
213 should be used is determined by `$draft` in your ``definition``. When not
214 specified, the latest implementation is used (draft-07).
216 .. code-block:: python
218 validate = fastjsonschema.compile({
219 '$schema': 'http://json-schema.org/draft-04/schema',
220 'type': 'number',
221 })
223 You can pass mapping from URI scheme to function that should be used to
224 retrieve remote references used in your ``definition`` in parameter
225 ``handlers``. When no handler is registered for a scheme, the URI is
226 fetched automatically via :mod:`urllib` (for example ``http``, ``https``,
227 or ``file`` URLs).
229 .. warning::
231 Do not compile or validate untrusted schemas without custom
232 ``handlers``. A schema containing ``$ref`` can trigger outbound HTTP
233 requests to arbitrary URLs, including internal or loopback addresses
234 (server-side request forgery). Provide ``handlers`` to restrict which
235 URIs are resolved, or pre-resolve references before passing the schema
236 to this library.
238 .. code-block:: python
240 def http_handler(uri):
241 if not uri.startswith('https://schemas.example.com/'):
242 raise ValueError('ref not allowed')
243 import urllib.request
244 with urllib.request.urlopen(uri) as response:
245 return json.loads(response.read())
247 validate = fastjsonschema.compile(definition, handlers={
248 'http': http_handler,
249 'https': http_handler,
250 })
252 Also, you can pass mapping for custom formats. Key is the name of your
253 formatter and value can be regular expression, which will be compiled or
254 callback returning `bool` (or you can raise your own exception).
256 .. code-block:: python
258 validate = fastjsonschema.compile(definition, formats={
259 'foo': r'foo|bar',
260 'bar': lambda value: value in ('foo', 'bar'),
261 })
263 Note that formats are automatically used as assertions. It can be turned
264 off by passing `use_formats=False`. When disabled, custom formats are
265 disabled as well. (Added in 2.19.0.)
267 If you don't need detailed exceptions, you can turn the details off and gain
268 additional performance by passing `detailed_exceptions=False`.
270 By default, the execution stops with the first validation error. If you need
271 to collect all the errors, turn this off by passing `fast_fail=False`.
273 Exception :any:`JsonSchemaDefinitionException` is raised when generating the
274 code fails (bad definition).
276 Exception :any:`JsonSchemaValueException` is raised from generated function when
277 validation fails (data do not follow the definition).
279 Exception :any:`JsonSchemaValuesException` is raised from generated function when
280 validation fails (data do not follow the definition) contatining all the errors
281 (when fast_fail is set to `False`).
282 """
283 resolver, code_generator = _factory(
284 definition,
285 handlers,
286 formats,
287 use_default,
288 use_formats,
289 detailed_exceptions,
290 fast_fail,
291 )
292 global_state = code_generator.global_state
293 # Do not pass local state so it can recursively call itself.
294 exec(code_generator.func_code, global_state)
295 func = global_state[resolver.get_scope_name()]
296 if formats:
297 return update_wrapper(partial(func, custom_formats=formats), func)
298 return func
301# pylint: disable=dangerous-default-value
302def compile_to_code(
303 definition: Definition,
304 handlers: Handlers = {},
305 formats: Formats = {},
306 use_default: bool = True,
307 use_formats: bool = True,
308 detailed_exceptions: bool = True,
309 fast_fail: bool = True,
310) -> str:
311 """
312 Generates validation code for validating JSON schema passed in ``definition``.
313 Example:
315 .. code-block:: python
317 import fastjsonschema
319 code = fastjsonschema.compile_to_code({'type': 'string'})
320 with open('your_file.py', 'w') as f:
321 f.write(code)
323 You can also use it as a script:
325 .. code-block:: bash
327 echo "{'type': 'string'}" | python3 -m fastjsonschema > your_file.py
328 python3 -m fastjsonschema "{'type': 'string'}" > your_file.py
330 Exception :any:`JsonSchemaDefinitionException` is raised when generating the
331 code fails (bad definition).
333 Remote ``$ref`` URIs are resolved the same way as in :any:`compile`; see its
334 documentation for ``handlers`` and security considerations.
335 """
336 _, code_generator = _factory(
337 definition,
338 handlers,
339 formats,
340 use_default,
341 use_formats,
342 detailed_exceptions,
343 fast_fail,
344 )
345 return (
346 'VERSION = "' + VERSION + '"\n' +
347 code_generator.global_state_code + '\n' +
348 code_generator.func_code
349 )
352def _factory(
353 definition: Definition,
354 handlers: Handlers,
355 formats: Formats = {},
356 use_default: bool = True,
357 use_formats: bool = True,
358 detailed_exceptions: bool = True,
359 fast_fail: bool = True,
360) -> tuple[RefResolver, CodeGeneratorDraft04]:
361 resolver = RefResolver.from_schema(definition, handlers=handlers, store={})
362 code_generator = _get_code_generator_class(definition)(
363 definition,
364 resolver=resolver,
365 formats=formats,
366 use_default=use_default,
367 use_formats=use_formats,
368 detailed_exceptions=detailed_exceptions,
369 fast_fail=fast_fail,
370 )
371 return resolver, code_generator
374def _get_code_generator_class(schema: Definition) -> type[CodeGeneratorDraft04]:
375 # Schema in from draft-06 can be just the boolean value.
376 if isinstance(schema, dict):
377 schema_version = schema.get('$schema', '')
378 if 'draft-04' in schema_version:
379 return CodeGeneratorDraft04
380 if 'draft-06' in schema_version:
381 return CodeGeneratorDraft06
382 if 'draft-07' in schema_version:
383 return CodeGeneratorDraft07
384 if 'draft/2019' in schema_version or 'draft-2019' in schema_version:
385 return CodeGeneratorDraft2019
386 return CodeGeneratorDraft2019