Shortcuts on this page
r m x toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# ___
2# \./ DANGER: This project implements some code generation
3# .--.O.--. techniques involving string concatenation.
4# \/ \/ If you look at it, you might die.
5#
7r"""
8Installation
9************
11.. code-block:: bash
13 pip install fastjsonschema
15Support only for Python 3.3 and higher.
17About
18*****
20``fastjsonschema`` implements validation of JSON documents by JSON schema.
21The library implements JSON schema drafts 04, 06, and 07. The main purpose is
22to have a really fast implementation. See some numbers:
24 * Probably the most popular, ``jsonschema``, can take up to 5 seconds for valid
25 inputs and 1.2 seconds for invalid inputs.
26 * Second most popular, ``json-spec``, is even worse with up to 7.2 and 1.7 seconds.
27 * Last ``validictory``, now deprecated, is much better with 370 or 23 milliseconds,
28 but it does not follow all standards, and it can be still slow for some purposes.
30With this library you can gain big improvements as ``fastjsonschema`` takes
31only about 25 milliseconds for valid inputs and 2 milliseconds for invalid ones.
32Pretty amazing, right? :-)
34Technically it works by generating the most stupid code on the fly, which is fast but
35is hard to write by hand. The best efficiency is achieved when a validator is compiled
36once and used many times, of course. It works similarly like regular expressions. But
37you can also generate the code to a file, which is even slightly faster.
39You can run the performance benchmarks on your computer or server with the included
40script:
42.. code-block:: bash
44 $ make performance
45 fast_compiled valid ==> 0.0993900
46 fast_compiled invalid ==> 0.0041089
47 fast_compiled_without_exc valid ==> 0.0465258
48 fast_compiled_without_exc invalid ==> 0.0023688
49 fast_file valid ==> 0.0989483
50 fast_file invalid ==> 0.0041104
51 fast_not_compiled valid ==> 11.9572681
52 fast_not_compiled invalid ==> 2.9512092
53 jsonschema valid ==> 5.2233240
54 jsonschema invalid ==> 1.3227916
55 jsonschema_compiled valid ==> 0.4447982
56 jsonschema_compiled invalid ==> 0.0231333
57 jsonspec valid ==> 4.1450569
58 jsonspec invalid ==> 1.0485777
59 validictory valid ==> 0.2730411
60 validictory invalid ==> 0.0183669
62This library follows and implements `JSON schema draft-04, draft-06, and draft-07
63<http://json-schema.org>`_. Sometimes it's not perfectly clear, so I recommend also
64check out this `understanding JSON schema <https://spacetelescope.github.io/understanding-json-schema>`_.
66Note that there are some differences compared to JSON schema standard:
68 * Regular expressions are full Python ones, not only what JSON schema allows. It's easier
69 to allow everything, and also it's faster to compile without limits. So keep in mind that when
70 you will use a more advanced regular expression, it may not work with other libraries or in
71 other languages.
72 * Because Python matches new line for a dollar in regular expressions (``a$`` matches ``a`` and ``a\\n``),
73 instead of ``$`` is used ``\Z`` and all dollars in your regular expression are changed to ``\\Z``
74 as well. When you want to use dollar as regular character, you have to escape it (``\$``).
75 * JSON schema says you can use keyword ``default`` for providing default values. This implementation
76 uses that and always returns transformed input data.
78Usage
79*****
81.. code-block:: python
83 import fastjsonschema
85 point_schema = {
86 "type": "object",
87 "properties": {
88 "x": {
89 "type": "number",
90 },
91 "y": {
92 "type": "number",
93 },
94 },
95 "required": ["x", "y"],
96 "additionalProperties": False,
97 }
99 point_validator = fastjsonschema.compile(point_schema)
100 try:
101 point_validator({"x": 1.0, "y": 2.0})
102 except fastjsonschema.JsonSchemaException as e:
103 print(f"Data failed validation: {e}")
105API
106***
107"""
108from functools import partial, update_wrapper
110from .draft04 import CodeGeneratorDraft04
111from .draft06 import CodeGeneratorDraft06
112from .draft07 import CodeGeneratorDraft07
113from .exceptions import (
114 JsonSchemaException,
115 JsonSchemaValueException,
116 JsonSchemaValuesException,
117 JsonSchemaDefinitionException,
118)
119from .ref_resolver import RefResolver
120from .version import VERSION
122__all__ = (
123 'VERSION',
124 'JsonSchemaException',
125 'JsonSchemaValueException',
126 'JsonSchemaValuesException',
127 'JsonSchemaDefinitionException',
128 'validate',
129 'compile',
130 'compile_to_code',
131)
134def validate(
135 definition: dict | bool,
136 data,
137 handlers: dict = {},
138 formats: dict = {},
139 use_default: bool = True,
140 use_formats: bool = True,
141 detailed_exceptions: bool = True,
142 fast_fail: bool = True,
143):
144 """
145 Validation function for lazy programmers or for use cases when you need
146 to call validation only once, so you do not have to compile it first.
147 Use it only when you do not care about performance (even though it will
148 be still faster than alternative implementations).
150 .. code-block:: python
152 import fastjsonschema
154 fastjsonschema.validate({'type': 'string'}, 'hello')
155 # same as: compile({'type': 'string'})('hello')
157 Preferred is to use :any:`compile` function.
158 """
159 return compile(definition, handlers, formats, use_default, use_formats, detailed_exceptions, fast_fail)(data)
162#TODO: Change use_default to False when upgrading to version 3.
163# pylint: disable=redefined-builtin,dangerous-default-value,exec-used
164def compile(
165 definition: dict | bool,
166 handlers: dict = {},
167 formats: dict = {},
168 use_default: bool = True,
169 use_formats: bool = True,
170 detailed_exceptions: bool = True,
171 fast_fail: bool = True,
172):
173 """
174 Generates validation function for validating JSON schema passed in ``definition``.
175 Example:
177 .. code-block:: python
179 import fastjsonschema
181 validate = fastjsonschema.compile({'type': 'string'})
182 validate('hello')
184 This implementation supports keyword ``default`` (can be turned off
185 by passing `use_default=False`):
187 .. code-block:: python
189 validate = fastjsonschema.compile({
190 'type': 'object',
191 'properties': {
192 'a': {'type': 'number', 'default': 42},
193 },
194 })
196 data = validate({})
197 assert data == {'a': 42}
199 Supported implementations are draft-04, draft-06 and draft-07. Which version
200 should be used is determined by `$draft` in your ``definition``. When not
201 specified, the latest implementation is used (draft-07).
203 .. code-block:: python
205 validate = fastjsonschema.compile({
206 '$schema': 'http://json-schema.org/draft-04/schema',
207 'type': 'number',
208 })
210 You can pass mapping from URI to function that should be used to retrieve
211 remote schemes used in your ``definition`` in parameter ``handlers``.
213 Also, you can pass mapping for custom formats. Key is the name of your
214 formatter and value can be regular expression, which will be compiled or
215 callback returning `bool` (or you can raise your own exception).
217 .. code-block:: python
219 validate = fastjsonschema.compile(definition, formats={
220 'foo': r'foo|bar',
221 'bar': lambda value: value in ('foo', 'bar'),
222 })
224 Note that formats are automatically used as assertions. It can be turned
225 off by passing `use_formats=False`. When disabled, custom formats are
226 disabled as well. (Added in 2.19.0.)
228 If you don't need detailed exceptions, you can turn the details off and gain
229 additional performance by passing `detailed_exceptions=False`.
231 By default, the execution stops with the first validation error. If you need
232 to collect all the errors, turn this off by passing `fast_fail=False`.
234 Exception :any:`JsonSchemaDefinitionException` is raised when generating the
235 code fails (bad definition).
237 Exception :any:`JsonSchemaValueException` is raised from generated function when
238 validation fails (data do not follow the definition).
240 Exception :any:`JsonSchemaValuesException` is raised from generated function when
241 validation fails (data do not follow the definition) contatining all the errors
242 (when fast_fail is set to `False`).
243 """
244 resolver, code_generator = _factory(
245 definition,
246 handlers,
247 formats,
248 use_default,
249 use_formats,
250 detailed_exceptions,
251 fast_fail,
252 )
253 global_state = code_generator.global_state
254 # Do not pass local state so it can recursively call itself.
255 exec(code_generator.func_code, global_state)
256 func = global_state[resolver.get_scope_name()]
257 if formats:
258 return update_wrapper(partial(func, custom_formats=formats), func)
259 return func
262# pylint: disable=dangerous-default-value
263def compile_to_code(
264 definition: dict | bool,
265 handlers: dict = {},
266 formats: dict = {},
267 use_default: bool = True,
268 use_formats: bool = True,
269 detailed_exceptions: bool = True,
270 fast_fail: bool = True,
271):
272 """
273 Generates validation code for validating JSON schema passed in ``definition``.
274 Example:
276 .. code-block:: python
278 import fastjsonschema
280 code = fastjsonschema.compile_to_code({'type': 'string'})
281 with open('your_file.py', 'w') as f:
282 f.write(code)
284 You can also use it as a script:
286 .. code-block:: bash
288 echo "{'type': 'string'}" | python3 -m fastjsonschema > your_file.py
289 python3 -m fastjsonschema "{'type': 'string'}" > your_file.py
291 Exception :any:`JsonSchemaDefinitionException` is raised when generating the
292 code fails (bad definition).
293 """
294 _, code_generator = _factory(
295 definition,
296 handlers,
297 formats,
298 use_default,
299 use_formats,
300 detailed_exceptions,
301 fast_fail,
302 )
303 return (
304 'VERSION = "' + VERSION + '"\n' +
305 code_generator.global_state_code + '\n' +
306 code_generator.func_code
307 )
310def _factory(
311 definition: dict | bool,
312 handlers: dict,
313 formats: dict = {},
314 use_default: bool = True,
315 use_formats: bool = True,
316 detailed_exceptions: bool = True,
317 fast_fail: bool = True,
318):
319 resolver = RefResolver.from_schema(definition, handlers=handlers, store={})
320 code_generator = _get_code_generator_class(definition)(
321 definition,
322 resolver=resolver,
323 formats=formats,
324 use_default=use_default,
325 use_formats=use_formats,
326 detailed_exceptions=detailed_exceptions,
327 fast_fail=fast_fail,
328 )
329 return resolver, code_generator
332def _get_code_generator_class(schema: dict | bool):
333 # Schema in from draft-06 can be just the boolean value.
334 if isinstance(schema, dict):
335 schema_version = schema.get('$schema', '')
336 if 'draft-04' in schema_version:
337 return CodeGeneratorDraft04
338 if 'draft-06' in schema_version:
339 return CodeGeneratorDraft06
340 return CodeGeneratorDraft07