source code: py/test/pytests/README.md
Chrome OS Reference Python Factory Tests¶
This folder contains all reference factory tests for Chrome OS Factory Software.
For more details of individual test, please look at the SDK document or read the source of individual tests.
Pytest Overview¶
Take start.py
as an example, a pytest should contain 4 sections.
Copyright header
Test documentation
Imports
Implementation
Copyright header¶
Always add copyright header to the beginning of the file.
# Copyright 2022 The ChromiumOS Authors
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
2022
is year of file created date.
Test Documentation¶
To help both developers and non-developers understand better how these tests works, how to use it and what’s needed to make it run, we have defined a template for documentation. The tests will be processed by Sphinx Documentation, so you just need to write the module doc string in reStructedText format with following sections:
"""Test summary here.
Description
-----------
A section for who wants to know what this test is, and how to use it.
You probably want to write the first section as a general description, and
remaining sections to explain more details, and how to assign the arguments
if the test arguments are pretty complicated (or try using a JSON configuration
via `config_utils` instead).
Test Procedure
--------------
Write "This is an automated test without user interaction." as first line if
the test does not need user interaction.
This is a section as "reference to write SOP for operators". Use simpler words
and abstraction of what will happen.
Dependency
----------
This is a section about "what people will need when they want to port the test
for a new platform, for example Android or even Windows.
Try to list everything needed outside Chrome OS factory repo, especially
kernel and driver dependency.
If you are using Device API (`cros.factory.device.*`), try to list the explicit
API you are using, with few typical dependency that it needs.
Examples
--------
Examples of how to use this test (in test list). Usually we want a "minimal" one
explaining what default behavior it will do, with few more examples to
demonstrate how to use the arguments.
"""
To preview your changes to doc, run make doc
in chroot and browse
build/doc/pytests/index.html
.
For more examples on how to write these docs and how they looks on SDK site, try to look at following tests:
Imports¶
Import lines are organized in 3 categories:
Standard python libraries, e.g.
import os
Third-party modules, e.g.
import jsonlibrpc
ChromeOS Factory modules
Implementation¶
All pytests are a python class inherits test_case.TestCase
,
which is a subclass of unittest.TestCase
from Python
unittest module. When a
pytest is executed, the following functions will be called (in exact order):
setUp()
runTest()
tearDown()
Writing a new test¶
All the tests must have implementation file named in
lowercase_with_underline
style and the Python class named in CamelCase
style, following PEP-8 and Chrome
OS Factory Coding Style. You can write test as a single Python file like
start.py
, or implement it as Python package in its own folder like
probe/
.
To use your test (say mytest.py
), define an entry in
test list:
{
"pytest_name": "mytest"
}
Writting a test with reboot process¶
There are two ways to write a test that contains reboot process.
We recommended to use
shutdown.py
if feasible.
You can addRebootStep
as an independent test item in the test group, it will trigger a reboot and continue executing the remain test items after reboot.For example:
{ "label": "Test Group Label", "subtests": [ "PytestWithReboot", "RebootStep", "PytestWithReboot" ] }
Sometimes the reboot is triggered by 3rd-party commands/operations, which is not supported by
shutdown.py
.
In this situation, you can setallow_reboot: true
for the test item in test_list, and then useAddTask(task, *args, reboot=True, **kwargs)
to add tasks in your pytest.For example:
def Task(arg): # task ... def RebootTask(arg): # task with reboot process ... AddTask(Task, arg) AddTask(RebootTask, arg, reboot=True) AddTask(Task, arg)
See JSON_TEST_LIST.md and test_case.py for more details.
Using arguments¶
To read arguments specified from test list, use
cros.factory.utils.arg_utils.Arg
by adding declarations as class variable. For
example:
ARGS = [Arg('wait_secs', int, 'Wait for N seconds.', default=0)]
Then you can use this by reading self.args.wait_secs
.
Since JSON serialization doesn’t support tuple
type, new pytests shouldn’t
use tuple
in argument type, and should use list
instead.
Schema validation¶
If the argument is complicated, you may want to define a schema rule to validate the structure of the schema. For example:
Arg(..., schema=schema_object)
The schema object should be one of the schema classes defined in schema.py. Then the argument value would automatically be validated with the schema rule.
You can check tablet_rotation.py as an example usage.
Using user interface¶
The Chrome OS Factory Software provides a web based user interface. The test UI
object will be injected to test instance as self.ui
. Read
Test UI API
for more details.