You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
65 lines
2.8 KiB
65 lines
2.8 KiB
7 months ago
|
# Test Runner Developer Guide
|
||
|
|
|
||
|
|
Learn about test runners and how to create a new test runner class.
|
||
|
|
|
||
|
|
##### Table of Contents
|
||
|
|
1. [Test Runner Details](#test-runner-details)
|
||
|
|
2. [Creating a Test Runner](#creating-a-test-runner)
|
||
|
|
|
||
|
|
## <a name="test-runner-details">Test Runner Details</a>
|
||
|
|
|
||
|
|
The test runner class is responsible for test execution. Its primary logic
|
||
|
|
involve construction of the commandline given a ```TestInfo``` and
|
||
|
|
```extra_args``` passed into the ```run_tests``` method. The extra_args are
|
||
|
|
top-level args consumed by atest passed onto the test runner. It is up to the
|
||
|
|
test runner to translate those args into the specific args the test runner
|
||
|
|
accepts. In this way, you can think of the test runner as a translator between
|
||
|
|
the atest CLI and your test runner's CLI. The reason for this is so that atest
|
||
|
|
can have a consistent CLI for args instead of requiring the users to remember
|
||
|
|
the differing CLIs of various test runners. The test runner should also
|
||
|
|
determine its specific dependencies that need to be built prior to any test
|
||
|
|
execution.
|
||
|
|
|
||
|
|
## <a name="creating-a-test-runner">Creating a Test Runner</a>
|
||
|
|
|
||
|
|
First thing to choose is where to put the test runner. This will primarily
|
||
|
|
depend on if the test runner will be public or private. If public,
|
||
|
|
```test_runners/``` is the default location.
|
||
|
|
|
||
|
|
> If it will be private, then you can
|
||
|
|
> follow the same instructions for ```vendorsetup.sh``` in
|
||
|
|
> [constants override](atest_structure.md#constants-override) where you will
|
||
|
|
> add the path of where the test runner lives into ```$PYTHONPATH```. Same
|
||
|
|
> rules apply, rerun ```build/envsetup.sh``` to update ```$PYTHONPATH```.
|
||
|
|
|
||
|
|
To create a new test runner, create a new class that inherits
|
||
|
|
```TestRunnerBase```. Take a look at ```test_runners/example_test_runner.py```
|
||
|
|
to see what a simple test runner will look like.
|
||
|
|
|
||
|
|
**Important Notes**
|
||
|
|
You'll need to override the following parent methods:
|
||
|
|
* ```host_env_check()```: Check if host environment is properly setup for the
|
||
|
|
test runner. Raise an expception if not.
|
||
|
|
* ```get_test_runner_build_reqs()```: Return a set of build targets that need
|
||
|
|
to be built prior to test execution.
|
||
|
|
* ```run_tests()```: Execute the test(s).
|
||
|
|
|
||
|
|
And define the following class vars:
|
||
|
|
* ```NAME```: Unique name of the test runner.
|
||
|
|
* ```EXECUTABLE```: Test runner command, should be an absolute path if the
|
||
|
|
command can not be found in ```$PATH```.
|
||
|
|
|
||
|
|
There is a parent helper method (```run```) that should be used to execute the
|
||
|
|
actual test command.
|
||
|
|
|
||
|
|
Once the test runner class is created, you'll need to add it in
|
||
|
|
```test_runner_handler``` so that atest is aware of it. Try-except import the
|
||
|
|
test runner in ```_get_test_runners``` like how ```ExampleTestRunner``` is.
|
||
|
|
```python
|
||
|
|
try:
|
||
|
|
from test_runners import new_test_runner
|
||
|
|
test_runners_dict[new_test_runner.NewTestRunner.NAME] = new_test_runner.NewTestRunner
|
||
|
|
except ImportError:
|
||
|
|
pass
|
||
|
|
```
|