v811_spc009/tools/asuite/atest/docs/develop_test_runners.md

Test Runner Developer Guide

Learn about test runners and how to create a new test runner class.

Table of Contents

1. Test Runner Details
2. Creating a Test Runner

Test Runner Details

The test runner class is responsible for test execution. Its primary logic involve construction of the commandline given a TestInfo and extra_args passed into the run_tests method. The extra_args are top-level args consumed by atest passed onto the test runner. It is up to the test runner to translate those args into the specific args the test runner accepts. In this way, you can think of the test runner as a translator between the atest CLI and your test runner's CLI. The reason for this is so that atest can have a consistent CLI for args instead of requiring the users to remember the differing CLIs of various test runners. The test runner should also determine its specific dependencies that need to be built prior to any test execution.

Creating a Test Runner

First thing to choose is where to put the test runner. This will primarily depend on if the test runner will be public or private. If public, test_runners/ is the default location.

If it will be private, then you can follow the same instructions for vendorsetup.sh in constants override where you will add the path of where the test runner lives into $PYTHONPATH. Same rules apply, rerun build/envsetup.sh to update $PYTHONPATH.

To create a new test runner, create a new class that inherits TestRunnerBase. Take a look at test_runners/example_test_runner.py to see what a simple test runner will look like.

Important Notes You'll need to override the following parent methods:

• host_env_check(): Check if host environment is properly setup for the test runner. Raise an expception if not.
• get_test_runner_build_reqs(): Return a set of build targets that need to be built prior to test execution.
• run_tests(): Execute the test(s).

And define the following class vars:

• NAME: Unique name of the test runner.
• EXECUTABLE: Test runner command, should be an absolute path if the command can not be found in $PATH.

There is a parent helper method (run) that should be used to execute the actual test command.

Once the test runner class is created, you'll need to add it in test_runner_handler so that atest is aware of it. Try-except import the test runner in _get_test_runners like how ExampleTestRunner is.

try:
    from test_runners import new_test_runner
    test_runners_dict[new_test_runner.NewTestRunner.NAME] = new_test_runner.NewTestRunner
except ImportError:
    pass