You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
133 lines
4.6 KiB
133 lines
4.6 KiB
4 months ago
|
# CrosVM Continuous Integration
|
||
|
|
|
||
|
|
Crosvm has a complex set of dependencies and requirements on the host machine to
|
||
|
|
successfully build and run test cases. To allow for consistent testing in our
|
||
|
|
continuous integration system (kokoro) and reproduction of those tests locally,
|
||
|
|
we provide docker containers containing the build toolchain and a VM for
|
||
|
|
testing.
|
||
|
|
|
||
|
|
## How to run crosvm tests
|
||
|
|
|
||
|
|
### Setting up the source
|
||
|
|
|
||
|
|
Since crosvm is part of chromiumos, and uses a couple of it's projects as
|
||
|
|
dependencies, you need a standard chromiumos checkout as described by the
|
||
|
|
[ChromiumOS Developer Guide](https://chromium.googlesource.com/chromiumos/docs/+/master/developer_guide.md#Get-the-Source).
|
||
|
|
|
||
|
|
To reduce the number of repositories to download, you can use the `-g crosvm`
|
||
|
|
argument on `repo init`. This will be significantly faster:
|
||
|
|
|
||
|
|
In summary:
|
||
|
|
|
||
|
|
```
|
||
|
|
$ repo init -u https://chromium.googlesource.com/chromiumos/manifest.git --repo-url https://chromium.googlesource.com/external/repo.git -g crosvm
|
||
|
|
$ repo sync -j4
|
||
|
|
$ cd src/platform/crosvm
|
||
|
|
```
|
||
|
|
|
||
|
|
### Installing Podman (or Docker)
|
||
|
|
|
||
|
|
See [Podman Installation](https://podman.io/getting-started/installation) for
|
||
|
|
instructions on how to install podman.
|
||
|
|
|
||
|
|
For Googlers, see [go/dont-install-docker](http://go/dont-install-docker) for
|
||
|
|
special instructions on how to set up podman.
|
||
|
|
|
||
|
|
If you already have docker installed, that will do as well. However podman is
|
||
|
|
recommended as it will not run containers with root privileges.
|
||
|
|
|
||
|
|
### Running all tests
|
||
|
|
|
||
|
|
To run all tests, just run:
|
||
|
|
|
||
|
|
```
|
||
|
|
./test_all
|
||
|
|
```
|
||
|
|
|
||
|
|
This will run all tests using the x86 and aarch64 builder containers. What does
|
||
|
|
this do?
|
||
|
|
|
||
|
|
1. It will start `./ci/[aarch64_]builder --vm`.
|
||
|
|
|
||
|
|
The builder will build ChromeOS dependencies from your local repo checkout.
|
||
|
|
If you make modifications to these dependencies (e.g. minijail, tpm2, cras)
|
||
|
|
these will be included in tests.
|
||
|
|
|
||
|
|
Then it will start a VM for running tests in the background. The VM is
|
||
|
|
booting while the next step is running.
|
||
|
|
|
||
|
|
2. Then it will call `./run_tests` inside the builder
|
||
|
|
|
||
|
|
The script will pick which tests to execute and where. Simple tests can be
|
||
|
|
executed directly, other tests require privileged access to devices and will
|
||
|
|
be loaded into the VM to execute.
|
||
|
|
|
||
|
|
Each test will in the end be executed by a call to
|
||
|
|
`cargo test -p crate_name`.
|
||
|
|
|
||
|
|
Intermediate build data is stored in a scratch directory at `./target/ci/` to
|
||
|
|
allow for faster subsequent calls (Note: If running with docker, these files
|
||
|
|
will be owned by root).
|
||
|
|
|
||
|
|
### Fast, iterative test runs
|
||
|
|
|
||
|
|
For faster iteration time, you can directly invoke some of these steps directly:
|
||
|
|
|
||
|
|
To only run x86 tests: `./ci/[aarch64_]builder --vm ./run_tests`.
|
||
|
|
|
||
|
|
To run a simple test (e.g. the tempfile crate) that does not need the vm:
|
||
|
|
`./ci/[aarch64_]builder cargo test -p tempfile`.
|
||
|
|
|
||
|
|
Or run a single test (e.g. kvm_sys) inside the vm:
|
||
|
|
`./ci/[aarch64*]builder --vm cargo test -p kvm_sys`.
|
||
|
|
|
||
|
|
Since the VM (especially the fully emulated aarch64 VM) can be slow to boot, you
|
||
|
|
can start an interactive shell and run commands from there as usual. All cargo
|
||
|
|
tests will be executed inside the VM, without the need to restart the VM between
|
||
|
|
test calls.
|
||
|
|
|
||
|
|
```
|
||
|
|
host$ ./ci/aarch64_builder --vm
|
||
|
|
crosvm-aarch64$ ./run_tests
|
||
|
|
crosvm-aarch64$ cargo test -p kvm_sys
|
||
|
|
...
|
||
|
|
```
|
||
|
|
|
||
|
|
### Reproducing Kokoro locally
|
||
|
|
|
||
|
|
Kokoro uses the same builders and the same `run_tests` script to run tests.
|
||
|
|
However, to keep the build stable, it syncs the chromiumos checkout to the fixed
|
||
|
|
manifest found at `./ci/kokoro/manifest.xml`.
|
||
|
|
|
||
|
|
To run tests using the same manifest, as well as the same build process that
|
||
|
|
Kokoro uses, you can run: `./ci/kokoro/simulate_all`.
|
||
|
|
|
||
|
|
## Implementation Overview
|
||
|
|
|
||
|
|
Directories:
|
||
|
|
|
||
|
|
- ci/build_environment: Contains tooling for building the dependencies of
|
||
|
|
crosvm.
|
||
|
|
- ci/crosvm_aarch64_builder: An x86 docker image to cross-compile for aarch64
|
||
|
|
and test with user-space emulation.
|
||
|
|
- ci/crosvm_base: Docker image shared by crosvm_builder and
|
||
|
|
crosvm_aarch64_builder
|
||
|
|
- ci/crosvm_builder: A native docker image for building and testing crosvm
|
||
|
|
- ci/crosvm_test_vm: Dockerfile to build the VM included in the builder
|
||
|
|
containers.
|
||
|
|
- ci/kokoro: Configuration files and build scripts used by Kokoro to run crosvm
|
||
|
|
tests.
|
||
|
|
|
||
|
|
Scripts:
|
||
|
|
|
||
|
|
- ci/aarch64_builder: Script to start the crosvm_aarch64_builder container
|
||
|
|
- ci/builder: Script to start the crosvm_builder container
|
||
|
|
- ci/run_container.sh: Implementation behind the above scripts.
|
||
|
|
- test_runner.py: Implementation behind the `./test_all` script.
|
||
|
|
|
||
|
|
### Building and uploading a new version of builders
|
||
|
|
|
||
|
|
The docker images for all builders can be built with `make` and uploaded with
|
||
|
|
`make upload`. Of course you need to have docker push permissions for
|
||
|
|
`gcr.io/crosvm-packages/` for the upload to work.
|