What is LAVA?¶
LAVA is the Linaro Automation and Validation Architecture.
LAVA is a continuous integration system for deploying operating systems onto physical and virtual hardware for running tests. Tests can be simple boot testing, bootloader testing and system level testing, although extra hardware may be required for some system tests. Results are tracked over time and data can be exported for further analysis.
LAVA is a collection of participating components in an evolving architecture. LAVA aims to make systematic, automatic and manual quality control more approachable for projects of all sizes.
LAVA is designed for validation during development - testing whether the code that engineers are producing “works”, in whatever sense that means. Depending on context, this could be many things, for example:
- testing whether changes in the Linux kernel compile and boot
- testing whether the code produced by gcc is smaller or faster
- testing whether a kernel scheduler change reduces power consumption for a certain workload
LAVA is good for automated validation. LAVA tests the Linux kernel on a range of supported boards every day. LAVA tests proposed android changes in gerrit before they are landed, and does the same for other projects like gcc. Linaro runs a central validation lab in Cambridge, containing racks full of computers supplied by Linaro members and the necessary infrastucture to control them (servers, serial console servers, network switches etc.)
LAVA is good for providing developers with the ability to run customised test on a variety of different types of hardware, some of which may be difficult to obtain or integrate. Although LAVA has support for emulation (based on QEMU), LAVA is best at providing test support for real hardware devices.
LAVA is principally aimed at testing changes made by developers across multiple hardware platforms to aid portability and encourage multi-platform development. Systems which are already platform independent or which have been optimised for production may not necessarily be able to be tested in LAVA or may provide no overall gain.
This overview document explains LAVA using
http://validation.linaro.org/ which is the official production instance of
LAVA hosted by Linaro. Where examples reference
replace with the fully qualified domain name of your LAVA instance.
What is LAVA not?¶
- LAVA is not a set of tests - it is infrastructure to enable users to run their own tests. LAVA concentrates on providing a range of deployment methods and a range of boot methods. Once the login is complete, the test consists of whatever scripts the test writer chooses to execute in that environment.
- LAVA is not a test lab - it is the software that can used in a test lab to control test devices.
- LAVA is not a complete CI system - it is software that can form part of a CI loop. LAVA supports data extraction to make it easier to produce a frontend which is directly relevant to particular groups of developers.
- LAVA is not a build farm - other tools need to be used to prepare binaries which can be passed to the device using LAVA.
- LAVA is not a production test environment for hardware - LAVA is focused on developers and may require changes to the device or the software to enable automation. These changes are often unsuitable for production units. LAVA also expects that most devices will remain available for repeated testing rather than testing the software with a changing set of hardware.
LAVA is currently in the middle of a lengthy migration from its original design (known as the V1 model) to a new design, called the Pipeline model or the V2 model. During this migration, LAVA installations will be able to support test devices and test jobs targeting both models. These help pages are divided into V1 and V2 accordingly.
While this migration is taking place, it is expected that some LAVA instances will only support V2, some will support both versions and some may only support V1. However, V1 support is deprecated and will be removed at some point in 2017. Instances which do not migrate to V2 will not be able to receive updates beyond that point so users are strongly encouraged to move to V2 as soon as possible.
Please subscribe to the Mailing lists for information and support.
V1 refers to the components of LAVA which are related to:
- JSON job submissions
- Bundles, BundleStreams and the
- Image Reports and Image Reports 2.0
All code supporting V1 is deprecated as of the 2016.2 release and is scheduled to be removed from the codebase during 2017.
When the code objects implementing V1 are removed, the corresponding database records, tables, indexes and relationships will be deleted during later upgrades. Instances which want to continue using V1 from 2017 onwards must not install updates or all V1 data will be lost.
V2 refers to the pipeline model - a new design for how the test job is constructed. It gives test writers much more freedom to write new test jobs using new protocols and test methods, and it also delivers a much simpler way of deploying distributed instances.
- YAML job submissions, supporting comments
- Results, Queries and Charts
- Live result reporting (no final submission stage)
- Simplified setup for distributed instances
The code supporting V2 is being extended to support a wider range of devices and deployment methods. The migration to V2 is expected to last until the end of 2016.
LAVA is free software and is provided “as is” without warranty of any kind. Support is offered using the methods below and we will try to help resolve queries.
Whenever you look for support for LAVA, there are some guidelines to follow:
- If you are having problems, it may be helpful to check the mailing list archives first - somebody else may have already seen and solved a similar problem.
- Avoid putting LAVA job output directly into your email to a list or IRC channel. Mailing list posts can include a few lines but not IRC.
- If you are using LAVA V1, rather than LAVA V2, you need to
always re-run your test with
logging_level: DEBUGif you have not done so already.
- Always use a pastebin for log output, and include a link to the paste in your post. Pastebin services are provided online by multiple people. Some are open to anyone, such as pastebin.com and paste.debian.net. Others (like the internal Linaro pastebin) are restricted and will require users to register. Pastes will typically expire automatically, depending on the options selected by the user creating the paste.
- Paste content from the complete log, not the summary, so that you get the complete lines.
- Include the job definition you used, either in this paste or another paste.
- If you are administering your own instance, also include the device type template and an export of the device dictionary.
- Provide details of which server you are using (with a URL if it is publicly visible or a version string from the documentation pages if not) and details of the actual device(s) in use.
The primary method for support for LAVA is our mailing lists.
A few guidelines apply to all such lists:
- Reply to the list, adding the submitter in CC where appropriate.
- If your job uses URLs which are not visible to the rest of the list, include a rough outline of how those were built and what versions of tools were used.
- Avoid top posting.
- Always provide as much context as you can when phrasing your question to the list.
The lava-users mailing list concentrates on support for current LAVA tests, involving test writers, individual admins and LAVA developers. Users are encouraged to contribute to answer queries from other users.
lava-devel is an alias to linaro-validation
and is aimed at supporting test writers and admins who are adapting to the
LAVA V2 support and discussions relating to announcements from the
LAVA developers. Replies to the lava-announce list are directed here.
Subscribing to the lava-announce list is recommended for everyone using LAVA, whether writing tests or viewing reports or administering a LAVA instance.
As the work on LAVA V2 continues, it will become increasingly important that all users of LAVA are aware of the upcoming changes, new methods available in the refactoring and the removal of old methods.
Replies to this list are sent to the lava-devel list - if you are
not subscribed to
lava-devel, please ask other users to CC you on
IRC is a common
support method for developers. Our team is spread geographically
around the world, with some members in Europe, America and Asia. We
are usually talking on our IRC channel
Guidelines apply to IRC as well:
- Use a proxy or other service which keeps you connected to IRC. Developers are based in multiple timezones and not everyone can answer all queries. Therefore, you may have to wait several hours until the relevant person or people are awake. Check back for replies on the channel intermittently. If you disconnect, you will not see any replies sent whilst you were disconnected from the channel.
- Ask your question, do not wait to see people joining or talking. Don’t ask if you may ask a question!
- As with mailing lists, it is even more important with IRC that you always use a pastebin. See Guidelines.
- Do not assume that the person someone else spoke to last is also able to answer your question.
- Do not assume that the person you spoke to last is also able to answer your other question(s).
- Reply directly to a person by putting their IRC nickname at the start of your message to the channel. In a busy channel, it can be hard to spot replies not made to you.
- Developers are busy - IRC is part of our development process, so please be considerate of the amount of time involved, there is code to write and there are bug fixes to make for other users as well.
- Avoid personal messages unless there is a clear privacy issue involved or you know the person well.
- You may well find that one of the Mailing lists actually provides a faster answer to your question, especially if you are new to LAVA.