VBoxValidationKitReadMe.txt revision c9e322b412e61d2e8a70a12e23ddad716cd85a3f
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart PoetteringThe VirtualBox Validation Kit
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering=============================
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart PoetteringThe VirtualBox Validation Kit is our new public tool for doing automated
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poetteringtesting of VirtualBox. We are continually working on adding new features
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poetteringand guest operating systems to our battery of tests.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart PoetteringWe warmly welcome contributions, new ideas for good tests and fixes.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart PoetteringDirectory Layout
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering----------------
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The documentation for the test suite mostly lives here, the exception being
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering readme.txt files that are better off living near what they concern.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering For a definition of terms used here, see the Definitions / Glossary section
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering of ./docs/AutomaticTestingRevamp.txt / ./docs/AutomaticTestingRevamp.html.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Python module implementing the base test drivers and supporting stuff.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The base test driver implementation is found in ./testdriver/base.py while
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering the VBox centric specialization is in ./testdriver/vbox.py. Various VBox
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering API wrappers that makes things easier to use and glosses over a lot of API
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering version differences that live in ./testdriver/vboxwrappers.py.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Test VM collections are often managed thru ./testdriver/vboxtestvms.py, but
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering doesn't necessarily have to be, it's up to the individual test driver.
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers For logging, reporting result, uploading useful files and such we have a
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering reporter singleton sub-package, ./testdriver/reporter.py. It implements
c978343015c787713651dff571acb5207367f5f2Lennart Poettering both local (for local testing) and remote (for testboxes + test manager)
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering There is also a VBoxTXS client implementation in txsclient.py and a stacked
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering test driver for installing VBox (vboxinstaller.py). Most test drivers will
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering use the TXS client indirectly thru vbox.py methods. The installer driver
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering is a special trick for the testbox+testmanager setup.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The python scripts driving the tests. These are organized by what they
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering test and are all derived from the base classes in ./testdriver (mostly from
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering vbox.py of course). Most tests use one or more VMs from a standard set of
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering preconfigured VMs defined by ./testdriver/vboxtestvms.py (mentioned above),
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering though the installation tests used prepared ISOs and floppy images.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Text documents describing the preconfigured test VMs defined by
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering ./testdrive/vboxtestvms.py. This will also contain description of how to
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering prepare installation ISOs when we get around to it (soon).
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Test utilities and lower level test programs, compiled from C, C++ and
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Assembly mostly. Generally available for both host and guest, i.e. in the
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering zip and on the VBoxValidationKit.iso respectively.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The Test eXecution Service (VBoxTXS) found in ./utils/TestExecServ is one
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering of the more important utilities. It implements a remote execution service
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering for running programs/tests inside VMs and on other test boxes. See
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering ./utils/TestExecServ/vboxtxs-readme.txt for more details.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering A simple network bandwidth and latency test program can be found in
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering Boot sector test environment. This allows creating floppy images in
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering assembly that tests specific CPU or device behaviour. Most tests can be
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering put on a USB stick, floppy or similar and booted up on real hardware for
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering comparison. All floppy images can be used for manual testing by developers
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering and most will be used by test drivers (./tests/*/td*.py) sooner or later.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The boot sector environment is heavily bound to yasm and it's ability to
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering link binary images for single assembly input units. There is a "library"
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering of standard initialization code and runtime code, which include switch to
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering all (well V8086 mode is still missing, but we'll get that done eventually)
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering processor modes and paging modes. The image specific code is split into
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering init/driver code and test template, the latter can be instantiated for each
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering process execution+paging mode.
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers Python package containing common python code.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering./testboxscript/
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The testbox script. This is installed on testboxes used for automatic
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers testing with the testmanager.
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers./testmanager/
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering The VirtualBox Test Manager (server side code). This is written in Python
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering and currently uses postgresql as database backend for no particular reason
599659860c770058f2eb04d578c521c16e0b1853Lennart Poettering other than that it was already installed on the server the test manager was
599659860c770058f2eb04d578c521c16e0b1853Lennart Poettering going to run on. It's relatively generic, though there are of course
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering things in there that are of more use when testing VirtualBox than other
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering things. A more detailed account (though perhaps a little dated) of the
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering test manager can be found in ./docs/AutomaticTestingRevamp.txt and
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers./testanalysis/
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers A start a local test result analysis, comparing network test output. We'll
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering probably be picking this up again later.
f18ca9dcdeda247e208f7143e834fd2fb2070d80Kay Sievers Various code snippets that may be turned into real tests at some point.
6d0274f11547a0f11200bb82bf598a5a253e12cfLennart Poettering:Copyright: Copyright (C) 2010-2014 Oracle Corporation.