cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncThe VirtualBox Validation Kit
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync=============================
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncIntroduction
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync------------
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncThe VirtualBox Validation Kit is our new public tool for doing automated
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsynctesting of VirtualBox. We are continually working on adding new features
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncand guest operating systems to our battery of tests.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncWe warmly welcome contributions, new ideas for good tests and fixes.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsyncDirectory Layout
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync----------------
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The documentation for the test suite mostly lives here, the exception being
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync readme.txt files that are better off living near what they concern.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync For a definition of terms used here, see the Definitions / Glossary section
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync of ./docs/AutomaticTestingRevamp.txt / ./docs/AutomaticTestingRevamp.html.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync./testdriver/
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Python module implementing the base test drivers and supporting stuff.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The base test driver implementation is found in ./testdriver/base.py while
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync the VBox centric specialization is in ./testdriver/vbox.py. Various VBox
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync API wrappers that makes things easier to use and glosses over a lot of API
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync version differences that live in ./testdriver/vboxwrappers.py.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Test VM collections are often managed thru ./testdriver/vboxtestvms.py, but
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync doesn't necessarily have to be, it's up to the individual test driver.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync For logging, reporting result, uploading useful files and such we have a
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync reporter singleton sub-package, ./testdriver/reporter.py. It implements
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync both local (for local testing) and remote (for testboxes + test manager)
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync There is also a VBoxTXS client implementation in txsclient.py and a stacked
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync test driver for installing VBox (vboxinstaller.py). Most test drivers will
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync use the TXS client indirectly thru vbox.py methods. The installer driver
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync is a special trick for the testbox+testmanager setup.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The python scripts driving the tests. These are organized by what they
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync test and are all derived from the base classes in ./testdriver (mostly from
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync vbox.py of course). Most tests use one or more VMs from a standard set of
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync preconfigured VMs defined by ./testdriver/vboxtestvms.py (mentioned above),
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync though the installation tests used prepared ISOs and floppy images.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Text documents describing the preconfigured test VMs defined by
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync ./testdrive/vboxtestvms.py. This will also contain description of how to
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync prepare installation ISOs when we get around to it (soon).
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Test utilities and lower level test programs, compiled from C, C++ and
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Assembly mostly. Generally available for both host and guest, i.e. in the
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync zip and on the VBoxValidationKit.iso respectively.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The Test eXecution Service (VBoxTXS) found in ./utils/TestExecServ is one
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync of the more important utilities. It implements a remote execution service
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync for running programs/tests inside VMs and on other test boxes. See
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync ./utils/TestExecServ/vboxtxs-readme.txt for more details.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync A simple network bandwidth and latency test program can be found in
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync./bootsectors/
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Boot sector test environment. This allows creating floppy images in
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync assembly that tests specific CPU or device behaviour. Most tests can be
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync put on a USB stick, floppy or similar and booted up on real hardware for
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync comparison. All floppy images can be used for manual testing by developers
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync and most will be used by test drivers (./tests/*/td*.py) sooner or later.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The boot sector environment is heavily bound to yasm and it's ability to
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync link binary images for single assembly input units. There is a "library"
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync of standard initialization code and runtime code, which include switch to
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync all (well V8086 mode is still missing, but we'll get that done eventually)
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync processor modes and paging modes. The image specific code is split into
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync init/driver code and test template, the latter can be instantiated for each
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync process execution+paging mode.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Python package containing common python code.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync./testboxscript/
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The testbox script. This is installed on testboxes used for automatic
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync testing with the testmanager.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync./testmanager/
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync The VirtualBox Test Manager (server side code). This is written in Python
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync and currently uses postgresql as database backend for no particular reason
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync other than that it was already installed on the server the test manager was
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync going to run on. It's relatively generic, though there are of course
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync things in there that are of more use when testing VirtualBox than other
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync things. A more detailed account (though perhaps a little dated) of the
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync test manager can be found in ./docs/AutomaticTestingRevamp.txt and
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync./testanalysis/
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync A start a local test result analysis, comparing network test output. We'll
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync probably be picking this up again later.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync Various code snippets that may be turned into real tests at some point.
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync:Status: $Id$
cf22150eaeeb72431bf1cf65c309a431454fb22bvboxsync:Copyright: Copyright (C) 2010-2014 Oracle Corporation.