7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# This file and its contents are supplied under the terms of the
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# Common Development and Distribution License ("CDDL"), version 1.0.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# You may only use this file in accordance with the terms of version
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# 1.0 of the CDDL.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# A full copy of the text of the CDDL should have accompanied this
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# source. A copy of the CDDL is also available via the Internet at
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy# Copyright (c) 2013 by Delphix. All rights reserved.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyillumos Testing README
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy1. A Brief History of usr/src/test
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy2. How to Run These Tests
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy3. How to Develop New Tests
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy4. Porting Tests from Other Frameworks
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy--------------------------------------------------------------------------------
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy1. A Brief History of usr/src/test
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyThe tests here come in two varieties - tests written from scratch, and tests
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedythat have been ported from the Solaris Test Collection. Not all of the STC
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedytests have been ported, and a forked repository of those that have been made
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedypublicly available may currently be found here:
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy https://bitbucket.org/illumos/illumos-stc/
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyRegardless of origin, all of these tests are executed using the run(1) script
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedydescribed in the next section.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy2. How to Run These Tests
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyCurrently, all the test suites under usr/src/test provide a wrapper script
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyaround run(1). These wrappers allow environment variables to be set up which
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedymay (for example) allow the script to specify which disks may be used by a test
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedysuite, and which must be preserved. Additionally, the wrappers allow options to
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyrun(1) to be passed through so that a user may specify a custom configuration
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyfile for a test suite. For specifics on the options available in the framework
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyitself, please see the run(1) manpage.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy3. How to Develop New Tests
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyNew tests should mimic the directory layout of existing tests to the degree
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedypossible. This includes the following directories:
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedycmd - Any support binaries or scripts used by the tests in this package.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedydoc - READMEs or other support documentation to be delivered with the package.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyrunfiles - Configuration files that dictate how the tests are run.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedytests - The tests themselves (see below).
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyThe tests you create will be run, and given a PASS or FAIL status in accordance
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedywith the exit value returned by the test. A test may also be marked SKIPPED in
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedythe event that a prerequisite test is marked FAIL, or marked KILLED in the
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyevent the test times out. Note that there is no way to force a test to be
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedymarked SKIPPED; this is intentional. If a test must be skipped due to
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyinsufficient resources for example, then a wrapper script should be provided
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedythat chooses or creates an appropriate configuration file. The goal of every
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyrun is that every test is marked PASS.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy4. Porting Tests from Other Frameworks
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedySTF (Solaris Test Framework)
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyPorting tests from this framework is relatively straightforward. For the most
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedypart the tests can be arranged in the new directory structure and added to the
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyconfiguration file. The template for a configuration file can easily be created
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyusing the -w option to run(1). There are a few other changes that may be
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedySome STF tests consume the values of variables from the user's environment.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyThese variables must be set before beginning the test run either manually, or
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyvia a wrapper script.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyGroups of tests in STF can automatically source .cfg files specified in the
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedy'stf_description' file that exists in an STF directory. Ported tests that
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyrequire any of these variables must manually source the .cfg file instead.
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren KennedyThe configuration file of a newly ported test must specify the user the test
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyshould run as, along with the timeout value. In STF, these are also specified
7b1753e64ab6c8c68e7cd79acbedfdfba12bd7f8John Wren Kennedyin the 'stf_description' file.