Automated Testing Framework (atf)

Copyright (c) 2007 The NetBSD Foundation, Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

.Dd November 1, 2010 .Dt ATF-RUN 1 .Os .Sh NAME .Nm atf-run .Nd executes a collection of test programs .Sh SYNOPSIS .Nm .Op Fl v Ar var1=value1 Op .. Fl v Ar varN=valueN .Op Ar test_program1 Op Ar .. test_programN .Nm .Fl h .Sh DESCRIPTION .Nm executes a collection of test programs or, in other words, a complete test suite. The results of each test program are collected by the tool, and are then multiplexed into a single machine-parseable report; see .Xr atf-formats 5 for more details. This report can later be transformed into many different and saner formats using the .Nm atf-report tool.

p The list of test programs to execute is read from an

a Atffile present in the current directory. This file describes the test suite stored in the directory it lives in, which aside from the list of test programs also includes meta-data and configuration variables.

p .Nm is also in charge of reading the configuration files that tune the behavior of each test program and passing down the necessary variables to them. More details on how this is done are given in the .Sx Configuration section.

p In the first synopsis form, .Nm parses the

a Atffile in the current directory and runs all the test programs specified in it. If any test program names are given as part of the command line, those are the ones executed instead of the complete list.

p In the second synopsis form, .Nm will print information about all supported options and their purpose.

p The following options are available: l -tag -width XvXvarXvalueXX t Fl h Shows a short summary of all available options and their purpose. t Fl v Ar var=value Sets the configuration variable .Ar var to the given value .Ar value . .El .Ss Configuration .Nm reads configuration data from multiple places. After all of these places have been analyzed, a list of variable-value pairs are passed to the test programs to be run.

p The following locations are scanned for configuration data, in order. Items down the list override values defined above them: l -enum t Configuration variables defined in the

a Atffile . t Configuration variables defined in the system-wide configuration file shared among all test suites. This lives in

a ${ATF_CONFDIR}/common.conf . t Configuration variables defined in the system-wide test-suite-specific configuration file. This lives in

a ${ATF_CONFDIR}/<test-suite>.conf . t Configuration variables defined in the user-specific configuration file shared among all test suites. This lives in

a ${HOME}/.atf/common.conf . t Configuration variables defined in the user-specific test-suite-specific configuration file. This lives in

a ${HOME}/.atf/<test-suite>.conf . t Configuration variables provided as part of the command line through the .Fl v option. .El

p The value of .Va ATF_CONFDIR in the above list determined as detailed in .Xr atf-config 1 .

p The following configuration variables are globally recognized: l -tag -width XunprivilegedXuserXX t Va unprivileged-user The name of the system user that atf-run will drop root privileges into for test cases defining .Sq require.user=unprivileged . Note that this is .Em not provided for security purposes ; this feature is only for the convenience of the user. .El .Ss Hooks .Nm Ns 's internal behavior can be customized by the system administrator and the user by means of hooks. These hooks are written in the shell script language for simplicity and are stored in the following files, which are read in the order provided below: l -enum t ${ATF_CONFDIR}/atf-run.hooks t ${HOME}/.atf/atf-run.hooks .El

p The following hooks are supported: l -tag -width infoXstartXhookXX t info_start_hook Called before .Nm executes any test program. The purpose of this hook is to write additional .Sq info stanzas to the top of the output report; these are defined by the .Sq application/X-atf-tps format described in .Xr atf-formats 5 . Always use the .Sq atf_tps_writer_info function to print these.

p This takes no parameters. t info_end_hook Similar to .Sq info_start_hook but executed after all test programs have been run so that additional .Sq info stanzas can be added to the bottom of the output report.

p This takes no parameters. .El

p All hooks are accompanied by a function named .Sq default_<hook_name> that can be executed by them to invoke the default behavior built into .Nm . For example, in order to extend the default .Sq info_start_hook hook, we could write the following function: d -literal -offset indent info_start_hook() { default_info_start_hook "${@}" atf_tps_writer_info "uptime" "$(uptime)" } .Ed .Sh SEE ALSO .Xr atf-report 1 , .Xr atf-test-program 1 , .Xr atf 7