NEWS revision ef421f66f47224a42073deaf087378c5d0c9952e
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsMajor changes between releases Automated Testing Framework
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews===========================================================================
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.12
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews***********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on November 7th, 2010.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the ATF_REQUIRE_THROW_RE to atf-c++, which is the same as
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews ATF_REQUIRE_THROW but allows checking for the validity of the exception's
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews error message by means of a regular expression.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the ATF_REQUIRE_MATCH to atf-c++, which allows checking for a
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews regular expression match in a string.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Changed the default timeout for test cases from 5 minutes to 30 seconds.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews 30 seconds is long enough for virtually all tests to complete, and 5
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews minutes is a way too long pause in a test suite where a single test case
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Deprecated the use.fs property. While this seemed like a good idea in
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the first place to impose more control on what test cases can do, it
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews turns out to be bad. First, use.fs=false prevents bogus test cases
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews from dumping core so after-the-fact debugging is harder. Second,
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews supporting use.fs adds a lot of unnecessary complexity. atf-run will
dbb012765c735ee0d82dedb116cdc7cf18957814Evan Hunt now ignore any value provided to use.fs and will allow test cases to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews freely access the file system if they wish to.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the atf_tc_get_config_var_as_{bool,long}{,_wd} functions to the atf-c
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews library. The 'text' module became private in 0.11 but was being used
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews externally to simplify the parsing of configuration variables.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Made atf-run recognize the 'unprivileged-user' configuration variable
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews and automatically drop root privileges when a test case sets
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews require.user=unprivileged. Note that this is, by no means, done for
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews security purposes; this is just for user convenience; tests should, in
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews general, not be blindly run as root in the first place.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.11
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews***********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on October 20th, 2010.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* The ATF_CHECK* macros in atf-c++ were renamed to ATF_REQUIRE* to match
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews their counterparts in atf-c.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Clearly separated the modules in atf-c that are supposed to be public
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews from those that are implementation details. The header files for the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews internal modules are not installed any more.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Made the atf-check tool private. It is only required by atf-sh and being
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews public has the danger of causing confusion. Also, making it private
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews simplifies the public API of atf.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Changed atf-sh to enable per-command error checking (set -e) by default.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews This catches many cases in which a test case is broken but it is not
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews reported as such because execution continues.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed the XSTL and CSS stylesheets to support expected failures.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.10
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews***********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on July 2nd, 2010.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsMiscellaneous features
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added expected failures support to test cases and atf-run. These
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews include, for example, expected clean exits, expected reception of fatal
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews signals, expected timeouts and expected errors in condition checks.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews These statuses can be used to denote test cases that are known to fail
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews due to a bug in the code they are testing. atf-report reports these
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews tests separately but they do not count towards the failed test cases
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews allow easy checking of call failures that update errno.
ba751492fcc4f161a18b983d4f018a1a52938cb9Evan Hunt* Added the has.cleanup meta-data property to test caes that specifies
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews whether the test case has a cleanup routine or not; its value is
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews automatically set. This property is read by atf-run to know if it has to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews run the cleanup routine; skipping this run for every test case
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews significantly speeds up the run time of test suites.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews take the expected exception as the first argument and the statement to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews execute as the second argument.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in atf-check
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Changed atf-check to support negating the status and output checks by
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews prefixing them with not- and added support to specify multiple checkers
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews for stdout and stderr, not only one.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the match output checker to atf-check to look for regular
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews expressions in the stdout and stderr of commands.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Modified the exit checks in atf-check to support checking for the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews reception of signals.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsCode simplifications and cleanups
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed usage messages from test programs to simplify the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews implementation of every binding by a significant amount. They just now
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews refer the user to the appropriate manual page and do not attempt to wrap
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews lines on terminal boundaries. Test programs are not supposed to be run
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews by users directly so this minor interface regression is not important.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed the atf-format internal utility, which is unused after the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews change documented above.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed the atf-cleanup internal utility. It has been unused since the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews test case isolation was moved to atf-run in 0.8
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Splitted the Makefile.am into smaller files for easier maintenance and
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews dropped the use of M4. Only affects users building from the repository
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Intermixed tests with the source files in the source tree to provide
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews them more visibility and easier access. The tests directory is gone from
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the source tree and tests are now suffixed by _test, not prefixed by t_.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Simplifications to the atf-c library: removed the io, tcr and ui
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews modules as they had become unnecessary after all simplifications
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews introduced since the 0.8 release.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed the application/X-atf-tcr format introduced in 0.8 release.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews Tests now print a much simplified format that is easy to parse and nicer
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews to read by end users. As a side effect, the default for test cases is
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews now to print their results to stdout unless otherwise stated by providing
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the -r flag.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed XML distribution documents and replaced them with plain-text
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews documents. They provided little value and introduced a lot of complexity
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews to the build system.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Simplified the output of atf-version by not attempting to print a
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews revision number when building form a distfile. Makes the build system
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews easier to maintain.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.9
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on June 3rd, 2010.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added atf-sh, an interpreter to process test programs written using
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the shell API. This is not really a shell interpreter by itself though:
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews it is just a wrapper around the system shell that eases the loading of
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the necessary ATF libraries.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed atf-compile in favour of atf-sh.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the use.fs metadata property to test case, which is used to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews specify which test cases require file system access. This is to
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt highlight dependencies on external resources more clearly and to speed up
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt the execution of test suites by skipping the creation of many unnecessary
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews work directories.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed test programs to get a sane default value for their source
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews directory. This means that it should not be necessary any more to pass
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews -s when running test programs that do not live in the current directory.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Defining test case headers became optional. This is trivial to achieve
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews in shell-based tests but a bit ugly in C and C++. In C, use the new
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews ATF_TC_WITHOUT_HEAD macro to define the test case, and in C++ use
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt ATF_TEST_CASE_WITHOUT_HEAD.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan HuntChanges in version 0.8
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt**********************
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan HuntExperimental version released on May 7th, 2010.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Test programs no longer run several test cases in a row. The execution
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt of a test program now requires a test case name, and that single test
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt case is executed. To execute several test cases, use the atf-run utility
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs no longer fork a subprocess to isolate the execution of
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews test cases. They run the test case code in-process, and a crash of the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews test case will result in a crash of the test program. This is to ease
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews debugging of faulty test cases.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs no longer isolate their test cases. This means that they
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews will not create temporary directories nor sanitize the environment any
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews more. Yes: running a test case that depends on system state by hand will
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews most likely yield different results depending on where (machine,
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews directory, user environment, etc.) it is run. Isolation has been moved
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs no longer print a cryptic format (application/X-atf-tcs)
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews on a special file channel. They can now print whatever they want on the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews screen. Because test programs can now only run one test case every time,
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews providing controlled output is not necessary any more.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs no longer write their status into a special file
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews descriptor. Instead, they create a file with the results, which is later
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews parsed by atf-run. This changes the semantics of the -r flag.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* atf-run has been adjusted to perform the test case isolation. As a
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews result, there is now a single canonical place that implements the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews isolation of test caes. In previous releases, the three language
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews bindings (C, C++ and shell) had to be kept in sync with each other (read:
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews not a nice thing to do at all). As a side effect of this change, writing
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews bindings for other languages will be much, much easier from now on.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* atf-run forks test programs on a test case basis, instead of on a test
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews program basis as it did before. This is to provide the test case
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews isolation that was before implemented by the test programs themselves.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Removed the atf-exec tool. This was used to implement test case
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews isolation in atf-sh, but it is now unnecessary.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* It is now optional to define the descr meta-data property. It has been
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews proven to be mostly useless, because test cases often carry a descriptive
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews name of their own.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.7
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on December 22nd, 2009.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added build-time checks to atf-c and atf-c++. A binding for atf-sh
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews will come later.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Migrated all build-time checks for header files to proper ATF tests.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews This demonstrates the use of the new feature described above.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added an internal API for child process management.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Converted all plain-text distribution documents to a Docbook canonical
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews version, and include pre-generated plain text and HTML copies in the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews distribution file.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Simplified the contents of the Makefile.am by regenerating it from a
ba751492fcc4f161a18b983d4f018a1a52938cb9Evan Hunt canonical Makefile.am.m4 source. As a side-effect, some dependency
ba751492fcc4f161a18b983d4f018a1a52938cb9Evan Hunt specifications were fixed.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Migrated all checks from the check target to installcheck, as these
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews require ATF to be installed.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed sign comparison mismatches triggered by the now-enabled
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews -Wsign-compare.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed many memory and object leaks.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.6
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on January 18th, 2009.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Make atf-exec be able to kill its child process after a certain period
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews of time; this is controlled through the new -t option.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Change atf-sh to use atf-exec's -t option to control the test case's
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews timeouts, instead of doing it internally. Same behavior as before, but
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews noticeably faster.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* atf-exec's -g option and atf-killpg are gone due to the previous
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the atf-check(1) tool, a program that executes a given command
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews and checks its exit code against a known value and allows the management
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews of stdout and stderr in multiple ways. This replaces the previous
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews atf_check function in the atf-sh library and exposes this functionality
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews to both atf-c and atf-c++.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the ATF_REQUIRE family of macros to the C interface. These help
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews in checking for fatal test conditions. The old ATF_CHECK macros now
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews perform non-fatal checks only. I.e. by using ATF_CHECK, the test case
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews can now continue its execution and the failures will not be reported
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews until the end of the whole run.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Extended the amount of ATF_CHECK_* C macros with new ones to provide
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews more features to the developer. These also have their corresponding
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews counterparts in the ATF_REQUIRE_* family. The new macros (listing the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews suffixes only) are: _EQ (replaces _EQUAL), _EQ_MSG, _STREQ and
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.5
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on May 1st, 2008.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Clauses 3 and 4 of the BSD license used by the project were dropped.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews All the code is now under a 2-clause BSD license compatible with the GNU
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews General Public License (GPL).
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added a C-only binding so that binary test programs do not need to be
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews tied to C++ at all. This binding is now known as the atf-c library.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Renamed the C++ binding to atf-c++ for consistency with the new atf-c.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Renamed the POSIX shell binding to atf-sh for consistency with the new
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews atf-c and atf-c++.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added a -w flag to test programs through which it is possible to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews specify the work directory to be used. This was possible in prior
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews releases by defining the workdir configuration variable (-v workdir=...),
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews but was a conceptually incorrect mechanism.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs now preserve the execution order of test cases when they
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews are given in the command line. Even those mentioned more than once are
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews executed multiple times to comply with the user's requests.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.4
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan HuntExperimental version released on February 4th, 2008.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added two new manual pages, atf-c++-api and atf-sh-api, describing the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews C++ and POSIX shell interfaces used to write test programs.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added a pkg-config file, useful to get the flags to build against the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews C++ library or to easily detect the presence of ATF.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added a way for test cases to require a specific architecture and/or
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews machine type through the new 'require.arch' and 'require.machine'
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt meta-data properties, respectively.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Added the 'timeout' property to test cases, useful to set an
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt upper-bound limit for the test's run time and thus prevent global test
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt program stalls due to the test case's misbehavior.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Added the atf-exec(1) internal utility, used to execute a command
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt after changing the process group it belongs to.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Added the atf-killpg(1) internal utility, used to kill process groups.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Multiple portability fixes. Of special interest, full support for
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews SunOS (Solaris Express Developer Edition 2007/09) using the Sun Studio 12
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews C++ compiler.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed a serious bug that prevented atf-run(1) from working at all
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews under Fedora 8 x86_64. Due to the nature of the bug, other platforms
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews were likely affected too.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.3
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan HuntExperimental version released on November 11th, 2007.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added XML output support to atf-report. This is accompanied by a DTD
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews for the format's structure and sample XSLT/CSS files to post-process this
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews output and convert it to a plain HTML report.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Changed atf-run to add system information to the report it generates.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews This is currently used by atf-report's XML output only, and is later
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews printed in the HTML reports in a nice and useful summary table. The user
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews and system administrator are allowed to tune this feature by means of
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Removed the test cases' 'isolated' property. This was intended to
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews avoid touching the file system at all when running the related test case,
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews but this has not been true for a long while: some control files are
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt unconditionally required for several purposes, and we cannot easily get
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews rid of them. This way we remove several critical and delicate pieces of
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Improved atf-report's CSV output format to include information about
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt test programs too.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Fixed the tests that used atf-compile to not require this tool as a
12bf5d4796505b4c20680531da96a31e6c2c1144Evan Hunt helper. Avoids systems without build-time utilities to skip many tests
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt that could otherwise be run. (E.g. NetBSD without the comp.tgz set
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Many general cleanups: Fixed many pieces of code marked as ugly and/or
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.2
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on September 20th, 2007.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* Test cases now get a known umask on entry.
98922b2b2b024dcca25be7c220cf3b16b1e6c4b5Evan Hunt* atf-run now detects many unexpected failures caused by test programs and
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews reports them as bogus tests. atf-report is able to handle these new
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews errors and nicely reports them to the user.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* All the data formats read and written by the tools have been
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews documented and cleaned up. These include those grammars that define how
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the different components communicate with each other as well as the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews format of files written by the developers and users: the Atffiles and the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews configuration files.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Added the atf-version tool, a utility that displays information about
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews the currently installed version of ATF.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test cases can now define an optional cleanup routine to undo their
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews actions regardless of their exit status.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* atf-report now summarizes the list of failed (bogus) test programs
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews when using the ticker output format.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Test programs now capture some termination signals and clean up any
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews temporary files before exiting the program.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* Multiple bug fixes and improvements all around.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsChanges in version 0.1
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews**********************
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark AndrewsExperimental version released on August 20th, 2007.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews* First public version. This was released coinciding with the end of the
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews Google Summer of Code 2007 program.
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrews===========================================================================
c3c8823fed039b3a2b8e5ca8bc2f3301d1dd840eMark Andrewsvim: filetype=text:textwidth=75:expandtab:shiftwidth=2:softtabstop=2