=========================================================================== 5.7 Fixed: TESTNG-169 Error message: is depending on nonexistent method null ("null" is uninformative) Fixed: -listener takes comma-separated classes Added: RetryAnalyzer (experimental) (Jeremie) =========================================================================== 5.6 Added: SkipException/TimeBombedSkipException for manual skipping Added: can now be disabled at xml level using Added: Suite files that only contain other suites do not get reported Fixed: @BeforeClass methods would incorrectly report cyclic graphs Added: get/setAttribute to ITestContext Added: plugging in factory objects to handle the actual instantiation of tests Added: dataProvider to @Factory Added: ISuite now gives access to the current XmlSuite Fixed: TESTNG-139 dependsOnMethods gets confused when dependency is "protected" Fixed: TESTNG-141 junit attribute set to false in testng-failed.xml when it should be true Fixed: TESTNG-142 Exceptions in DataProvider are not reported as failed test Added: Improved behavior for @Before/@AfterClass when using @Factory (http://forums.opensymphony.com/thread.jspa?threadID=6594&messageID=122294#122294) Added: Support for concurrent execution for invocationCount=1 threadPoolSize>1 and @DataProvider (http://forums.opensymphony.com/thread.jspa?threadID=64738&tstart=0) Added: New TestNG specific XML report, generated by default in 'xml' subdirectory of test-output Added: support in strprotocol for passing the ITest.getTestName() information Fixed: TESTNG-152 If DataProvider is not found, the exception message should tell exactly what happened Eclipse plug-in Fixed: Bug that made group launch configurations unusable Fixed: The plugin doesn't create the correct launch configuration for @Factory Fixed: Method based launchers cannot be editted Fixed: Plugin hangs while executing test with dataprovider that sends \n, \r messages Added: display ITest.getTestName() IDEA plug-in Fixed: IDEA 7.0 compatibility Fixed: occasional 'illegal arguments exception' Fixed: TESTNG-151 Final passing test result is not properly hidden Added: Auto-completion for dependsOnMethods Added: Highlighting of invalid groups/methods in dependsOn* =========================================================================== 5.5 Fixed: @BeforeGroup methods were run twice when in a base class Fixed: @BeforeGroup methods were run twice with a @Test at class level Fixed: parallel="tests" didn't work as advertised Added: Support for thread-count at test level Added: Method selectors receive a Context and can stop the chain with setStopped() Fixed: XmlMethodSelector was always run first regardless of its priority Added: @BeforeGroups/@AfterGroups can live in classes without @Test methods Added: DataProvider can now take an ITestContext parameter Fixed: Wasn't parsing correctly Fixed: Annotation Transformers now work on class-level annotations Fixed: Some class-level @Test attributes were not always honored Added: Clean separation between @Test invocation events and @Configuration invocation events (see also TESTNG-111) Added: Test instances created by @Factory now run in multiple threads in parallel mode Fixed: @Before/@AfterGroups invocation order Fixed: TESTNG-27: Parameters are not used on level anymore Fixed: TESTNG-107 don't create an output directory if "outputDirectory" is null Fixed: TESTNG-127 UseDefaultListeners in Ant Task does not work Fixed: TESTNG-119 Running TestNG runner with invalid '-sourcedir' on JDK14 JavaDoc annotated test classes won't fail. Fixed: TESTNG-113 Dependent methods within the same static inner class are not found Fixed: TESTNG-125 TestNG failed for test classes under *.java*.* pakages Eclipse plug-in Fixed: issue with launch configuration Fixed: TESTNG-124: setting location of testng reports output =========================================================================== 5.4 Fixed: Ant task issue with paths containing spaces Added: for @BeforeGroups and @AfterGroups specifying the groups() attribute will auto-include the method into those groups by default (previously you had to also provide the value() attribute). Added: the load @Tests (invocationCount + threadPoolSize) are triggered simultaneous Fixed: reports are correctly displaying the thread info Added: @DataProvider name defaults to method name Added: support for remote protocol to pass parameter information Fixed: TextReporter logs information about the parameters of the test methods Fixed: concurrency issue in JUnitXMLReporter Fixed: output of JUnitXMLReporter must be CDATA Fixed: XML unsupported annotations/parallel attribute values are reported Eclipse plug-in Fixed: groups with multi-attribute javadoc annotations Fixed: consistent behavior for dependsOnMethods Fixed: consistent behavior for tests with dependsOnGroups (a warning is emitted) Fixed: consistent merge of configuration arguments when an existing launch configuration exists =========================================================================== 5.3 Fixed: use a single instance of bsh.Interpreter Added: @Before/@AfterMethod can declare a java.lang.reflect.Method parameter to be informed about the @Test method Fixed: super classes must not be listed in testng-failures.xml Fixed: parallel attribute must not appear if empty or null in testng-failures.xml Fixed: parsing for javadoc annotations is done on request only Added: improved multiple suite summary page report Added: -target option deprecated in favor of -annotations javadoc|jdk Fixed: filesets in the ant task didn't work if the paths have spaces in them Fixed: Before/After Suite were behaving wrong in parallel execution Added: A generic/extensible RemoteTestNG was added to the core Fixed: Before/AfterGroup-s were behaving wrong when using invocationCount, dataProvider and threadPoolSize Fixed: improved support for running different annotation type tests in the same suite Fixed: testng-failed.xml was generated even if there were no failures/skipps Fixed: -usedefaultlisteners was wrongly passed to JVM instead of TestNG options Added: Attribute dataProviderClass for @Test and @testng.test Fixed: Forgot to account for cases where both invocationCount and DataProviders are present Fixed: AfterGroups were invoked out of order with invocationCount and DataProviders Fixed: Reporter.getOutput() returned an empty array if a timeOut was specified Added: testng.xml now supports Added: ant task can receive several listeners Fixed: TESTNG-109 Skipped tests with expected exceptions are reported as failures Added: ant task can now select the parallel mode for running tests Fixed: ant task correctly deals with empty groups and excludedgroups parameters Added: ant task can override default suite and test names Added: comand line support for setting parallel mode, suite and test names Eclipse plug-in Added: Support for configuring per project usedefaultlisteners Added: Contextual drop-down menu on failures tab of the TestNG view to enable running/debugging method failure only Added: Suppport for configuring per project TestNG jar usage (project provided one or plugin provided one) =========================================================================== 5.2 Added: "-usedefaultlisteners true/false" to command line and ant Added: EmailableReporter (from Paul Mendelson) Added: parallel can now be "methods" or "tests". Boolean version deprecated Added: TestNGAntTask now uses the @ syntax to invoke TestNG Added: Command line understands @ syntax Added: JUnitConverter uses the new syntax Added: -groups to JUnitConverter Fixed: Throw proper exception when a DataProvider declares parameters Added: completely revamped JUnit support (should run all kind of JUnit tests) Fixed: TESTNG-40 (Bug in testng-failed.xml generation) Fixed: TESTNG-106 (Failed "@BeforeSuite" method just skipps the last test in xml-file) Fixed: Success on 0 tests (http://forums.opensymphony.com/thread.jspa?threadID=41213) Eclipse plug-in Added: TESTNG-105 Automaticaly define TESTNG_HOME classpath variable =========================================================================== 5.1 Added: @Test(sequential = true) Fixed: TESTNG-102 (Incorrect ordering of @BeforeMethod calls when a dependency is specified) Fixed: TESTNG-101 (HTML output contains nested

tags and a missing tag) Added: support for specifying test-only classpath (http://forums.opensymphony.com/thread.jspa?messageID=78048&tstart=0) Fixed: TESTNG-93 (method selectors filtering @BeforeMethod) Fixed: TESTNG-81 (Assert.assertFalse() displays wrong expected, actual value) Fixed: TESTNG-59 (multiple method selectors usage results in no tests run) Fixed: TESTNG-56 (invocation of @Before/AfterClass methods in parallel/sequential scenarios) Fixed: TESTNG-40 (failures suite does not contain @Before/After Suite/Test methods) Fixed: TESTNG-37 (allow passing null parameter value from testng.xml) Fixed: TESTNG-7 (display classname when hovering method) Eclipse plug-in Added: run contextual test classes with parameters from suite definition files Added: TESTNG-100 (Show HTML reports after running tests) Added: TESTNG-97 (Double click top stack to raise comparison) Added: TESTNG-84 (plug-in UI for suite option should support absolute path) Added: TESTNG-20 (copy stack trace) Fixed: TESTNG-72 (display groups with non-array values) Fixed: TESTNG-64 (Eclipse plug-in applies added groups to all launch configurations) Fixed: TESTNG-28 (Cannot select groups from dependent eclipse projects) Fixed: TESTNG-25 (do not display fully qualified method name when running contextual test class) Improved behavior: TESTNG-98 (temporary files have guaranteed fixed names) TESTNG-95 (Assertion failed comparison trims trailing ">") TESTNG-70 (TestNG prevents eclipse from opening an older CVS version of a java class) display of test hierarchy information (TESTNG-29) =========================================================================== 5.0.1 Eclipse plug-in Added: Output directory for the tests Added: Can now specify listener classes =========================================================================== 5.0.1 Fixed: reports generated by SuiteHTMLReporter do not work with JDK1.4 =========================================================================== 5.0 Added: Ant task: support for JVM, workingDir, timeout Added: Stack traces can be interactively shown in the HTML reports Added: Link to testng.xml in the reports Added: New structure for reports, suites go in their individual directory Added: @Test(suiteName) and @Test(testName) Added: The stack traces in reports do not include TestNG frames (system property testng.exception) (see: http://groups.google.com/group/testng-dev/browse_thread/thread/9f4d46ade10b0fda) Fixed: Exit with error when no methods are run (see: http://groups.google.com/group/testng-dev/browse_thread/thread/3c26e8a5658f22ac) Added: List of methods in alphabetical order Fixed: Class-scoped annotations were not recognized when inherited Added: Deprecated @Configuration and introduced @BeforeSuite/Test/Class/TestMethod Added: Deprecated @ExpectedExceptions and moved it into @Test Added: expectedExceptions to @Test, deprecated @ExpectedExceptions Added: New annotations: @BeforeSuite, @BeforeTest, etc... Fixed: Was returning an exit code of 0 if a cyclic graph was detected Added: Interface org.testng.ITest so that tests can declare a name Fixed: The Text reporter was reporting the square of the actual number of methods Fixed: Bug reported by Eran about dependencies with an afterClass method Added: IHookCallBack now receives the ITestResult in its run() method Added: Name of suite for command line can be set with -Dtestng.suite.name=xxx Fixed: TestNGAntTask was hardcoding m_haltOnFSP to true Fixed: Passing a null parameter caused an NPE in the reports Added: "listener" to the ant task (and documentation) Added: if patch-testng-sourcedir.properties is found in the classpath with a property "sourcedir" containing a ; separated list of directories, this list will override -sourcedir. =========================================================================== 4.7 Added: Maven 2 plug-in Fixed: Message formattings in TestNG assertion utility class Fixed: @Factory methods were counted as @Test as well http://jira.opensymphony.com/browse/TESTNG-51 Fixed: All DataProvider parameters were shown in the HTML report Fixed: Bug in testng-failed.xml generation Fixed: bug when using a jar file to load the test classes Added: alwaysRun for before @Configuration methods http://jira.opensymphony.com/browse/TESTNG-35 Fixed: groupless @Configurations were not invoked if a method depends on a group http://jira.opensymphony.com/browse/TESTNG-45 Added: beforeGroups/afterGroups to @Configuration Eclipse plugin: Added: last contextual launch is available in Eclipse launcher lists Fixed: 3.2M5 integration (removed dependency on non-existing class) Fixed: testng-failures.xml generation =========================================================================== 4.6 Added: Documentation contains the new reports Added: TestNG.setUseDefaultListeners(boolean) Added: Descriptions now appear in TextReporter (verbose>=2) and the HTML reports Added: description attribute to @Test and @Configuration Added: combined Reporter output in the reports Added: methods not run in the reports Added: org.testng.IReporter Added: threadPoolSize to @Test Added: Reports now show relative timings (start at 0) Added: Reports now show different colors depending on the methods' classes Added: Reports now show all parameters used to invoke the test method Added: org.testng.Reporter Added: DataProviders can accept a Method as first parameter Fixed: Extraneous implicit inclusion of a method Eclipse plugin: Added: Run/Debug as TestNG test from the editor contextual menu Fixed: TESTNG-24: 'Run as testng test' does not appear of the Test annotation does not have a group Fixed: TESTNG-18: Eclipse plugin ignores Factory annotation Fixed: TESTNG-21: Show differences when double clicking assertion exceptions Added: UI allows setting orientation (even more space) http://forums.opensymphony.com/thread.jspa?threadID=17225&messageID=33805#33805 =========================================================================== 4.5 Core: Fixed: Methods were not implicitly included, only groups Fixed: Bug with failed parent @Configuration don't skip child @Configuration/@Test invocations Fixed: Bug with overridding @Configuration methods (both parent and child were run) Fixed: Bug when overriding beforeClass methods in base class (cyclic graph) Added: Support for JAAS (see org.testng.IHookable) Fixed: Problem with nested classes inside Fixed: ArrayIndexOutOfBoundsException for jMock Added: dependsOnMethods can contain methods from another class Fixed: JUnitConverter required -restore, not any more (option is now a no-op) Fixed: JUnit mode wasn't invoking setName() on test classes Added: Regular expressions for classes in Added: Distributed TestNG Fixed: Command line parameters and testng.xml are now cumulative Fixed: Reports now work for multiple suites Fixed: Was ignoring abstract classes even if they have non-abstract instances Fixed: If setUp() failed, methods were not skipped Fixed: Was not clearly indicating when beforeSuite fails Added: @Configuration.inheritGroups Fixed: inconsistency between testng.xml and objects regarding method selectors Eclipse plug-in: New look for the progress view. =========================================================================== 4.4 Core: Fixed: testng-failures.xml was not excluding methods from base classes Fixed: Bug in suites of suites for JUnit mode =========================================================================== 4.3 Core: Fixed: testng-failures.xml was not excluding methods from base classes Fixed: Bug in suites of suites for JUnit mode Added: Excluded groups on command line and ant task Fixed: When including a group, implicitly include groups depended upon Fixed: When depending on several groups, wasn't skipped if one of them failed Fixed: Failures weren't reported accurately in the JUnitReports report Fixed: Wasn't throwing an exception if depending on a non-existing group =========================================================================== 4.2 Core: Fixed: wasn't excluding methods in base classes Added: alwaysRun for tests (soft dependencies) Fixed: Class-level enabled=false were not honored Fixed: Bug with multiple dataproviders on same class Fixed: Bug with dataprovider defined in the parent class Fixed: Bug with dataprovider defined in a subclass Fixed: Bug with dataprovider defined in an abstract class Fixed: testng-failures generation was excluding the methods even if a failed test depended on it =========================================================================== 4.1 Core: Added: @DataProviders can return Iterable Fixed: Superclass test methods were not called in the presence of a class @Test Added: Reporter class to log messages in the HTML reports =========================================================================== 4.0 Core: Fixed: suite methods now invoked only once in a hierarchy Added: @DataProvider and @testng.data-provider Fixed: Interleave order now respected for before/afterClass methods Added: Can now invoke java -jar testng-2.6.jar <...> Added: Support for BeanShell Added: Method Selectors (IMethodSelector) Fixed: In the absence of dependencies, @Configuration methods respect inheritance Fixed: Bug in multithreaded dependencies on methods Fixed: dependsOnGroups wasn't working on regular expressions Fixed: Bug in when directories contain spaces in their names Fixed: Introduced a JDK5 dependency in the JDK1.4 build (getEnclosingClass()) Fixed: Output directory in ant task was not honored if it didn't exist Fixed: Problem with timeout according to http://forums.opensymphony.com/thread.jspa?threadID=6707 Eclipse plug-in: Fixed: Wasn't handling linked directories correctly Fixed: Bug in QuickFix implementation Added: Quick Fix for JUnit conversion (Annotations and JavaDoc) Fixed: Methods Run as TestNG test Added: Package level Run as TestNG test Fixed: Resources from the linked directories are using a wrong path when passed to command line TestNG IDEA plug-in: Added: Support for JDK 1.4 (both projects and IDEA itself) Fixed: Classes that contained only configuration were ignored =========================================================================== 2.5 Added: ITestListener.onTestStart(ITestResult) Added: Support for Added: Resource files for easier ant taskdefs Fixed: @Configuration methods were not invoked with individual test methods Fixed: Bug with ExpectedExceptions Fixed: Didn't support nested factory classes Fixed: NPE if -target is omitted with JDK 1.4 Fixed: @Configuration failures in a class would cause other classes to fail Added: alwaysRun Fixed: beforeTestClass/afterTestClass were broken for a pathological case Added: @Configuration(alwaysRun) Added: JUnitConverter task Fixed: < and > characters in reports were not escaped Eclipse plug-in: Fixed: Class dialog wasn't showing @Factory classes IDEA plug-in: First release! Documentation: Added: Brand new look!!! Added: Section on testng.xml Fixed: Numbering of sections =========================================================================== 2.4 Changed: New package: testng.org Fixed: Bug with @ExpectedException occuring the parallel mode Fixed: Bug with parameters and beforeTest Added: IInstanceInfo support Fixed: methods were not excluded when included by groups Fixed: testng-failures.xml is now including also the beforeSuite/afterSuite methods Fixed: generating the testng-failures.xml is now working as expected Fixed: Factories call all the tests even if some of them fail along the way Fixed: Better JUnit support (wasn't creating individual instances) Fixed: dependsOnGroups didn't work across different classes Added: command line (and Ant) -groups option Added: @Parameters (and made parameters attribute deprecated) Added: Parameters for constructors Fixed: Better interleaving of before/afterTestMethods Fixed: Ant task Fixed: TestNGException thrown when TestNG conditions are not fulfilled Documentation: - New assert classes - New ways to launch - JUnitConverter documentation - new beforeSuite/afterSuite =========================================================================== 2.3 Fixed: Spaces are now legal in JavaDoc comments Added: documentation for @Factory Fixed: factories were called multiple times Added: beforeSuite and afterSuite Fixed: inheritance and scope now working properly for annotations Fixed: dependsOnMethods wasn't working for 1.4 Added: Better stack traces Added: Better syntax for included/excluded methods Fixed: Better verbose support Fixed: Various fixes for the Eclipse plug-in Added: Can specify a class name on the command line Fixed: Default package bug in JUnitConverter Added: Regression tests for JUnitConverter Added: -quiet option to JUnitConverter =========================================================================== 2.2 Fixed: Wasn't handling several testng.xml files correctly Fixed: Renamed -src to -sourcedir Fixed: Complains if no sourcedir is specified in 1.4 Added: In 1.4, don't require annotations="javadoc" Fixed: If setUp fails, complain and mark test methods as skips Fixed: Dependent methods weren't working for 1.4 =========================================================================== 2.1 Added: Parser can accept an InputStream for testng.xml Fixed: expected-exceptions now fails if test passes Fixed: reports now use the suite name in HTML Added: invocationCount and successPercentage Added: dependsOnMethods Added: timeOut works in non-parallel mode =========================================================================== 2.0 Added: port on JDK 1.4 =========================================================================== 1.3 Added: new view: classes (still experimental) Added: timeout on methods Added: thread-count Added: TestNG is now multithread, see "parallel" in =========================================================================== 1.2 Added: JUnitConverter Fixed: Bug with afterClasses (test: AfterClassCalledAtTheEnd) =========================================================================== 1.1 Added: new links for methods and groups in the HTML report Added: Added: to =========================================================================== 1.0 Fixed: Updated to the new DTD Fixed: Suite table of contents displays failures first Fixed: Bug in afterTestClass Added: Validating testng.xml Added: Scoped parameters Added: testng.xml Removed: Property quiet Changed: Verbose is now an integer Added: Dependent methods =========================================================================== 0.9 Added: Groups of groups Added: Groups for Configuration methods Added: Parameters =========================================================================== 0.2 Fixed: Merged TestMethod and TestClass into Test Added: HTML report Added: Regexps for groups Fixed: Inheritance of methods Fixed: ExpectedException is now called ExpectedExceptions