There are several improvements to the unit test framework in this commit. * Test methods are no longer interleaved between classes. All test methods in a class are run together, with @BeforeClass and @AfterClass methods called immediately before and after the methods are run. As part of this fix, you are now required to include sequential=true in every class level @Test annotation. If you don't do this, the build will complain. * Added a TestCaseUtils.restartServer() method that will do an in core restart of the directory server during the tests. This can be used in a @BeforeClass method to ensure that the tests start with a clean directory server, and also in an @AfterClass method to cleanup after a test that makes a lot of configuration changes. So if you introduce a new test that runs fine in isolation but fails when run with other tests, you could try calling TestCaseUtils.restartServer() in its @BeforeClass method. The TestCaseUtils.restartServer() method will reinitialize the server and reload the original test configuration, but it's not quite the same as creating a completely new process. Specifically, it cannot undo any changes that were made to static member variables of a class. I've fixed a handful of places in the server where this was a problem, but there might be more lurking. If you write a test that changes static member variables of a class, please make sure that it cleans up after itself in an @AfterMethod or @AfterClass test. * The tests now use significantly much less memory. I saw a peak of only 80MB. There were two main problems. 1) TestNG holds on to all parameters and results for the whole test, and 2) since the test classes themselves live for the duration of the tests, their member variables were holding onto a lot of garbage. The in-core restart made this problem much worse because we ended up with lots of copies of the Schema, ConfigEntryS, etc. I've introduced some hacks to fix this. Basically the code uses devious methos to go in and null out the parameters and member variables after the test has run. If you're curious about the details of how we've addressed this take a look at the comments in DirectoryServerTestCase. From now on, all test classes must inherit directly or indirectly from DirectoryServerTestCase. The build will fail if they don't. * Upgrade to TestNG b5.7. There is a fix in this release that helps our tests to run in order, and I've had to make a couple of more fixes, which they will eventually put back into the trunk. * In classes with a class-level @Test annotation, TestNG treats any public method (except @Before/After* methods) as a test method. The build now points this out and asks you to either add a specific @Test annotation to the method or change the method to be non-public. I've fixed up several places where a test wasn't annotated and others where a non-test method was being treated as a test method. * The tests now report progress as they run. Run 'build testhelp' to see details on how to control the output. I've also added some new test properties, mainly to make debugging the tests easier * test.remote.debug.port: This test property allows you to remotely attach a debugger to the unit tests. If you provide a valid port value, the unit tests will not start to run until the debugger is attached. * testng.verbosity0to5: This test property controls the debugging output of TestNG. This output is useful to check the order in which test methods are invoked or other details on what TestNG is doing. Valid values are integer values from 0 (no output) to 5 (maximum output). Since this implicitly sets -Dorg.opends.test.suppressOutput=false, other stderr/stdout output generated by the unit tests will also be displayed. * org.opends.test.copyClassesToTestPackage=true: This test property copies the classes into the test server root. This enables you to run the server tools on the test server. It can slow down the test startup so the files are no longer copied by default. Thanks to Neil for trying out these changes and to him and Bo for helping me track down some of the memory leaks.