1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<div class="intro">
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>This example shows how to use advanced test options, which allow you to specify additional information about how a test should be run.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove Each <a href="index.html#testcase"><code>TestCase</code></a> can specify up to three different options for tests,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove including tests that should be ignored, tests that should throw an error, and tests that should fail.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<div class="example yui3-skin-sam">
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove {{>test-advanced-test-options-source}}
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<h2 class="first">Advanced Test Options</h2>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>This example begins by creating a namespace and a <code>Y.Test.Case</code> object:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.TestCase({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests"
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>This <code>Y.Test.Case</code> serves as the basis for this example.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<h3>Using <code>_should</code></h3>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>Immediately after the <code>name</code> of the <code>Y.Test.Case</code> is defined, there is a <code>_should</code> property.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove This property specifies information about how tests <em>should</em> behave and is defined as an object literal with one
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove or more of the following properties: <code>fail</code>, <code>error</code>, and <code>ignore</code>.Each of these three
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove is also defined as an object literal whose property names map directly to the names of test methods in the <code>Y.Test.Case</code>.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove This example uses all three properties:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.TestCase({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * Specifies tests that "should" be doing something other than the expected.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * Tests listed in here should fail, meaning that if they fail, the test
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * has passed. This is used mostly for YuiTest to test itself, but may
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * be helpful in other cases.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the test named "testFail" should fail
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testFail: true
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * Tests listed here should throw an error of some sort. If they throw an
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * error, then they are considered to have passed.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * You can specify "true" for each test, in which case any error will
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * cause the test to pass.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testGenericError: true,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * You can specify an error message, in which case the test passes only
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * if the error thrown matches the given message.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testStringError: "I'm a specific error message.",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testStringError2: "I'm a specific error message.",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * You can also specify an error object, in which case the test passes only
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * if the error thrown is on the same type and has the same message.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError: new TypeError("Number expected."),
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError2: new TypeError("Number expected."),
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError3: new TypeError("Number expected.")
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * Tests listed here should be ignored when the test case is run. For these tests,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove * setUp() and tearDown() are not called.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testIgnore: true
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>This <code>Y.Test.Case</code> specifies one test that should fail, six that should throw an error, and one that should be ignored.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>In the <code>fail</code> section, the test method <code>testFail()</code> is specified as one that should fail. By adding the
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove property <code>testFail</code> and settings its value to true, the <code>Y.Test.Runner</code> knows that this test is expected to fail.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove If the test were to be run without failing, it would be considered a failure of the test. This feature is useful when testing
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove YUI Test itself or addon components to YUI Test.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>Moving on to the <code>error</code> section, there are six tests specified that should throw an error. There are three different ways
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove to indicate that a test is expected to throw an error. The first is simply to add a property with the same name as the test method
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove and set its value equal to true (similar to specifying tests that should fail). In this example, the <code>testGenericError()</code>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove method is specified this way. When specified like this, the test passes regardless of the type of error that occurs. This can be
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove dangerous since unexpected errors will also cause the test to pass. To be more specific, set the property value for the test method
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove to an error message string. When a string is used instead of the Boolean true, the test passes only when an error is thrown and that
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove error message matches the string. In this example, <code>testStringError()</code> and <code>testStringError2()</code> expect an error
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove to be thrown with an error message of "I'm a specific error message." If any other error occurs inside of the these methods,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove the test will fail because the error message doesn't match. The last way to specify an error should occur is to create an actual error
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove object, which is the case with <code>testObjectError()</code>, <code>testObjectError2()</code>, and <code>testObjectError3()</code>.
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove When specified in this way, a test will pass only when an error is thrown whose constructor and error message match that of the
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove error object.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The last section is <code>ignore</code>, which determines tests that should be ignored. In this example, the method <code>testIgnore()</code>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove is set to be ignored when the <code>Y.Test.Case</code> is executed. Test in the <code>ignore</code> section are specified the same way
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove as those in the <code>fail</code> section, by adding the name as a property and setting its value to true.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<h3>Creating the test methods</h3>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The next part of the example contains the actual test methods. Since each test method is specified as having a certain behavior in
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove <code>_should</code>, they each do something to show their particular functionality.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The first method is <code>testFail()</code>, which does nothing but purposely fail. Since this method is specified as one that should
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove fail, it means that this test will pass:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.Test.Case({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testFail : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //force a failure - but since this test "should" fail, it will pass
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove Y.Assert.fail("Something bad happened.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>This method uses <code>Assert.fail()</code> to force the test to fail. This type of method is helpful if you are creating your own
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove type of assert methods that should fail when certain data is passed in.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>Next, the test methods that should error are defined. The <code>testGenericError()</code> method is specified as needing to throw
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove an error to pass. In the <code>error</code> section, <code>testGenericError</code> is set to true, meaning that any error causes
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove this method to pass. To illustrate this, the method simply throws an error:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.Test.Case({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testGenericError : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new Error("Generic error");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The fact that this method throws an error is enough to cause it to pass (the type of error and error message don't matter). The next
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove two methods, <code>testStringError()</code> and <code>testStringError2()</code> are specified as throwing an error with a specific
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove message ("I'm a specific error message."):</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.Test.Case({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testStringError : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //throw a specific error message - this will pass because it "should" happen
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new Error("I'm a specific error message.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testStringError2 : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //throw a specific error message - this will fail because the message isn't expected
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new Error("I'm a specific error message, but a wrong one.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The <code>testStringError()</code> method will pass when executed because the error message matches up exactly with the one
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove specified in the <code>error</code> section. The <code>testStringError2()</code> method, however, will fail because its
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove error message is different from the one specified.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>To be more specific, <code>testObjectError()</code>, <code>testObjectError2()</code>, and <code>testObjectError3()</code>,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove specified an error type (<code>TypeError</code>) and an error messsage ("Number expected."):</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.Test.Case({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //throw a specific error and message - this will pass because it "should" happen
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new TypeError("Number expected.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError2 : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //throw a specific error and message - this will fail because the type doesn't match
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new Error("Number expected.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testObjectError3 : function() {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //throw a specific error and message - this will fail because the message doesn't match
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove throw new TypeError("String expected.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>Of the these three methods, only <code>testObjectError()</code> will pass because it's the only one that throws a <code>TypeError</code>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove object with the message, "Number expected." The <code>testObjectError2()</code> method will fail because the type of error
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove being thrown (<code>Error</code>) is different from the expected type (<code>TypeError</code>), as specified in the <code>error</code>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove section. The last method, <code>testObjectError3()</code>, also fails. Though it throws the right type of error, the error message
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove doesn't match the one that was specified.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>The last method in the <code>Y.Test.Case</code> is <code>testIgnore()</code>, which is specified to be ignored. To be certain, this
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove method pops up a message:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.example.test.AdvancedOptionsTestCase = new Y.Test.Case({
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove //the name of the test case - if not provided, one is automatically generated
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove name: "Advanced Options Tests",
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove testIgnore : function () {
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove alert("You'll never see this.");
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>If this test weren't ignored, then the alert should be displayed. Since it is ignored, though, you will never see the alert. Additionally,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove there is a special message displayed in the <code>Y.Console</code> when a test is ignored.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<h3>Running the tests</h3>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>With all of the tests defined, the last step is to run them:</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove//create the console
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove verbose : true,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove newestOnTop : false
1ac6d38f8eef642ee4f7191b863a986f4cb7571fDav Glass})).render('#testLogger');
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove//add the test suite to the runner's queue
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan GroveY.Test.Runner.add(Y.example.test.AdvancedOptionsTestCase);
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove//run the tests
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<p>Before running the tests, it's necessary to create a <code>Y.Console</code> object to display the results (otherwise the tests would run
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove but you wouldn't see the results). After that, the <code>Y.Test.Runner</code> is loaded with the <code>Y.Test.Suite</code> object by calling
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove <code>add()</code> (any number of <code>Y.Test.Case</code> and <code>Y.Test.Suite</code> objects can be added to a <code>Y.Test.Runner</code>,
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove this example only adds one for simplicity). The very last step is to call <code>run()</code>, which begins executing the tests in its
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove queue and displays the results in the <code>Y.Console</code>.</p>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove<h2>Complete Example Source</h2>
1b7d9ee6f1128c8cb5e16c3a11ba045998296171Ryan Grove{{>test-advanced-test-options-source}}