Create theWebTestClassInitializeandWebTestClassCleanupattributes
to mark methods that should run at the beginning and at the end of a test class.
One of the features of NUnit that I really loved was the fact they walked the class inheritance chain looking for their [TestSetup] attribute. This turned out to be really powerful because I could create a base test class with a [TestSetup] attribute over
the TestSetup() Method. That method did all the common stuff and then called an empty virtual method that individual tests could override.
I structure our test cases in a very specific manner. I do not test by functionality I test by Page. The idea is if you are a developer and you make a change to a page, you can come into our test framework and run all the tests for the page you just changed,
on your dev box BEFORE you checking. Kindof like an automated deck check.
What this means is we can use a TestSetup() type method to get all the data ready for each test on the page. For example If we are testing a ShoppingList we want to make sure its cleared before each test begins. And since we have all the ShoppingList tests
in one TestClass, we can isolate all the setup and teardown code need for a single page into its class.
The problem this creates is that there is now a lot of duplicate code in each of these test pages. Code that authenticates and logs into the site, code that navigates to the page being tested, and all the small stuff shared across every page.
Here is some PseudoCode that shows what I did in NUnit and WatiN.
[WebTestClass]
public class BaseTestClass
{
[WebTestClassInitialize]
public void TestSetup()
{
// Login to the test site and open the test page
TestSite.Authenticate();
StartPageAdapter.NavigateTo();
CustomTestSetup();
}
public virtual void CustomTestSetup() {}
}
// ------------------------------------
public class CustomerAccountsPageTests: BaseTestClass
{
public override CustomTestSetup()
{
// Reset Test Data before each test on this page
AutomationDataManager.ResetTestData();
}
[WebTestMethod]
[WebTestTag(WebTestCategoryTag.Smoke)]
public void ValidateCustomerAccount1()
{
// Do Test
Assert.IsTrue(SomeTest.Results);
}
}
A couple of important points.
The [WebTestClass] attribute is on the base class and not the derived class. So the LATF would need to be able to reflect to get that as well
The [WebTestMethod] attribute would still have to be on each test method.
And if that wasn't enough, I think it would be really cool to be able to have [WebTestMethods] in the base class as well. So a set of basic tests that are run on every TestClass.
This is not a high priority for us, I am hacking around this by using a TestSetup(); method call at the start of each test, but the Attributes would allow me to remove this call and have it done automatically .. it would be cleaner.
Michael Cowan
Aisle7
QA Manager (QA Lead)
michael.cowan@healthnotes.com
I hear you. This is a feature that was sorely needed when we wrote the tests for MVC, I really want to get it in but it keeps falling down the priority rank and gets punted.
So there's two things here right:
Some Setup and TearDown attributes.
Take into account inheritance when we look for everything.
I have confidence that we will get #1 for our next iteration, we work out of 2 month iterations (the reason for such long iterations is that we are the QA team for ASP.NET, we support LTAF in addition to testing the product). I add #2 as a work item as well,
but I am not sure it will make the cut list.
I completely understand and I am just happy that you guys have gotten as much in as you have already!
I am going to throw out a lot of stuff like this with the goal of just generating thought and discussions. To be honest the benifits of the framework are so great that I have no problem working around things that I want.
Keep up the good work!
Michael Cowan
Aisle7
QA Manager (QA Lead)
michael.cowan@healthnotes.com
I wanted to give you and update on the setup and clean up attribute features. In the current internal branch (Hopefully to be release soon) we have implemented a feature that I think will solve your needs. We now call the dispose method when the test class
implements IDisposable.
You can add the above code to the Testcase class in the finally block of the executing method.
So how does this help? Here is the deal the idea of implementing the setup and clean up attributes is a good idea but why implement something that already exists in the language? If you look at the code you will notice that every time a test method is execute
we create a new instance of the class, which means the constructor is called, and with the fix above we will cause the dispose methods. This means that you can stick your setup code in the constructor and Implement IDisposable and stick your clean up code
in the dispose method. This code will run once for each test case inside of the class. Additional implementing this feature this way means that we by default (aka without any extra code) handle inheritance.
Matthew M. Osborn
http://blog.osbornm.com
http://www.codingqa.com
http://weblogs.asp.net/asptest
"Change the world or go home."
mcowan
Member
8 Points
42 Posts
WebTestClassInitialize Suggestion: Use reflection to look into base class for the attribute
Apr 05, 2009 06:49 PM|LINK
In the RoadMap you mentioned this:
Create the WebTestClassInitialize and WebTestClassCleanup attributes to mark methods that should run at the beginning and at the end of a test class.
One of the features of NUnit that I really loved was the fact they walked the class inheritance chain looking for their [TestSetup] attribute. This turned out to be really powerful because I could create a base test class with a [TestSetup] attribute over the TestSetup() Method. That method did all the common stuff and then called an empty virtual method that individual tests could override.
I structure our test cases in a very specific manner. I do not test by functionality I test by Page. The idea is if you are a developer and you make a change to a page, you can come into our test framework and run all the tests for the page you just changed, on your dev box BEFORE you checking. Kindof like an automated deck check.
What this means is we can use a TestSetup() type method to get all the data ready for each test on the page. For example If we are testing a ShoppingList we want to make sure its cleared before each test begins. And since we have all the ShoppingList tests in one TestClass, we can isolate all the setup and teardown code need for a single page into its class.
The problem this creates is that there is now a lot of duplicate code in each of these test pages. Code that authenticates and logs into the site, code that navigates to the page being tested, and all the small stuff shared across every page.
Here is some PseudoCode that shows what I did in NUnit and WatiN.
[WebTestClass] public class BaseTestClass { [WebTestClassInitialize] public void TestSetup() { // Login to the test site and open the test page TestSite.Authenticate(); StartPageAdapter.NavigateTo(); CustomTestSetup(); } public virtual void CustomTestSetup() {} } // ------------------------------------ public class CustomerAccountsPageTests: BaseTestClass { public override CustomTestSetup() { // Reset Test Data before each test on this page AutomationDataManager.ResetTestData(); } [WebTestMethod] [WebTestTag(WebTestCategoryTag.Smoke)] public void ValidateCustomerAccount1() { // Do Test Assert.IsTrue(SomeTest.Results); } }A couple of important points.
And if that wasn't enough, I think it would be really cool to be able to have [WebTestMethods] in the base class as well. So a set of basic tests that are run on every TestClass.
This is not a high priority for us, I am hacking around this by using a TestSetup(); method call at the start of each test, but the Attributes would allow me to remove this call and have it done automatically .. it would be cleaner.
Aisle7
QA Manager (QA Lead)
michael.cowan@healthnotes.com
farmas
Participant
1164 Points
259 Posts
Microsoft
Re: WebTestClassInitialize Suggestion: Use reflection to look into base class for the attribute
Apr 06, 2009 01:37 AM|LINK
I hear you. This is a feature that was sorely needed when we wrote the tests for MVC, I really want to get it in but it keeps falling down the priority rank and gets punted.
So there's two things here right:
I have confidence that we will get #1 for our next iteration, we work out of 2 month iterations (the reason for such long iterations is that we are the QA team for ASP.NET, we support LTAF in addition to testing the product). I add #2 as a work item as well, but I am not sure it will make the cut list.
- Federico
mcowan
Member
8 Points
42 Posts
Re: WebTestClassInitialize Suggestion: Use reflection to look into base class for the attribute
Apr 06, 2009 03:10 AM|LINK
I completely understand and I am just happy that you guys have gotten as much in as you have already!
I am going to throw out a lot of stuff like this with the goal of just generating thought and discussions. To be honest the benifits of the framework are so great that I have no problem working around things that I want.
Keep up the good work!
Aisle7
QA Manager (QA Lead)
michael.cowan@healthnotes.com
osbornm
Participant
914 Points
196 Posts
Microsoft
Re: WebTestClassInitialize Suggestion: Use reflection to look into base class for the attribute
Feb 09, 2010 06:54 PM|LINK
Hello Michael,
I wanted to give you and update on the setup and clean up attribute features. In the current internal branch (Hopefully to be release soon) we have implemented a feature that I think will solve your needs. We now call the dispose method when the test class implements IDisposable.
<div style="PADDING-BOTTOM: 0px; MARGIN: 0px; PADDING-LEFT: 0px; PADDING-RIGHT: 0px; DISPLAY: inline; FLOAT: none; PADDING-TOP: 0px" id=scid:9ce6104f-a9aa-4a17-a79f-3a39532ebf7c:c2bd88eb-4734-4903-ace9-013e245ce97b class=wlWriterEditableSmartContent mce_style="padding-bottom: 0px; margin: 0px; padding-left: 0px; padding-right: 0px; display: inline; float: none; padding-top: 0px"> <div style="BORDER-BOTTOM: #000080 1px solid; BORDER-LEFT: #000080 1px solid; FONT-FAMILY: 'Courier New', Courier, Monospace; COLOR: #000; FONT-SIZE: 10pt; BORDER-TOP: #000080 1px solid; BORDER-RIGHT: #000080 1px solid" mce_style="border: #000080 1px solid; color: #000; font-family: 'Courier New', Courier, Monospace; font-size: 10pt"> <div style="BACKGROUND: #ddd; MAX-HEIGHT: 300px; OVERFLOW: auto" mce_style="background: #ddd; max-height: 300px; overflow: auto">- if (webTestObject
is IDisposable)
- {
- ((IDisposable)webTestObject).Dispose();
- }
</div></div></div>You can add the above code to the Testcase class in the finally block of the executing method.
So how does this help? Here is the deal the idea of implementing the setup and clean up attributes is a good idea but why implement something that already exists in the language? If you look at the code you will notice that every time a test method is execute we create a new instance of the class, which means the constructor is called, and with the fix above we will cause the dispose methods. This means that you can stick your setup code in the constructor and Implement IDisposable and stick your clean up code in the dispose method. This code will run once for each test case inside of the class. Additional implementing this feature this way means that we by default (aka without any extra code) handle inheritance.
http://blog.osbornm.com
http://www.codingqa.com
http://weblogs.asp.net/asptest
"Change the world or go home."