I recently had the opportunity to setup two applications in a test environment where I could compare the performance of an ASP.Net web forms application and an ASP.Net MVC application. Both applications run on a Windows 2003 IIS 6.0 machine with two 2.4GHz
processors and 2G of memory. Both applications communicate with a SQL Server 2000 database server which has the same hardware profile and uses the same database objects. Both applications make use of Ajax. Each application runs in it's own virtual directory.
Both applications use n-Tier architecture. The database objects use MS Data Application Blocks. Both applications use standard ASP.Net authentication.
The test client is running WAPT 6.0. It's got a dual-core, 2.8 GHz processor and 2G of memory as well.
I performing the following tests from the test client.
20 simultaneous users and 1000 complete sessions.
50 simultaneous users and 1000 complete sessions.
100 simultaneous users and 1000 complete sessions.
200 simultaneous users and 1000 complete sessions.
500 simultaneous users and 2000 complete sessions.
A session is as follows. The test user logs in, searches for a record (uses Ajax), selects a record, updates the record (uses Ajax), and logs out. There is no delay between requests. This test beats the server to death until all the sessions are complete.
So the MVC application out-performs the Web Forms application up to 100 simultaneous users. Between 100 and 200 simultaneous users, the MVC application begins to break down; note the error count. At 500 simultaneous users the web forms application starts
to break down, but only at 12% of the rate of the MVC application.
I am surprised by these results. I thought that the MVC application would be 50% faster than the Web Forms application. The Web Forms application has to rebuild the entire control graph dynamically each postback in order to handle events, and still performs
well under load. Also, the MVC application's CPU utilization and error count were higher than the Web Form application's CPU utilization and error count.
stackoverflow.com uses ASP.NET MVC ... I wonder what their experience might be.
Just curious, what are the types of errors that the ASP.NET MVC application is getting?
B-) Please help me by completing my school survey about computer programmers on my website. Thank you!!! Gerry Lowry +1 705-429-7550 wasaga beach, ontario, canada
In the Application Event Log, I am getting a "System.OutOfMemoryException" exception and IIS crashes. When this happens, the CPU goes into oscillation, spiking between 10 and 100%. Then things settle down after IIS reloads.
In the WAPT load testing software, I see "FAILURE: Socket error: Receive Timeout".
The problem gets worse as the number of users increases from 100 to 500.
My guess is that the DB is the bottleneck. MVC is more efficient at poundng the DB. Once the DB is saturated you are just queueing more DB requests - and these timeout. Without an analyis of the entire workflow, you can't draw any conclusions from your test.
I don't think it's the database. I would get timeout errors, and other exceptions from the data layer. Remember, I'm using the DAAB. I'm not getting these types of exceptions.
I am getting "out of memory" errors, which are causing IIS to crash. But the out of memory exceptions only occur on the MVC side, at least until we get to 500 simultaneous users.
This implies that the ASP.Net application has a smaller footprint per session, because it can handle more simultaneous users without error.
What if you Moq'd the DB? That would eliminate the DB as a factor, n'est-ce pas?
B-) Please help me by completing my school survey about computer programmers on my website. Thank you!!! Gerry Lowry +1 705-429-7550 wasaga beach, ontario, canada
I'm running two applications under the same load test conditions. One application doesn't run out of memory. One application does. Something is wrong with memory management in the MVC application. I watched it run out of memory at 200 users, after which
IIS crashes and the CPUs oscillate wildly.
Memory Utilization
100 Users 200 Users
ASP.Net Web Forms 400M 450M
ASP.Net MVC 1000M 2000M
If I Moq the database interface, then I'm essentially comparing a unit test MVC application against a load test Web Forms application. That hardly seems fair.
I'm thinking it has something to do with the MVC design and poor memory management caused by the IoC. I'm using Castle Windsor. And something is not releasing memory.
Good point. I was not thinking. One of the key advantages of ASP.NET MVC over WebForms is easier testability.
However, instead of thinking about comparing when you Moq the database interface, think about this:
if the ASP.NET MVC application runs out of memory when you Moq it, it is likely that you have discovered a problem with ASP.NET MVC memory management (I hope that is not the case; fingers crossed). OTOH, if the memory errors do not happen, then the theory
suggested by Rick (ricka6) might be valid.
Either way, the issue is serious. It reminds me of a company whose accounting software would not run under Windows NT because the NT computers were too fast.
Errors are errors and absolutely the cause must be identified so the errors can be chased away forever.
You are doing some very good investigating. While I can not speak for them, I'm guessing Microsoft will take your investigations very seriously. We (I) can not bet the ranch (although I already have) on ASP.NET MVC if it breaks down as you scale it up.
Regards,
Gerry (Lowry)
B-) Please help me by completing my school survey about computer programmers on my website. Thank you!!! Gerry Lowry +1 705-429-7550 wasaga beach, ontario, canada
>>Something is wrong with memory management in the MVC application.
Your right. You've got might have a memory leak in your code. Depending on the application, web forms can have higher memory footprints (viewstate, controls). In some case MVC can have higher memory load. I would investigate where the memory
is going.
The CLR Profiler has built-in support for ASP.NET applications.
In the meantime, I'm increasing the memory in my virtual machine so that I can get better load testing measurements temporarily. I see a 33% increase in sessions per second with MVC. So I hope I can resolve this memory issue with more memory, but this is
not an acceptable long-term solution.
Also, I'm going to replace the Castle Windsor IoC framework with the Unity IoC framework, and see if that makes a difference.
rgalante
Member
14 Points
26 Posts
Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 02, 2009 05:19 PM|LINK
I recently had the opportunity to setup two applications in a test environment where I could compare the performance of an ASP.Net web forms application and an ASP.Net MVC application. Both applications run on a Windows 2003 IIS 6.0 machine with two 2.4GHz processors and 2G of memory. Both applications communicate with a SQL Server 2000 database server which has the same hardware profile and uses the same database objects. Both applications make use of Ajax. Each application runs in it's own virtual directory.
Both applications use n-Tier architecture. The database objects use MS Data Application Blocks. Both applications use standard ASP.Net authentication.
The test client is running WAPT 6.0. It's got a dual-core, 2.8 GHz processor and 2G of memory as well.
I performing the following tests from the test client.
20 simultaneous users and 1000 complete sessions.
50 simultaneous users and 1000 complete sessions.
100 simultaneous users and 1000 complete sessions.
200 simultaneous users and 1000 complete sessions.
500 simultaneous users and 2000 complete sessions.
A session is as follows. The test user logs in, searches for a record (uses Ajax), selects a record, updates the record (uses Ajax), and logs out. There is no delay between requests. This test beats the server to death until all the sessions are complete.
Web Forms MVC
20 users: 2.94 sessions / sec (0 errors) 4.17 sessions / sec (0 errors)
50 users: 3.13 sessions / sec (0 errors) 4.35 sessions / sec (0 errors)
100 users: 3.23 sessions / sec (0 errors) 4.35 sessions / sec (7 errors) MVC application
200 users: 3.03 sessions / sec (0 errors) 3.85 sessions / sec (26 errors) MVC application
500 users: 3.03 sessions / sec (67 errors) 3.92 sessions / sec (547 errors) MVC application
So the MVC application out-performs the Web Forms application up to 100 simultaneous users. Between 100 and 200 simultaneous users, the MVC application begins to break down; note the error count. At 500 simultaneous users the web forms application starts to break down, but only at 12% of the rate of the MVC application.
I am surprised by these results. I thought that the MVC application would be 50% faster than the Web Forms application. The Web Forms application has to rebuild the entire control graph dynamically each postback in order to handle events, and still performs well under load. Also, the MVC application's CPU utilization and error count were higher than the Web Form application's CPU utilization and error count.
These figures are reproducible.
gerrylowry
All-Star
20513 Points
5712 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 02, 2009 05:57 PM|LINK
stackoverflow.com uses ASP.NET MVC ... I wonder what their experience might be.
Just curious, what are the types of errors that the ASP.NET MVC application is getting?
rgalante
Member
14 Points
26 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 02, 2009 06:12 PM|LINK
In the Application Event Log, I am getting a "System.OutOfMemoryException" exception and IIS crashes. When this happens, the CPU goes into oscillation, spiking between 10 and 100%. Then things settle down after IIS reloads.
In the WAPT load testing software, I see "FAILURE: Socket error: Receive Timeout".
The problem gets worse as the number of users increases from 100 to 500.
ricka6
All-Star
15070 Points
2272 Posts
Microsoft
Moderator
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 02, 2009 07:40 PM|LINK
My guess is that the DB is the bottleneck. MVC is more efficient at poundng the DB. Once the DB is saturated you are just queueing more DB requests - and these timeout. Without an analyis of the entire workflow, you can't draw any conclusions from your test.
rgalante
Member
14 Points
26 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 02, 2009 07:53 PM|LINK
I don't think it's the database. I would get timeout errors, and other exceptions from the data layer. Remember, I'm using the DAAB. I'm not getting these types of exceptions.
I am getting "out of memory" errors, which are causing IIS to crash. But the out of memory exceptions only occur on the MVC side, at least until we get to 500 simultaneous users.
This implies that the ASP.Net application has a smaller footprint per session, because it can handle more simultaneous users without error.
gerrylowry
All-Star
20513 Points
5712 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 03, 2009 12:04 AM|LINK
What if you Moq'd the DB? That would eliminate the DB as a factor, n'est-ce pas?
rgalante
Member
14 Points
26 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 03, 2009 12:31 PM|LINK
I'm running two applications under the same load test conditions. One application doesn't run out of memory. One application does. Something is wrong with memory management in the MVC application. I watched it run out of memory at 200 users, after which IIS crashes and the CPUs oscillate wildly.
Memory Utilization
100 Users 200 Users
ASP.Net Web Forms 400M 450M
ASP.Net MVC 1000M 2000M
If I Moq the database interface, then I'm essentially comparing a unit test MVC application against a load test Web Forms application. That hardly seems fair.
I'm thinking it has something to do with the MVC design and poor memory management caused by the IoC. I'm using Castle Windsor. And something is not releasing memory.
gerrylowry
All-Star
20513 Points
5712 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 03, 2009 01:27 PM|LINK
Good point. I was not thinking. One of the key advantages of ASP.NET MVC over WebForms is easier testability.
However, instead of thinking about comparing when you Moq the database interface, think about this:
if the ASP.NET MVC application runs out of memory when you Moq it, it is likely that you have discovered a problem with ASP.NET MVC memory management (I hope that is not the case; fingers crossed). OTOH, if the memory errors do not happen, then the theory suggested by Rick (ricka6) might be valid.
Either way, the issue is serious. It reminds me of a company whose accounting software would not run under Windows NT because the NT computers were too fast.
Errors are errors and absolutely the cause must be identified so the errors can be chased away forever.
You are doing some very good investigating. While I can not speak for them, I'm guessing Microsoft will take your investigations very seriously. We (I) can not bet the ranch (although I already have) on ASP.NET MVC if it breaks down as you scale it up.
Regards,
Gerry (Lowry)
ricka6
All-Star
15070 Points
2272 Posts
Microsoft
Moderator
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 03, 2009 04:28 PM|LINK
>>Something is wrong with memory management in the MVC application.
Your right. You've
gotmight have a memory leak in your code. Depending on the application, web forms can have higher memory footprints (viewstate, controls). In some case MVC can have higher memory load. I would investigate where the memory is going.The CLR Profiler has built-in support for ASP.NET applications.
CLR Profiler: http://www.microsoft.com/downloads/details.aspx?FamilyId=A362781C-3870-43BE-8926-862B40AA0CD0&displaylang=en
tmarq has a nice blog of how to use the profiler with ASP.NET: http://blogs.msdn.com/tmarq/archive/2007/06/09/the-clr-profiler.aspx
rgalante
Member
14 Points
26 Posts
Re: Performance: ASP.Net Web Forms vs ASP.Net MVC
Nov 03, 2009 05:31 PM|LINK
Thanks for the tips. I'll look at those articles.
In the meantime, I'm increasing the memory in my virtual machine so that I can get better load testing measurements temporarily. I see a 33% increase in sessions per second with MVC. So I hope I can resolve this memory issue with more memory, but this is not an acceptable long-term solution.
Also, I'm going to replace the Castle Windsor IoC framework with the Unity IoC framework, and see if that makes a difference.
I'll post my results.