We have migrated an ASP.NET 4 app to a new server (2008 R2) and we are experiencing production issues - occasional stack overflows in the ASP.NET worker process.
Sometimes the faulting module is MSVCR100_CLR0400.dll, which is from the VC++ redistributable - but I don't think C++ has been used anywhere in the project. Note that when I force a stack overflow in my code I can see the faulty module indicated in the log
entry
Also, this happens at a particular place in the code - but not every time, and is unreproducible on test environments. It's also not a concurrency issue - it has been recorded with just 1 online user.
I got dumps of the error, I open them in visual studio and debug - it complains that it doesn't have symbols for w3wp and offers to show disassembly. The call stack has just a single entry in the case when the faulting module is CLR:
clr.dll!000007fef9afd01b()
In the case of MSVCR100_CLR0400.dll the call stack I get from the dump is:
MSVCR100_CLR0400.dll!000007fef9a3c9fa()
[Frames below may be incorrect and/or missing, no symbols loaded for MSVCR100_CLR0400.dll] MSVCR100_CLR0400.dll!000007fef9a57fcb()
MSVCR100_CLR0400.dll!000007fef9a33ee4()
MSVCR100_CLR0400.dll!000007fef9a33f5d()
clr.dll!000007fef9b05a08()
Again, when I force a stack overflow in my code - I see the correct line and call stack, with the thousands recursive calls that caused it. Not in the production situation described - I get just the above entries in the call stack.
I've tried the basic troubleshooting techniques - try-catch everywhere, logging after each line, Application_Error - nothing. Execution just stops at some point and I get the error entry in Windows logs
App pool is in integrated mode, running under a service account created for this purpose.
What can I do? Does an "aspnet_regiis -i" have a chance to fix the issue? Or some web app / app pool setting?
It's defnitely linked to the application behaviour - it's always at a specific step in the app - but how can I troubleshoot it? How can I get more details on the error?
Vroomfundel
0 Points
2 Posts
Occasional stack overflow in w3wp - how to troubleshoot?
Nov 04, 2012 11:49 AM|LINK
We have migrated an ASP.NET 4 app to a new server (2008 R2) and we are experiencing production issues - occasional stack overflows in the ASP.NET worker process.
These get logged in the event logs:
Faulting application name: w3wp.exe, version: 7.5.7601.17514, timestamp: 0x4ce7afa2
Faulting module name: clr.dll, version: 4.0.30319.269, time stamp: 0x4ee9d6e1
Exception code: 0xc00000fd
Fault offset: 0x0000000000003fbc
Faulting process id: 0x17ec
Faulting application start time: 0x01cdb8e2663f86bc
Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: 45eb70a1-24da-11e2-a0b9-005056ba357d
Sometimes the faulting module is MSVCR100_CLR0400.dll, which is from the VC++ redistributable - but I don't think C++ has been used anywhere in the project. Note that when I force a stack overflow in my code I can see the faulty module indicated in the log entry
Also, this happens at a particular place in the code - but not every time, and is unreproducible on test environments. It's also not a concurrency issue - it has been recorded with just 1 online user.
I got dumps of the error, I open them in visual studio and debug - it complains that it doesn't have symbols for w3wp and offers to show disassembly. The call stack has just a single entry in the case when the faulting module is CLR:
clr.dll!000007fef9afd01b()
In the case of MSVCR100_CLR0400.dll the call stack I get from the dump is:
MSVCR100_CLR0400.dll!000007fef9a3c9fa()
[Frames below may be incorrect and/or missing, no symbols loaded for MSVCR100_CLR0400.dll] MSVCR100_CLR0400.dll!000007fef9a57fcb()
MSVCR100_CLR0400.dll!000007fef9a33ee4()
MSVCR100_CLR0400.dll!000007fef9a33f5d()
clr.dll!000007fef9b05a08()
Again, when I force a stack overflow in my code - I see the correct line and call stack, with the thousands recursive calls that caused it. Not in the production situation described - I get just the above entries in the call stack.
I've tried the basic troubleshooting techniques - try-catch everywhere, logging after each line, Application_Error - nothing. Execution just stops at some point and I get the error entry in Windows logs
App pool is in integrated mode, running under a service account created for this purpose.
What can I do? Does an "aspnet_regiis -i" have a chance to fix the issue? Or some web app / app pool setting?
It's defnitely linked to the application behaviour - it's always at a specific step in the app - but how can I troubleshoot it? How can I get more details on the error?