Background threads in ASP.net applications (Part 2 – threading side effects)

This post has been republished via RSS; it originally appeared at: IIS Support Blog articles.

In the second article of the series on the dangers of ASP.net background threading, to illustrate the dangers of such an architecture, I will start by introducing one more modification to the code of the sample application. This modification will be added to the code of the Global.asax that starts off the infinite loop thread that runs outside of the ASP.net thread pool. Here is the listing again, but notice the lines of code that have been added:

 

private void UpdateSatelliteInfo() { string url = "https://api.wheretheiss.at/v1/satellites/25544"; Satellite satellite; string content; HttpWebRequest request; StreamReader reader;int loopIteration = 1; do { //should we have a loop iteration that yields modulo 5 = 0, crash if ((loopIteration % 5) == 0) { throw new Exception("Random crash"); } request = (HttpWebRequest)WebRequest.Create(url); reader = new StreamReader(request.GetResponse().GetResponseStream()); content = reader.ReadToEnd(); reader.Close(); satellite = JsonConvert.DeserializeObject(content); satellite.Time = UnixTimeStampToDateTime(satellite.Timestamp); Application["satellite"] = satellite; //sleep for 5 Seconds System.Threading.Thread.Sleep(5000); //increment the loop iteration loopIteration = loopIteration + 1; } while (true); }

 


The first thing added is the creation of a private variable called loopIteration which we set to the value 1. Then inside the do {} while(true) infinite loop, I added a check to see if the result of the division of the value of the new variable by 5 is not zero – that is to say, each time the value of the variable is a multiple of 5 (5, 10, 15, etc), the if branch will be taken and an exception will be thrown by the code.

If you attempt to run the sample in IIS, you will see that after a time, the application pool will crash, and you will get an event 5011 logged by WAS (the Windows Process Activation Service) inside the System application log:

 

A process serving application pool 'MvcSample' suffered a fatal communication error with the Windows Process Activation Service. The process id was '2952'. The data field contains the error number.

 

 

So what just happened?

When request execution happens inside IIS, ASP.net gives you a safety net. Almost all exceptions (except for a System.OutOfMemoryException, System.StackOverflowException and System.ExecutionEngineException) are caught by the runtime, if they are not caught by your application directly. Hence an exception like FileNotFoundException that would occur during a page execution (request 3 in the picture below) would wind up with a 500 error page being sent to the connecting user – who would see the ASP.net yellow screen of death, but the w3wp.exe process hosting the application would not crash. Hence all other threads inside the process could go on treating requests, and the only impacted person would be the user requesting that particular page.


For thread that we launched with the Application_Start() event handler, there is no associated context and the Thread itself is not part of the ASP.net thread pool. Hence, when an exception occurs, if it is not handled in the application code, it will be handed directly to the operating system to treat: the way Windows deals with such an exception is by crashing the w3wp.exe process – and all other threads and requests inside it. This is what happens when the throw statement is executed in the loop. If you are unlucky enough to send a request exactly when this happens, you will see a message indicating that “This page cannot be displayed” in your browser.

clipboard_image_0.png

Another, more subtle danger is application domain unloading and reloading. The application domain is a data structure that loads the entire code of your ASP.net application inside the w3wp.exe process. Since we cannot unload .dlls (assemblies) in .Net, when a change is made to the application on disk – files are changed inside the bin folder, the web.config file is updated, the entire application domain has to unload and reload with a new copy of your application. You can read more about this process by following this link:

http://linqto.me/AppDomainUnload

Supposing that the application domain has to be unloaded, but that the thread we launched at the beginning of the application is still running and holding on to objects in the current instance of the application domain, the unload cannot complete. Hence, you can wind up with behavior of the following sort: you perform a change to your application, and you do not see the change when refreshing the application in the browser. This is because the old app domain cannot unload until the spawned thread finishes dealing with all the objects it was holding on to inside the old app domain – and since we are in an infinite loop, this will be never happen.

Also, when you attempt to recycle the application pool in IIS, you may see warnings from WAS in the event log indicating that the w3wp.exe process serving the pool took too long to shut down, and hence was killed. IIS has a mechanism by which, a w3wp.exe process serving an application pool is allowed 90 seconds to gracefully stop all threads and shut down when a recycle has to occur. Passed this time period (controlled by the shutdownTimeLimit - http://www.iis.net/configreference/system.applicationhost/applicationpools/add/processmodel - parameter in the configuration), WAS will issue a kill process and force shutdown the w3wp.exe process. In the case of the sample application, the looping thread will never relinquish control and will not allow the process to shut down even after 90 seconds, so WAS will have to proceed with a kill.

Conclusion:

It is never a good idea to spawn up background threads in your application. If you need to have something treated on another thread outside the thread that does request treatment, consider calling the Threadpool.QueueWorkItem (http://msdn.microsoft.com/en-us/library/system.threading.threadpool.queueuserworkitem%28v=vs.110%29.aspx ) method call – this will use a free thread from the ASP.net threadpool to execute the code you give it. Another possibility is to consider asynchronous execution using the new .Net syntax with async and await.

 

written by: Paul Cociuba

reviewed by: Muna AlHasan

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.