i'm writing an asp.net c# web application that use different providers to authenticate the user. The function take all the providers with reflection, call them and store the result in a list. The problem is that if the provider use a lot of time to do some
operations the page load is incredible slow. I need to cycle all the providers and call for each one a specific function, so i thought to use multithreading to execute the functions in parallels. Is it possible? How?
Parallel extensions are able to make use of multiple cores in your machine. It is the question of balancing - test your solution using threads and using parallel extensions and see how you get better performance. I wrote about parallel extensions blog posting
Comparing LINQ and PLINQ performance, there is also source code available. Maybe it helps you.
Don't forget to mark solution providing post as "Answered".
this code is in a function that is called from the mvc page controller. First page execution give me ~13 seconds, the second ~26, the third ~39, etc. Why? Every time i have the old time + 13 seconds
i have a variable out of the foreach block that i need to fill with some data. I fill the variable in the method called in the block loop. I pass it as a reference so i can process it in the function and set the values. The problem is that the function that
fill the variable is called for each plugin loaded in my application and so the functions can be very slow or fast. What i want to do is call all the functions at the same time so that i can reduce the time to load the web page
This lock practically guarantees that instead of parallel processin there is parallel waiting and serial processing. Of course, with more resources used than before. You have to handle the situation differently. Let each loop in for return its result independently
and after for is done take the results and select the one you like.
The other whay is to stop processing as soon as you get value you expect. But you cannot use some higher scope variables like this if you want parallel processing.
Don't forget to mark solution providing post as "Answered".
wait wait wait... i don't understand what i should do to improve performances. Can you write me an example? Should i loop normally witouth parallel processing?
You should also think about caching the results returned by the providers, I mean is it really necessary to retrieve each request a fresh list from the providers? Decide what would be an acceptable latency and use that to set the cache expiration. You can
also combine, let's say for Provider A you cache 5 minutes, for provider B you cache 1 hour and for Provider C you don't cache at all.
And always identify the bottlenecks before you start optimizing the code. I am saying this because if you have 3 providers and split the 3 calls in seperate threads you have absolute no guarantee that it will be more performant unless it takes each provider
a more or less equal amount of time to execute. So if Provider A takes 30 seconds to complete and Provider B and Provider C each 1 second, you will have almost zero advantage of making it multithreaded. Therefor look where the real bottleneck(s) is/are and
decide on how to optimize that.
If you have any questions on that, we are all here to help :-)
Give a man a fish and you will feed him for a day. Teach a man to fish and you will feed him for a lifetime.
vecchia
Member
3 Points
60 Posts
Multithreading application
Sep 05, 2010 09:27 AM|LINK
Hi all,
i'm writing an asp.net c# web application that use different providers to authenticate the user. The function take all the providers with reflection, call them and store the result in a list. The problem is that if the provider use a lot of time to do some operations the page load is incredible slow. I need to cycle all the providers and call for each one a specific function, so i thought to use multithreading to execute the functions in parallels. Is it possible? How?
DigiMortal
Contributor
5658 Points
939 Posts
MVP
Re: Multithreading application
Sep 05, 2010 04:01 PM|LINK
To get started:
To get smarter:
Read these materials carefully because threading problems are way harder to debug and profile than "usual" problems.
Also visit my ASP.NET blog or follow me @ Twitter:twitter.com/gpeipman
vecchia
Member
3 Points
60 Posts
Re: Multithreading application
Sep 05, 2010 04:06 PM|LINK
mmm, i saw also Parallel class. I need to loop a list of plugins and call the method that i need, could i use Parallel.ForEach?
DigiMortal
Contributor
5658 Points
939 Posts
MVP
Re: Multithreading application
Sep 05, 2010 04:23 PM|LINK
Parallel extensions are able to make use of multiple cores in your machine. It is the question of balancing - test your solution using threads and using parallel extensions and see how you get better performance. I wrote about parallel extensions blog posting Comparing LINQ and PLINQ performance, there is also source code available. Maybe it helps you.
Also visit my ASP.NET blog or follow me @ Twitter:twitter.com/gpeipman
vecchia
Member
3 Points
60 Posts
Re: Multithreading application
Sep 05, 2010 04:29 PM|LINK
i don't understand the problem: this is my test code:
Parallel.ForEach(plugins, (plugin, state) => { lock (sync) { plugin.MyMethod(options, ref response); } if (response.status == "myresponse") { state.Break(); } });this code is in a function that is called from the mvc page controller. First page execution give me ~13 seconds, the second ~26, the third ~39, etc. Why? Every time i have the old time + 13 seconds
DigiMortal
Contributor
5658 Points
939 Posts
MVP
Re: Multithreading application
Sep 05, 2010 06:25 PM|LINK
Why are you using lock in for statement? If you need serial processing then you don't need to use parallel processing model.
Also visit my ASP.NET blog or follow me @ Twitter:twitter.com/gpeipman
vecchia
Member
3 Points
60 Posts
Re: Multithreading application
Sep 05, 2010 06:40 PM|LINK
i have a variable out of the foreach block that i need to fill with some data. I fill the variable in the method called in the block loop. I pass it as a reference so i can process it in the function and set the values. The problem is that the function that fill the variable is called for each plugin loaded in my application and so the functions can be very slow or fast. What i want to do is call all the functions at the same time so that i can reduce the time to load the web page
DigiMortal
Contributor
5658 Points
939 Posts
MVP
Re: Multithreading application
Sep 05, 2010 06:53 PM|LINK
This lock practically guarantees that instead of parallel processin there is parallel waiting and serial processing. Of course, with more resources used than before. You have to handle the situation differently. Let each loop in for return its result independently and after for is done take the results and select the one you like.
The other whay is to stop processing as soon as you get value you expect. But you cannot use some higher scope variables like this if you want parallel processing.
Also visit my ASP.NET blog or follow me @ Twitter:twitter.com/gpeipman
vecchia
Member
3 Points
60 Posts
Re: Multithreading application
Sep 05, 2010 07:01 PM|LINK
wait wait wait... i don't understand what i should do to improve performances. Can you write me an example? Should i loop normally witouth parallel processing?
gabriel.loza...
Contributor
3583 Points
800 Posts
Re: Multithreading application
Sep 05, 2010 08:16 PM|LINK
You should also think about caching the results returned by the providers, I mean is it really necessary to retrieve each request a fresh list from the providers? Decide what would be an acceptable latency and use that to set the cache expiration. You can also combine, let's say for Provider A you cache 5 minutes, for provider B you cache 1 hour and for Provider C you don't cache at all.
And always identify the bottlenecks before you start optimizing the code. I am saying this because if you have 3 providers and split the 3 calls in seperate threads you have absolute no guarantee that it will be more performant unless it takes each provider a more or less equal amount of time to execute. So if Provider A takes 30 seconds to complete and Provider B and Provider C each 1 second, you will have almost zero advantage of making it multithreaded. Therefor look where the real bottleneck(s) is/are and decide on how to optimize that.
If you have any questions on that, we are all here to help :-)