Last post Apr 21, 2020 07:33 PM by bruce (sqlwork.com)
Apr 21, 2020 12:55 PMemail@example.com|LINK
We have an ASP.NET WebAPI2 REST server. Requests to it are async where a request is submitted and returns a guid. And then the caller polls for the status on that guid. We have customers that will feed over 1,000 requests and then continue to feed up and
poll. And it can be 15 minutes until the last of the 1,000 is complete.
With a single server this works fine. Each request we create a Task in our app and IIS then handles processing them within its thread pool.
But load balancing is a problem. Handing 500 requests each to two servers is not necessarily equal. Once could complete in 2 minutes and the other take 13 minutes. Asking each how many pending jobs won't work because a job can be under 1 second or over 10
Logically the only thing I can think of is there's an intermediate store of requests. Then when a server has an available thread, it grabs the next request, processes it, and then saves it off.
Is this the best approach here?
And if so, does IIS have a way of signaling my app when it has an available thread from its pool?
thanks - dave
Apr 21, 2020 03:54 PM|mgebhard|LINK
I do not see the problem unless the application is designed to store state on the server. If that's the case then you need to redesign the application or implement sticky session.
If the clients are other machines, then your application should send a complete message to the other machine when the process competes.
Apr 21, 2020 07:33 PM|bruce (sqlwork.com)|LINK
you options will depend on your load balancer. Round robin is the typical configuration, but most support server load feature. There are typically hard to get correct. In load feature, the balancer needs to poll all the servers for their load.
IIS does not have a background thread feature, you are probably using an asp.net feature like worker thread pools, which run in the asp.net process, not IIS. You can easily create a counter of in process. if 5 sequential 1 sec requests are the same load
as one 5 sec request, then counts may work. its number running at a time that matters, not how long they run. but this will depend on your knowledge. some request may need more cpu or i/o so you will need a weighting algorythm.