Last post Jan 31, 2010 07:53 AM by Dango23
Jan 26, 2010 07:09 PM|Dango23|LINK
Let me tell you what I need to implement on my company .NET application and I will be more than glad to hear your advise how to do it:
For every user loged-in to the company website, We want to know on which page he is right now. It means, If user X is now on page “1.html”, I need to save this information somewhere. If user X after 30-60 seconds going to page “2.html” I want to update it fast
where he is now surfing. It means I want to be able to tell at REAL TIME, on each page, Which users are right now observing the page. This is the problem, now I'm looking for the right solution.
If some day my company website will have 400,000 user log-in at the same time, I am going to save to every one of them where he is surfing right now. Maybe I exaggerated, but we do have 45,000 users at the same time on the company web site and the number
is growing. I think big so the solution for this problem will be good for a lot of users.
So, my data object for every user should keep this information somewhere:
1. User code (User Number).
2. Which page user is now observing (Page Number).
This object is going to update every 30–60 seconds by using AJAX, so it should be updated fast. It is a lot of reading/writing data. If a user want to see which other users are now on the same page as he his, he will be able to see it, just by going through
the whole array of objects.
I don’t need this information to be saved in DB, because it doesn’t mean anything. It is temporary and changing A LOT. So, I think the right place to do it is using RAM at the the company server, using the Caching Data option. What do you think? Am I wrong?
If I am going to have a data table with lots of records and every record is going to be update every 30-60 seconds. Where should I save all this table data? I am using SQL Server 2008 to save important data as needed, but this data is not important to save,
just need to use it for my purpose. Where should I save this data? I think saving it in DB is a bad idea, because of the dynamic update of records every few seconds. I am thinking about 100,000 users who update this table every few seconds. It probably going
to kill my DB, so DB is not an option.
What is the right way to implement this kind of thing?
You may think of it like every page is "a chat room" and I want to know who enter now the room (the page), it doesn't have to be completely accurate in time, but approximately.
cache ram chat data time real time database db ajax
Jan 27, 2010 05:10 AM|vijay_010237866|LINK
one solution is to use application variable.
with full information like user name,page current.
update for the user on each page load as he changes to other
and u can look through applicaiton variable to get information about all the users currently logged
Jan 27, 2010 05:12 AM|amanbhullar|LINK
If the user hit is going in thousands, then you need to save all the information in DB.
At the max what you can do is took just one or two day information in the table. It will give you a performance boost.
Else your db will be too slow
Jan 27, 2010 08:51 AM|Dango23|LINK
Vijay, It could be a solution if I had only few users logged-in at the same time, but I'm thinking in big numbers.
Since application variables are defined globally, these variables are available in the memory until the application closes. Using large objects such as lists of all my users is a large data that may lead to some serious problems, I think.
Jan 27, 2010 09:15 AM|Dango23|LINK
amanbhullar, I can use my SQL Server 2008 and save for each user logged-in which page he is right now observing, but I think the it will kill my DB. It's going to be a table that update very fast all the time.
Think about 10,000 users logged-in. I am updating for each and every one of them is current page, it means every 30-60 seconds I'm updating 10,000 records at the same time and maybe removing some if user logged-out or maybe close his browser. I all the
time check to see where is the user right now.
And more than that. If I'm going to show each user which other users are also observing with him the same page, it means I am sending lots of queries all the time in order to get the updated list of users for each page. All the adding/updaing/removing from
all the users is a lot of work on a very short of time at the same table.
Jan 27, 2010 10:21 AM|atconway|LINK
It seems as if some of your statements contradict each other a bit. In one statement you say "You need to be able to see this data real time and it is important", but in another statement you say "The data isn't that important and it does not need to be
Well I am going to assume the data is needed, and I would not immediately think that saving this data off to the database is a terrible idea. 1st off SQL Server can handle a great deal of connections and load. For example take any of the major banking
websites and the transactions being processed by them. You don't think they have 10's of thousands of users simultaneously doing banking transactions and hitting their database? The absolutely do, and unless you are using Microsoft Access or something I would
not be overly concerned just yet about what SQL Server can handle.
The 1st step might be to create a proof of concept to really determine if all of your numbers are correct, and what load it really places on SQL server. As far as ADO.NET, you will have automatic connection pooling built in which will help with performance
in multiple repeat connections to the same database. You may find out this entire discussion is a moot point and the database can handle the load, and the server notices no performance issues. Especially since the queries you will be running will be concise
Now you did mention another route with caching or possibly storing this data in application server objects or with caching to be accessed in that manner. Now I can tell you for sure, that if you start using caching for that amount of data, you
will see significant web server (not database obviously) performance issues as caching does have overhead and needs to be used sparingly. If you were talking about 10-15 users, this might be a decent route, but not with the numbers you are indicating.
Caching is better used for global mostly static data that is unchanging; like a product catalog. Here is a nice description about how to use caching (http://visualstudiomagazine.com/Articles/2004/04/01/Speed-Up-Your-ASPNET-Pages.aspx?Page=2):
"Cache the server-based data that's used repeatedly and doesn't change between each request from a page. You must strike a balance between what data to cache, when, and for how long. Don't cache too many different items. You pay an overhead penalty for
maintaining the cache, and you pay an extra penalty as the system keeps the cache up to date tracking all those bits and pieces (see Figure 2).
Don't put items in the cache that expire quickly. The maintenance overhead might negate the benefit of caching if ASP.NET must update the cache with fresh data every second or two."
Another possibility, might be to use a store that is outside of ASP.NET but not a database. For example a text or XML file. The specifics on how to maintain them and read from them will be more difficult than SQL Server, but it is an option. Many users
would want to access the same file or set of files, and maintaining the queue of opening and closing the file could get cumbersome quickly. I just wanted to put it out there as an idea, although not the one I would attempt 1st.
I think you should try a combination of a lightweight Session Object that contains the user data having that data persisted to the database, and see what type of results you get. You may be surprised at SQL Servers ability to handle those transactions.
Some aspect of the performance of SQL Server also is provided by the hardware it sits on, but with a company your size I am sure SQL Server is probably got some horsepower behind it.
Hope this helps!
Jan 27, 2010 10:51 AM|Dango23|LINK
atconway, 10x for your detailed answer.
What I meant is the the DATA is important for only a few seconds, because it's being updata so fast
depending on the user browsing rate.
Jan 31, 2010 07:53 AM|Dango23|LINK
Update: I decided to use SQL Server to save the data. 10x!
BTY - I want to know how many people on my company web site are observing each web page in a real time. How can I do that? Only Ajax?
For example: If user "Tom" is logged-in now and observing page "1.aspx" I want to know about it. I also want to know when "Tom" stop observing that page. He can move to another page, than I know by server page request he is not any more at page "1.aspx",
but what if he moves to another web site? Or closing his browser? I just want to know while his browser on that page.
How can I know he has stopped observing that page, his browser closed or moved to another web site.
I think I need to send Ajax requests every 10 seconds in order to check it, but what kind of request?