Last post Feb 22, 2011 05:54 PM by tatsky
Feb 18, 2011 12:58 PM|tatsky|LINK
I am looking into ways I can reduce the load on my database, and so optimise a website. The scenario is this:
An ECommerce website with over 2500 product pages.
Each product Page contains standard info like Product Name, Image, Description, Related products, Customer Reviews
Each page also contains "You recently looked at" and "Your basket" type dynamic areas.
I want to be able to cache the standard information that is on the page, as there is little point loading this from the database each page load.
I was wondering it would be a stupid idea to have the static content elements published as HTML files on the disk, and have the site load that static HTML content into content holders on the page? I could even wrap the disk read up in some code so that
the html is cached in memory for say 30 seconds, to stop successive page loads hitting the disk for each load.
When the product is updated in the database, a new HTML file could be published to disk.
Would this be any benefit? Is there a better way to cache these elements? I know there is the page cache directive, but I do not want to cache the whole page, as there are some dynamic elements on the page which do require a database read.
My other thought was to have all the product info published to an XML file, and have the product page pull in the data using XSLT. A bit like the mechanism used in the Umbraco CMS product. However, I am not too sure whether the transformations would be
any quicker than just loading an HTML fragment from disk.
Any suggestions or theories on this are greatly welcomed.
Feb 18, 2011 02:09 PM|BoogleC|LINK
As I understand it (and things may have changed since I last read up on this), 1 HDD read is near as makes no difference one database read (if your database programming is up to scratch and I assume it is, you sound like you know what you're doing).
So what i'm saying is I don't think you'll benefit much (I wouldn't consider 2,500 pages a lot to be honest... especially if it's a fairly modern framework).
I'm not sure of your logic right now but maybe try writing a dll (if not already) to process requests for a one dynamic product page?
Maybe run some tests on a couple of different ideas also...
Feb 18, 2011 03:56 PM|atconway|LINK
Is there a better way to cache these elements? I know there is the page cache directive, but I do not want to cache the whole page, as there are some dynamic elements on the page which do require a database read.
Have you considered the Cache object? You could store product information that is mostly static in Cache for all Application instances, and not have to cache the entire page. I would read through these caching recommendations from the MSDN to choose which
one suits your situation best, but in the end I think Caching of some variation is the answer.
ASP.NET Caching Overview:
ASP.NET Caching: Techniques and Best Practices:
Caching in .NET Framework Applications:
Feb 19, 2011 01:01 PM|tatsky|LINK
Thanks for the input. I did wonder what the if there would be much difference between a HDD read and a DB read.
The pages are all served from 1 dynamic page, using some URL rewriting. The database code is pretty optimised, but I never stop asking myself if it could be better/more efficient.
However, looking at the scenario, the information about the products changes very rarely. So there is no need to keep loading it from the DB. Maybe it would be an idea to load that data and store it in the cache. I do this already for some other information
on the site, i.e content pages where the text is stored in the DB, but cached.
I could have a loader for the content which checks the cache for the product. if it doesn't exist then the DB is called, and then the data is cached. setting a sliding expiration would make sure the data doesn't hang around longer than is needed. So popular
pages would always be available in the cache, but less popular would drop out of the cache to save on memory.
I have used the page caching previously, but I found that was causing some issues with some sections I did not want to cache.
I think I might give caching the product data a go and see how that goes.
Feb 21, 2011 02:12 PM|atconway|LINK
I could have a loader for the content which checks the cache for the product. if it doesn't exist then the DB is called, and then the data is cached. setting a sliding expiration would make sure the data doesn't hang around longer than is needed.
This is what I would do in your situation, and is probably the best path. Just as you stated though, just start with caching groupings of data (i.e. the 'Products' list, etc.) on not entire pages; at least at 1st. I think you will achieve the desired outcome
with this approach.
Feb 22, 2011 07:05 AM|tatsky|LINK
I have now implimented my caching for product data, but have another issue to think about.
I know what products on my site are looked at, so I can run some reports to see how frequently products are looked at. So as an example I have looked at what products were viewed in a 24 hour period, and then looked at the products with the highest views.
I have then looked at the frequency of views on a page.
The result is on average a popular product is viewed every 16 minutes.
Befoee I checked this, I set my cache timeout to 30 seconds. This is great for situations where the customer hits refresh, or views a page in quick succession (the desired effect was to reduce load on the DB which this does).
But now I am wondering if I should set my expiration to something like 20 minutes. That would then have the data cached for 70% of reloads on the page.
As far as I can see, the only down side to this is the amount of data being put into the cache. However, the product data itself is not that big so this should not be a big issue.
Are there any considerations for how long to store some data in cache?
Feb 22, 2011 04:34 PM|atconway|LINK
Are there any considerations for how long to store some data in cache?
I would say the biggest one is the importance of showing truly up to date data. If you are in a situation where the report is historical and does not even have up to the date information, then a longer cache period makes sense. If you are reporting
daily transactions for the current day, then a short cache timeout or none at all make sense.
It is true you could begin to hog up a ton of memory on the server if you start dumping tons and tons into the cache, but if it is important enough then server scaling (web farm, more memory, etc) might be warranted for an important site.
Feb 22, 2011 05:54 PM|tatsky|LINK
Thanks for the reply
The data I am caching is basic product info stuff, i.e. Name, Description, Unique Code, Price etc. This data does not change regularly. And when it does change, I can have the cached version of it purged quite easily.
I have tested the cache with a sliding expiration of 30 minutes, and the performance increase is a marked improvement. I have over 2500 products on the site, but the usual 80/20 rule comes into play so I would expect 400-500 cached at any one time. At
the moment it is more like 200 being cached.
I will keep an eye on the memory usage over the next few weeks.
Now to look for the next area of optimisation.