There are several candidates for this problem. From a cursory inspection of your site i can suggest a couple of things to lookinto.
1. Caching - DNN caching settings (in host settings) can have an enormous impact on pageload.
Try them out to see what works, bear in mind this will only affect Non form pages and essentially will prevent unnecessary database access.
2. Sql server optimization - default sql setup is far from optimal for DNN and does a whole load of stuff you just don't need, like roll back transaction log support. There are great many issues here but some heavy optimization will have a massive effect on speed for servers with load. This can emerge as a problem as soon as traffic starts rising and WILL cause embarrassing 'application unavailable' errors under load. Optimize early on.
3. Keep Alives - This is an ASP.net issue but essentially the app is unloaded from memory when not being used, so your 10-20 delay is the app firing up for the first time. There are tools you can buy to sort this out, but also learn about the ASP.net availability issues.
4. ASP.net application pool availability and the default machine.config settings will affect performance/availability issues.
Hope this helps, there are more things that can be done, but i would have to charge consulting fees for any more advice ;). Besides if you look into all of this then your sites will be faster than 90% of DNN deployments out there.
Happy Hunting
Matt Gahan