Yes - it does make sense. That said I have no clue if you can programmatically recycle app domains, regardless it is IMO not the correct technical solution. The real problem you're trying to solve here is one of distributed caching. Basically, you should be caching on a machine that is logically or physically disparate from your web-servers. This way each web server grabs the latest cache from a different machine and cache gets updated in one place. There is a Microsoft product, Velocity, specifically for this. Unfortunately, it is not out yet (for production). It sounds like you're working on a production system, so you probably shouldn't rely on this.
There are other options available, though, including Memcache (http://jehiah.cz/projects/memcached-win32/) or you can just google "distributed caching .net". I've also read posts mentioning that you can solve the problem is distributed caching using SQL notification services, though I can't say I've seen an implementation.
Personally, though, unless you're working on a very large/high-traffic system...the added complexity of doing this likely will nnot be worth it. You problem is probably better solved by modifying settings on your proxy server. Most proxy servers have a setting called something like "Session Stickiness". Basically, the proxy server will make sure that User A is pushed to the same node within a certain timespan (say, 20 minutes). This means that if user-specific information is inserted into cache, the proxy will guide that user to the same machine. You can set the time-limit to the same duration as your session-timeout. If you are inserting non-user specific data into cache - it shouldn't matter since you'll want to have that cached on however many machines you've got running.
Hope this explanation makes sense.
Sanjay