I don't know if this is possible, but you could try common cache-breaking techniques to prevent this.
If you are initiating the request for each chunk, you can add a random number at the end of the query string for each one. That would prevent the browser from caching it.
There may also be properties you can set to prevent caching of Silverlight bits. I don't really know.