I've implemented an asp.net mvc site where users can download large files (up to 4.7GB) and works (most of the time), but the cpu usage goes up to 100% and sometimes causes the file download to drop off.
I've searched through (lost count) a lot of articles, posts, etc but seems there isn't any simple (or any) solution to the problem of downloading large files with successful results (dropping files, cpu/memory usage, etc)
Below is the code I have within my asp.net mvc site.
Any help to ensure the cpu doesn't go through the roof would be much appreciated.
The webserver I'm hosting the site on is Windows Server 2008, 4GB, IIS7 and this is the only site running off it at the moment.
Code below:
public class DownloadResult : ActionResult
{
private SoftwareDBEntities entities = new SoftwareDBEntities();
public DownloadResult() { }
public DownloadResult(string virtualPath)
{
this.VirtualPath = virtualPath;
}
public string VirtualPath { get; set; }
public string PhysicalPath { get; set; }
public string FileDownloadName { get; set; }
public PathType FilePathType { get; set; }
public enum PathType { Virtual, Physical }
public override void ExecuteResult(ControllerContext context)
{
System.IO.Stream iStream = null;
// Buffer to read 10K bytes in chunk:
byte[] buffer = new Byte[10000];
// Length of the file:
int length;
// Total bytes to read:
long dataToRead;
// Identify the file to download including its path.
string filepath = "";
if (FilePathType == PathType.Physical)
filepath = PhysicalPath;
else
filepath = context.HttpContext.Server.MapPath(this.VirtualPath);
// Identify the file name.
string filename = System.IO.Path.GetFileName(filepath);
try
{
// Open the file.
iStream = new System.IO.FileStream(filepath, System.IO.FileMode.Open,
System.IO.FileAccess.Read, System.IO.FileShare.Read);
// Total bytes to read:
dataToRead = iStream.Length;
context.HttpContext.Response.ContentType = "application/octet-stream";
context.HttpContext.Response.AddHeader("Content-Disposition", "attachment; filename=" + filename);
bool disconnected = false;
// Read the bytes.
while (dataToRead > 0)
{
// Verify that the client is connected.
if (context.HttpContext.Response.IsClientConnected)
{
// Read the data in buffer.
length = iStream.Read(buffer, 0, 10000);
// Write the data to the current output stream.
context.HttpContext.Response.OutputStream.Write(buffer, 0, length);
// Flush the data to the HTML output.
context.HttpContext.Response.Flush();
buffer = new Byte[10000];
dataToRead = dataToRead - length;
}
else
{
//prevent infinite loop if user disconnects
dataToRead = -1;
//record if the user disconnected during download
disconnected = true;
}
}
//if the user didn't disconnect then save to history
if (disconnected == false)
{
//some code to record the file download was successful...
}
}
catch (Exception ex)
{
// Trap the error, if any.
context.HttpContext.Response.Write("Error : " + ex.Message);
}
finally
{
if (iStream != null)
{
//Close the file.
iStream.Close();
}
context.HttpContext.Response.Close();
}
}
}
I've used the FileStreamResult class and all seemed to be working ok, but with large files my server's memory goes rapidly up to 3.8GB or so and then kills the process and downloading of file is terminated.
This is when only one person is download a file, and I need to allow multiple (max 30) concurrent downloads at any time.
Is there any way in IIS to cap the server from giving out so much memory so it doesn't crash, and just slow down the download speed?
Any help would be much appreciated.
The code I used is below:
public class DownloadResult : FileStreamResult
{
public DownloadResult(Stream fileStream, string contentType)
: base(fileStream, contentType)
{ }
public long? FileSize { get; set; }
protected override void WriteFile(HttpResponseBase response)
{
response.BufferOutput = false;
if (FileSize.HasValue)
{
response.AddHeader("Content-Length", FileSize.ToString());
}
base.WriteFile(response);
}
}
In the original code that you posted (It looks similar to the example on
http://support.microsoft.com/kb/812406), have you considered removing the allocation in the inner loop after flushing the response buffer? Given a 4GB file and ~10K buffer, you're looking at 419, 000+ allocation operations. You should be able to reuse the
buffer since you've already written the data and flushed the Response.
I don't know the .NET garbage collector that well, but given that the buffer is a fixed size, with that many allocations happening in a loop, I suspect they'll get bunched in the same generation.
Unfortunately that didn't work. I didn't see any improvement suprisingly.
My memory still progressively climbs up to 100% then just crashes my file download which then frees up the memory.
I've spent days trying to find a solution to this, and really thought there was one good solution that "worked". I just can't believe that no one has the solution.
I'll keep trying, although think I need some sleep... any further help would be VERY much appreciated.
Is it possible to link to the file directly (as a static file) on the web site? Alternatively, can you post the file on an FTP server somewhere and generate links to that file? Both of those would probably be much more performant than involving MVC.
From your post I find that you wanted to download large file by using HTTP protocol in MVC. Based on my experience, if we place files in a separate file server, it's much better for improving the performance of our website. And of cource we can create ftp
server or bt server for files. I am sure it's good for our sites.
A likely problem is that the server is caching the whole file content before sending it. I read on another board that the filestreamresult normally caches all the stream data before flushing. I'll post a link if i can find it.
gjen020
Member
9 Points
13 Posts
Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 04, 2009 12:20 PM|LINK
Hi,
I've implemented an asp.net mvc site where users can download large files (up to 4.7GB) and works (most of the time), but the cpu usage goes up to 100% and sometimes causes the file download to drop off.
I've searched through (lost count) a lot of articles, posts, etc but seems there isn't any simple (or any) solution to the problem of downloading large files with successful results (dropping files, cpu/memory usage, etc)
Below is the code I have within my asp.net mvc site.
Any help to ensure the cpu doesn't go through the roof would be much appreciated.
The webserver I'm hosting the site on is Windows Server 2008, 4GB, IIS7 and this is the only site running off it at the moment.
Code below:
public class DownloadResult : ActionResult { private SoftwareDBEntities entities = new SoftwareDBEntities(); public DownloadResult() { } public DownloadResult(string virtualPath) { this.VirtualPath = virtualPath; } public string VirtualPath { get; set; } public string PhysicalPath { get; set; } public string FileDownloadName { get; set; } public PathType FilePathType { get; set; } public enum PathType { Virtual, Physical } public override void ExecuteResult(ControllerContext context) { System.IO.Stream iStream = null; // Buffer to read 10K bytes in chunk: byte[] buffer = new Byte[10000]; // Length of the file: int length; // Total bytes to read: long dataToRead; // Identify the file to download including its path. string filepath = ""; if (FilePathType == PathType.Physical) filepath = PhysicalPath; else filepath = context.HttpContext.Server.MapPath(this.VirtualPath); // Identify the file name. string filename = System.IO.Path.GetFileName(filepath); try { // Open the file. iStream = new System.IO.FileStream(filepath, System.IO.FileMode.Open, System.IO.FileAccess.Read, System.IO.FileShare.Read); // Total bytes to read: dataToRead = iStream.Length; context.HttpContext.Response.ContentType = "application/octet-stream"; context.HttpContext.Response.AddHeader("Content-Disposition", "attachment; filename=" + filename); bool disconnected = false; // Read the bytes. while (dataToRead > 0) { // Verify that the client is connected. if (context.HttpContext.Response.IsClientConnected) { // Read the data in buffer. length = iStream.Read(buffer, 0, 10000); // Write the data to the current output stream. context.HttpContext.Response.OutputStream.Write(buffer, 0, length); // Flush the data to the HTML output. context.HttpContext.Response.Flush(); buffer = new Byte[10000]; dataToRead = dataToRead - length; } else { //prevent infinite loop if user disconnects dataToRead = -1; //record if the user disconnected during download disconnected = true; } } //if the user didn't disconnect then save to history if (disconnected == false) { //some code to record the file download was successful... } } catch (Exception ex) { // Trap the error, if any. context.HttpContext.Response.Write("Error : " + ex.Message); } finally { if (iStream != null) { //Close the file. iStream.Close(); } context.HttpContext.Response.Close(); } } }Kind Regards,
Grant
Augi
Contributor
6730 Points
1142 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 04, 2009 01:59 PM|LINK
Have you tried standard FilePathResult and FileStreamResult classes?
gjen020
Member
9 Points
13 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 04, 2009 11:26 PM|LINK
Hi,
I've used the FileStreamResult class and all seemed to be working ok, but with large files my server's memory goes rapidly up to 3.8GB or so and then kills the process and downloading of file is terminated.
This is when only one person is download a file, and I need to allow multiple (max 30) concurrent downloads at any time.
Is there any way in IIS to cap the server from giving out so much memory so it doesn't crash, and just slow down the download speed?
Any help would be much appreciated.
The code I used is below:
public class DownloadResult : FileStreamResult { public DownloadResult(Stream fileStream, string contentType) : base(fileStream, contentType) { } public long? FileSize { get; set; } protected override void WriteFile(HttpResponseBase response) { response.BufferOutput = false; if (FileSize.HasValue) { response.AddHeader("Content-Length", FileSize.ToString()); } base.WriteFile(response); } }jeloff
Contributor
2493 Points
432 Posts
Microsoft
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 05, 2009 01:29 AM|LINK
Hi
In the original code that you posted (It looks similar to the example on http://support.microsoft.com/kb/812406), have you considered removing the allocation in the inner loop after flushing the response buffer? Given a 4GB file and ~10K buffer, you're looking at 419, 000+ allocation operations. You should be able to reuse the buffer since you've already written the data and flushed the Response.
I don't know the .NET garbage collector that well, but given that the buffer is a fixed size, with that many allocations happening in a loop, I suspect they'll get bunched in the same generation.
Jacques
gjen020
Member
9 Points
13 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 05, 2009 04:30 AM|LINK
Hi Jacques,
Unfortunately that didn't work. I didn't see any improvement suprisingly.
My memory still progressively climbs up to 100% then just crashes my file download which then frees up the memory.
I've spent days trying to find a solution to this, and really thought there was one good solution that "worked". I just can't believe that no one has the solution.
I'll keep trying, although think I need some sleep... any further help would be VERY much appreciated.
Kind Regards,
Grant.
levib
Star
7702 Points
1099 Posts
Microsoft
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 05, 2009 05:54 AM|LINK
Is it possible to link to the file directly (as a static file) on the web site? Alternatively, can you post the file on an FTP server somewhere and generate links to that file? Both of those would probably be much more performant than involving MVC.
KeFang Chen ...
Star
8329 Points
852 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Aug 06, 2009 03:20 AM|LINK
Hello,
From your post I find that you wanted to download large file by using HTTP protocol in MVC. Based on my experience, if we place files in a separate file server, it's much better for improving the performance of our website. And of cource we can create ftp server or bt server for files. I am sure it's good for our sites.
setiri
Member
28 Points
14 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Jun 14, 2010 06:43 PM|LINK
A likely problem is that the server is caching the whole file content before sending it. I read on another board that the filestreamresult normally caches all the stream data before flushing. I'll post a link if i can find it.
setiri
Member
28 Points
14 Posts
Re: Downloading Large Files (4.7GB) with MVC - CPU 100%
Jun 14, 2010 06:44 PM|LINK
see if this post helps any:
http://forums.asp.net/t/1408527.aspx