Ok, I have some very preliminary results from experimenting with the ideas presented above. I build an HttpModule to process the upload in chunks successfully. I used a 64K buffer. It made a huge difference vesus using the .NET classes to pull in the file.
No more loading the entire file's content in memory!!! But, that leaves more work for us.... If you have an incoming http request and you capture it using the BeginRequest event in the ASP.NET pipeline and you access the HttpRequest object (from the Context
object) in any way, ASP.NET will load the entire request's content, regardless if it is a multipart request or not, into memory. I experimented with various ways before attempting to use the HttpWorkerRequest class. Bottom line, if you access even one property
of the HttpRequest object, you're getting everything all at once. Ok, so using the methods presented above, I was able to easily snag the httpWorkerRequest interface and exploit it. Indeed you have to use the ReadEntityBody method, but not exactly in the way
you might think. When a multipart HTTP request comes in, the first pice of the request is "preloaded." The ReadEntutyBody method is only good for the portion of the request that is NOT preloaded. Therefore every HTTP request that comes in has some or all of
its content preloaded. So you use the GetPreloadedEntityBody method to snag the preloaded portion of the request. The you use the IsEntireEntityBodyIsPreloaded method to check if you have the entire request or not. If not, then you can use the ReadEntityBody
method to finish reading the request. FYI - the preloaded portion of the request is always 49152 bytes on my system. So you chunk-in the rest of the request writing the chunk to disk and get th desired solution to the original problem presented in this thread.
But.....there's always a BUT! We're now talking about a raw HTTP request. (As if you didn't see that coming...) So now we need a reliable method of parsing the request as it comes in such that we are able to detect that an HTTP frame contains a file and we
are only writing that file's contents to disk. All in all, it shouldn't be too bad. I don't have time tonight to write a parser! I have real work to do, but this thread was very interesting! I wrote my code in VB.NET (sorry, C# dudes). Luckily, I code in the
framework, not like a typical VB programmer. So I'll place the code in the next reply. CodeRage!
Ok, here's the html file tha has the form: <form name="UploadForm" method="post" encoding="multipart/form-data"> File to Upload: <input type="file" name="file">
<input type="submit" value="Submit"> </form> Ok, now add the following to your Web.config file inside the element: This configures the timeout to 30 minutes, accepts up to 500MB files, and registers the HttpModule. My assembly name is HttpUploadApp as the namespace
is the same thing. Next is the HttpModule class. This is a rough example that writes every HTTP request that has a content body to a file whose name is a GUID with a ".txt" extension saved in a directory named "C:\Upload\". This was just for test purposes!!!
HttpModule class (VB.NET): Imports System Imports System.IO Imports System.Web Public Class UploadModule Implements IHttpModule Public Sub New() MyBase.New() End Sub Public Sub Dispose() Implements System.Web.IHttpModule.Dispose End Sub Public Sub Init(ByVal
context As System.Web.HttpApplication) Implements System.Web.IHttpModule.Init AddHandler context.BeginRequest, AddressOf BeginRequest End Sub Private Sub BeginRequest(ByVal source As Object, ByVal e As EventArgs) Dim app As HttpApplication = CType(source,
HttpApplication) Dim context As HttpContext = app.Context ' Get the HttpWorkerRequest Object!!! Dim hwr As HttpWorkerRequest = CType(context.GetType.GetProperty("WorkerRequest", Reflection.BindingFlags.Instance Or Reflection.BindingFlags.NonPublic).GetValue(context,
Nothing), HttpWorkerRequest) If hwr.HasEntityBody Then Dim g As Guid = Guid.NewGuid() Dim fn As String = "C:\Upload\" + g.ToString + ".txt" Dim fs As New FileStream(fn, FileMode.CreateNew, FileAccess.Write, FileShare.Read) Try Const BUFFSIZE As Integer = 65536
Dim Buffer As Byte() Dim Received As Integer = 0 Dim TotalReceived As Integer = 0 Dim ContentLength As Integer = CType(hwr.GetKnownRequestHeader(HttpWorkerRequest.HeaderContentLength), Integer) Buffer = hwr.GetPreloadedEntityBody() fs.Write(Buffer, 0, Buffer.Length)
Received = Buffer.Length TotalReceived += Received If Not hwr.IsEntireEntityBodyIsPreloaded Then While (ContentLength - TotalReceived) >= Received Buffer = New Byte(BUFFSIZE) {} Received = hwr.ReadEntityBody(Buffer, BUFFSIZE) fs.Write(Buffer, 0, Received)
TotalReceived += Received End While Received = hwr.ReadEntityBody(Buffer, (ContentLength - TotalReceived)) fs.Write(Buffer, 0, Received) TotalReceived += Received End If fs.Flush() Catch ex As Exception Finally fs.Close() End Try context.Response.Redirect("UploadForm.aspx")
End If End Sub End Class Ok, that's it. What I didn't address here was keeping th aspnet_wp.exe process from recyling itself by opening another client-side browser winfow that loads an .aspx page with a client-side (JavaScript) timer refreshing itself every
10 seconds or so. Anyone can write that code --- it's so simple!!! CodeRage!
YAHOO! This is just wonderful CodeRage! Your solution to redirect after upload is perfect! That solves the problem of signalling the rest of the pipeline. Yes, the http must be parsed. But as you point out, this is no big deal compared to what we've got not.
WOW! Was that nice or what? We're now able to do REAL uploading, and besides from the abcupload.net team, this must be valuable information to a lot of people! For me, exams are now over, THIS NUT HAS BEEN CRACKED, and the weather is not too cold...this has
really made my day, and many to come :-) Kind Regards, Rasmus
I did everything you said but every time I post I get the following error: Maximum request length exceeded. Exception Details: System.Web.HttpException: Maximum request length exceeded. Stack Trace: [HttpException (0x80004005): Maximum request length exceeded.]
System.Web.HttpRequest.GetEntireRawContent() +891 System.Web.HttpRequest.FillInFormCollection() +129 System.Web.HttpRequest.get_Form() +50 System.Web.TraceContext.InitRequest() +703 System.Web.TraceContext.VerifyStart() +160 System.Web.HttpApplication.System.Web.IHttpAsyncHandler.BeginProcessRequest(HttpContext
context, AsyncCallback cb, Object extraData) +216 System.Web.HttpRuntime.ProcessRequestInternal(HttpWorkerRequest wr) +479 I even modified the httpRuntime key in machine.config to have the value 524288000 and still get the same result.
Ok, boys I found your posts last week and started working on your problem. I hereby post you my C# code, since the code above is written in some sort of VB.net code :) I'm working on the rest of the story now to pipe everything from the request to the iis but
after I cut out the file from the request. In order not to do any redirection anymore. I hope you find this usefull : using System; using System.Web; using System.IO; using System.Reflection; namespace My.PlayGround { /// /// Summary description for UploadHandler.
/// public class UploadHandler : IHttpModule { public UploadHandler() { } public void Init(HttpApplication application) { // Register our event handler with with the application object application.BeginRequest += new EventHandler(this.BeginRequest); } public
void Dispose() {} public void BeginRequest(object sender, EventArgs args) { try { // Create an instance of th application object HttpApplication application = (HttpApplication) sender; // Create an instance of the HTTP worker request HttpWorkerRequest request
= (HttpWorkerRequest) application.Context.GetType().GetProperty("WorkerRequest", (BindingFlags)36).GetValue(application.Context, null); // Only trigger if the request is of type 'multipart/form-data' if(application.Context.Request.ContentType.IndexOf("multipart/form-data")
> -1) { // Create a new unique identifier to identify each request string guid = Guid.NewGuid().ToString(); // Please alter pathname because the root of the C drive is not really the place to put your tempfiles string filename = "c:\\requests\\request_" +
guid + ".txt"; // Check if a request body is sent by the browser if(request.HasEntityBody()) { // Get the content length of the request int content_length = Convert.ToInt32(request.GetKnownRequestHeader(HttpWorkerRequest.HeaderContentLength)); int content_received
= 0; // This is a nice feature to redirect users if they upload a file // larger then 100Mb BEFORE it is being uploaded if(content_length > 102400000) { application.Context.Response.Redirect("http://www.av.com"); } // Create a file to store the stream FileStream
newFile = new FileStream(filename, FileMode.Create); // Get the preloaded buffered data byte[] body = request.GetPreloadedEntityBody(); content_received += body.Length; // Write the preloaded data to new file newFile.Write(body, 0, body.Length); // Get all
the other data to be written to file if available if(!request.IsEntireEntityBodyIsPreloaded()) { // Create an input buffer to store the incomming data byte[] a_buffer = new byte[16384]; int bytes_read = 16384; while((content_length - content_received) >= bytes_read)
{ bytes_read = request.ReadEntityBody(a_buffer, a_buffer.Length); content_received += bytes_read; newFile.Write(a_buffer, 0, bytes_read); } // Read the last part of the stream bytes_read = request.ReadEntityBody(a_buffer, (content_length - content_received));
newFile.Write(a_buffer, 0, bytes_read); content_received += bytes_read; } // Flush all data to the file newFile.Flush(); // Close the file newFile.Close(); // Redirect to the page to avoid the browser hanging string current_page = application.Context.Request.CurrentExecutionFilePath;
current_page = current_page.Substring(current_page.LastIndexOf("/")+1) + "?" + application.Context.Request.QueryString; application.Context.Response.Redirect(current_page); } } } catch(System.Threading.ThreadAbortException) {} catch(Exception exception) {
object e = exception; } } } }
Bjron/CodeRage, Wow... Its a great code... Thanks for your code. Here is the thing, once i upload a file(hello.doc) to web server. Its corrupted with .txt extension. Even after giving rite extension. Its getting corrupted with some extra junk something like
this... -----------------------------7d315142b02c8 Content-Disposition: form-data; name="__VIEWSTATE" dDwtMTI3OTMzNDM4NDs7Ppg0hIHAIt5o+GJ8PugQQtOjF3TU -----------------------------7d315142b02c8 Content-Disposition: form-data; name="file"; filename="C:\download\ProgressBar_src.zip"
Content-Type: application/x-zip-compressed HOW DO I CORRECT THIS PROBLEM? Please let me know. Thx, Shan.
Hello. I have implemented the C# code available in this thread and in fact i get the file uploaded. My problem is that i loose at least some of the events that should fire. The upload process is trigered when i click an image button present in a form... Did
anyone has this problem. How can i solve this? Should the ....response.redirect be there in the end? Thank you. Telmo Sá ps: Could you also reply to telmo_sa@hotmail.com please?
Shan: What you are looking at is viewstate, added by asp.net. You can minimize viewstate, but you cannot get completely rid of it, without removing the runat=server attribute from the <form blabla > tag. This should solve your problem, if your form only contains
the file control. If your form contains other controls than just the file control, you have to rewrite all controls to old-style html controls, and add the runat=server to each control. Example: becomes: <input type=text runat=server blablabla> These types
of controls will not complain about not being inside a runat=server form. Kind regards, Rasmus
Telmo: The reason for response.redirect in the end, is exactly to prevent you from running into problems like the one you descripe. I think the problems is this: Once the file is uploaded, you have altered the request. Somewhere asp.net keeps track of the expected
length of the request. After upload, the request no longer has the expected length, as you have removed the file. This will lead asp.net to "hang", waiting for the missing bytes to be supplied by the client. I have not found a way to indicate that the file
has been removed; that is, I have not found the place where the expected request length is kept. So currently you do need to redirect after upload. Note: Abcupload.net does somehow solve this problem. kind regards, Rasmus
ups101, Well. Thanks for your reply. By the way, whats the point if the file got corrupted after Uploading to web server; Say like Once i upload a Doc file(hello.doc), i'm not able Open that file... There should be some way to work around to get rid of those
JUNK values(_ViewState .. blah blah). If anyone know how to solve this problem, let me know. Thx, Shan.
CodeRage
Member
15 Points
3 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Dec 19, 2002 03:55 AM|LINK
CodeRage
Member
15 Points
3 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Dec 19, 2002 04:07 AM|LINK
<input type="submit" value="Submit"> </form> Ok, now add the following to your Web.config file inside the element: This configures the timeout to 30 minutes, accepts up to 500MB files, and registers the HttpModule. My assembly name is HttpUploadApp as the namespace is the same thing. Next is the HttpModule class. This is a rough example that writes every HTTP request that has a content body to a file whose name is a GUID with a ".txt" extension saved in a directory named "C:\Upload\". This was just for test purposes!!! HttpModule class (VB.NET): Imports System Imports System.IO Imports System.Web Public Class UploadModule Implements IHttpModule Public Sub New() MyBase.New() End Sub Public Sub Dispose() Implements System.Web.IHttpModule.Dispose End Sub Public Sub Init(ByVal context As System.Web.HttpApplication) Implements System.Web.IHttpModule.Init AddHandler context.BeginRequest, AddressOf BeginRequest End Sub Private Sub BeginRequest(ByVal source As Object, ByVal e As EventArgs) Dim app As HttpApplication = CType(source, HttpApplication) Dim context As HttpContext = app.Context ' Get the HttpWorkerRequest Object!!! Dim hwr As HttpWorkerRequest = CType(context.GetType.GetProperty("WorkerRequest", Reflection.BindingFlags.Instance Or Reflection.BindingFlags.NonPublic).GetValue(context, Nothing), HttpWorkerRequest) If hwr.HasEntityBody Then Dim g As Guid = Guid.NewGuid() Dim fn As String = "C:\Upload\" + g.ToString + ".txt" Dim fs As New FileStream(fn, FileMode.CreateNew, FileAccess.Write, FileShare.Read) Try Const BUFFSIZE As Integer = 65536 Dim Buffer As Byte() Dim Received As Integer = 0 Dim TotalReceived As Integer = 0 Dim ContentLength As Integer = CType(hwr.GetKnownRequestHeader(HttpWorkerRequest.HeaderContentLength), Integer) Buffer = hwr.GetPreloadedEntityBody() fs.Write(Buffer, 0, Buffer.Length) Received = Buffer.Length TotalReceived += Received If Not hwr.IsEntireEntityBodyIsPreloaded Then While (ContentLength - TotalReceived) >= Received Buffer = New Byte(BUFFSIZE) {} Received = hwr.ReadEntityBody(Buffer, BUFFSIZE) fs.Write(Buffer, 0, Received) TotalReceived += Received End While Received = hwr.ReadEntityBody(Buffer, (ContentLength - TotalReceived)) fs.Write(Buffer, 0, Received) TotalReceived += Received End If fs.Flush() Catch ex As Exception Finally fs.Close() End Try context.Response.Redirect("UploadForm.aspx") End If End Sub End Class Ok, that's it. What I didn't address here was keeping th aspnet_wp.exe process from recyling itself by opening another client-side browser winfow that loads an .aspx page with a client-side (JavaScript) timer refreshing itself every 10 seconds or so. Anyone can write that code --- it's so simple!!! CodeRage!
ups101
Member
205 Points
41 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Dec 19, 2002 08:56 AM|LINK
Cyberfloatie
Member
102 Points
29 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Dec 20, 2002 04:49 AM|LINK
Bjorn.B
Member
10 Points
2 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Dec 20, 2002 08:27 AM|LINK
ShanL
Member
10 Points
2 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Jan 03, 2003 04:16 PM|LINK
telmo.sa
Member
5 Points
1 Post
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Jan 03, 2003 04:30 PM|LINK
ups101
Member
205 Points
41 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Jan 05, 2003 05:30 PM|LINK
ups101
Member
205 Points
41 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Jan 05, 2003 05:40 PM|LINK
ShanL
Member
10 Points
2 Posts
Re: HttpHandler or HttpModule for file upload, large files, progress indicator?
Jan 06, 2003 01:18 PM|LINK