I'm exporting some database rows into CSV files. I've been doing this always without an issue for several projects in the past. (on previous server) And, now, at local, it's totally fine. All 3100+ records are successfully exported.
But, when I tested the export (on new cloud server), it seems that content is cut off at 3067th record with that last record only having partial data. One column's value is cut off at that same point everytime I tried. (No exception was raised tho)
I'm guessing it's something to do with server configuration such as execution time or memory buffer or something.
Please give advice and help me. Totally don't know how to trace the cause.
You are giving us nothing to work with. Your problem is clearly explained but I find it very difficult for anyone to help you with just that. Show the code you use for creating the CSV. Tell us how you are examining the data and how you are extracting
it from the server (I suppose a download, but let us know so we don't have to guess).
Also examine the data at the breaking record. What does it have different to the others? If you can't find the difference, then post the record here as it may very well be valuable information.
Finally, what "cloud server"? Did you contract Azure, or are you talking about a different cloud service?
Thanks for the response. This is the code to prodce the CSV from DataTable.
Public Sub ExportCSV(ByVal dt As DataTable, ByVal filePath As String)
Dim output As New StreamWriter(filePath, False)
Dim iColCount As Integer = dt.Columns.Count
For i As Integer = 0 To iColCount - 1
output.Write(dt.Columns(i))
If i < iColCount - 1 Then
output.Write(",")
End If
Next
output.Write(output.NewLine)
For Each dr As DataRow In dt.Rows
For i As Integer = 0 To iColCount - 1
If Not Convert.IsDBNull(dr(i)) Then
output.Write("""" & dr(i).ToString & """")
End If
If i < iColCount - 1 Then
output.Write(",")
End If
Next
output.Write(output.NewLine)
Next
output.Close()
End Sub
That works flawlessly on local connected to server database. Following is the download code:
Public Sub PushDownload(ByVal pageLoc As Page, ByVal filePath As String, ByVal contentType As String)
Dim fileIF As New FileInfo(filePath)
If fileIF.Exists Then
pageLoc.Response.ClearContent()
pageLoc.Response.AddHeader("Content-Disposition", "attachment; filename=" & fileIF.Name)
pageLoc.Response.AddHeader("Content-Length", fileIF.Length.ToString())
pageLoc.Response.ContentType = contentType
Dim dlFile As Byte() = StreamFile(fileIF.FullName)
pageLoc.Response.OutputStream.Write(dlFile, 0, Convert.ToInt32(dlFile.Length))
pageLoc.Response.Flush()
pageLoc.Response.Close()
HttpContext.Current.ApplicationInstance.CompleteRequest()
fileIF.Delete()
End If
End Sub
These are the only two main functions in question. I've already looked at the breaking record. It is not any different from others at all. Just a normal record. (I can even assure that it's got nothing to do with unexpected commas. That's already taken care
of.)
The record contains personal particular and I can't really post it here. I'm sorry about that.
And, by "Cloud Server", I did not mean Azure. Sorry if it caused confusion. I'm just saying it's a new server on cloud infrastructure. (I wanted to highlight the fact that same codes worked on old server without any issue. But on new server, it's causing
such problem.)
Hope I provide more meaningful info. Please help me.
Well, exporting code looks OK to me so since you are unable to show the data I guess it will be up to you to determine the problem. Write a log file (see about log4net) and make sure you log the total number of records in the DataTable object to ensure
is the expected number of records. Also make sure your ForEach statement is running through all the records in the DataSet and that none are being skipped. Finally make sure you use a simple tool to verify the server's output. I would use Notepad to check
the CSV as opposed to, say, MS Excel or MS Access.
mgthantzin
Member
20 Points
11 Posts
Content of exported CSV File cut off
Feb 20, 2013 02:27 AM|LINK
Hi all
I'm exporting some database rows into CSV files. I've been doing this always without an issue for several projects in the past. (on previous server) And, now, at local, it's totally fine. All 3100+ records are successfully exported.
But, when I tested the export (on new cloud server), it seems that content is cut off at 3067th record with that last record only having partial data. One column's value is cut off at that same point everytime I tried. (No exception was raised tho)
I'm guessing it's something to do with server configuration such as execution time or memory buffer or something.
Please give advice and help me. Totally don't know how to trace the cause.
Regards
Thant Zin
webJose
Member
730 Points
186 Posts
Re: Content of exported CSV File cut off
Feb 20, 2013 02:57 AM|LINK
You are giving us nothing to work with. Your problem is clearly explained but I find it very difficult for anyone to help you with just that. Show the code you use for creating the CSV. Tell us how you are examining the data and how you are extracting it from the server (I suppose a download, but let us know so we don't have to guess).
Also examine the data at the breaking record. What does it have different to the others? If you can't find the difference, then post the record here as it may very well be valuable information.
Finally, what "cloud server"? Did you contract Azure, or are you talking about a different cloud service?
mgthantzin
Member
20 Points
11 Posts
Re: Content of exported CSV File cut off
Feb 21, 2013 02:32 AM|LINK
Hi webJose
Thanks for the response. This is the code to prodce the CSV from DataTable.
Public Sub ExportCSV(ByVal dt As DataTable, ByVal filePath As String) Dim output As New StreamWriter(filePath, False) Dim iColCount As Integer = dt.Columns.Count For i As Integer = 0 To iColCount - 1 output.Write(dt.Columns(i)) If i < iColCount - 1 Then output.Write(",") End If Next output.Write(output.NewLine) For Each dr As DataRow In dt.Rows For i As Integer = 0 To iColCount - 1 If Not Convert.IsDBNull(dr(i)) Then output.Write("""" & dr(i).ToString & """") End If If i < iColCount - 1 Then output.Write(",") End If Next output.Write(output.NewLine) Next output.Close() End SubThat works flawlessly on local connected to server database. Following is the download code:
Public Sub PushDownload(ByVal pageLoc As Page, ByVal filePath As String, ByVal contentType As String) Dim fileIF As New FileInfo(filePath) If fileIF.Exists Then pageLoc.Response.ClearContent() pageLoc.Response.AddHeader("Content-Disposition", "attachment; filename=" & fileIF.Name) pageLoc.Response.AddHeader("Content-Length", fileIF.Length.ToString()) pageLoc.Response.ContentType = contentType Dim dlFile As Byte() = StreamFile(fileIF.FullName) pageLoc.Response.OutputStream.Write(dlFile, 0, Convert.ToInt32(dlFile.Length)) pageLoc.Response.Flush() pageLoc.Response.Close() HttpContext.Current.ApplicationInstance.CompleteRequest() fileIF.Delete() End If End SubThese are the only two main functions in question. I've already looked at the breaking record. It is not any different from others at all. Just a normal record. (I can even assure that it's got nothing to do with unexpected commas. That's already taken care of.)
The record contains personal particular and I can't really post it here. I'm sorry about that.
And, by "Cloud Server", I did not mean Azure. Sorry if it caused confusion. I'm just saying it's a new server on cloud infrastructure. (I wanted to highlight the fact that same codes worked on old server without any issue. But on new server, it's causing such problem.)
Hope I provide more meaningful info. Please help me.
Thanks in advance.
Regards
Thant Zin
webJose
Member
730 Points
186 Posts
Re: Content of exported CSV File cut off
Feb 21, 2013 05:16 AM|LINK
Well, exporting code looks OK to me so since you are unable to show the data I guess it will be up to you to determine the problem. Write a log file (see about log4net) and make sure you log the total number of records in the DataTable object to ensure is the expected number of records. Also make sure your ForEach statement is running through all the records in the DataSet and that none are being skipped. Finally make sure you use a simple tool to verify the server's output. I would use Notepad to check the CSV as opposed to, say, MS Excel or MS Access.
mgthantzin
Member
20 Points
11 Posts
Re: Content of exported CSV File cut off
Feb 22, 2013 12:31 AM|LINK
Thank you for the suggestions. I will try.
Regards
Zin
mgthantzin
Member
20 Points
11 Posts
Re: Content of exported CSV File cut off
Feb 22, 2013 02:51 AM|LINK
I've fixed the problem by changing
to
And also removed:
It's working fine now.