I have an upload script that splits large files into chunks. This is great and all working fine, but, once uploaded, how do I join them together to recreate the original file, either using C# or a .NET component?
I'm using a jQuery uploader with Persits AspUpload component. For every 50mb of data being uploaded the jQuery script passes that data to AspUpload which then stores it until the whole file is uploaded. So a 500mb file would be uploaded in 10 chunks of data
(each data file is given a unique name and logged in a database). I'm not sure what the data type is, it's not set to store as binary so I'm assuming it's just the original file format split into chunks... Sorry if that's not enough information.
The ASP component isn't actually splitting the files as such. Assuming the file is over 50mb, the jQuery calculates when the 50mb mark is reached then passes the data to AspUpload which then stores it. I'm just left with x amounts of upload chunks.
I have the option to turn uploading in chunks off, but for large files (multiple gigbytes) it's essential.
I agree, but I'm providing a file encryption service (using persits AspUpload and AspEncrypt), so it needs to be http based as the files are actually encrypted as they're being uploaded (all file types are allowed), so should someone upload a virus it would
be encrypted before being saved to the server.
I've used to the uploader to upload a 9GB file (in chunks) successfully. I don't expect many people to use the service for files of that magnitude (pricing varies depending on how large the file is, by default anything 100mb or less is free), but it's a
good USP to offer 10GB uploads.
I've found a solution in the form of FFSJ.
I can join files through command line Parameters. The only problem is it's very, very slow and uses a lot of memory. If anyone knows of a similar, more efficient app, I'd greatly appreciate it.
Stoink
0 Points
9 Posts
Uploaded a file in chunks... now how to join them?
Apr 27, 2012 02:16 PM|LINK
Hi,
I have an upload script that splits large files into chunks. This is great and all working fine, but, once uploaded, how do I join them together to recreate the original file, either using C# or a .NET component?
Any help is much appreciated.
Thanks.
Curt_C
All-Star
66014 Points
7639 Posts
Moderator
Re: Uploaded a file in chunks... now how to join them?
Apr 27, 2012 02:17 PM|LINK
There is no way we can answer without more information. How are you splitting these files? What's the data look like?
v5.1 of iTracker (Inventory Tracker Starter Kit) is out, Download it now!
Stoink
0 Points
9 Posts
Re: Uploaded a file in chunks... now how to join them?
Apr 27, 2012 02:41 PM|LINK
I'm using a jQuery uploader with Persits AspUpload component. For every 50mb of data being uploaded the jQuery script passes that data to AspUpload which then stores it until the whole file is uploaded. So a 500mb file would be uploaded in 10 chunks of data (each data file is given a unique name and logged in a database). I'm not sure what the data type is, it's not set to store as binary so I'm assuming it's just the original file format split into chunks... Sorry if that's not enough information.
Curt_C
All-Star
66014 Points
7639 Posts
Moderator
Re: Uploaded a file in chunks... now how to join them?
Apr 27, 2012 02:52 PM|LINK
I'm unfamiliar with your methodology but if there is a method to split the data in a component there is surely a way to join the data built in.
v5.1 of iTracker (Inventory Tracker Starter Kit) is out, Download it now!
Stoink
0 Points
9 Posts
Re: Uploaded a file in chunks... now how to join them?
Apr 27, 2012 03:28 PM|LINK
The ASP component isn't actually splitting the files as such. Assuming the file is over 50mb, the jQuery calculates when the 50mb mark is reached then passes the data to AspUpload which then stores it. I'm just left with x amounts of upload chunks.
I have the option to turn uploading in chunks off, but for large files (multiple gigbytes) it's essential.
Curt_C
All-Star
66014 Points
7639 Posts
Moderator
Re: Uploaded a file in chunks... now how to join them?
Apr 30, 2012 12:46 PM|LINK
if you have files that large you shouldn't be using an HTTP based file upload, it's not stable enough. You will want to use an FTP based methodology.
v5.1 of iTracker (Inventory Tracker Starter Kit) is out, Download it now!
Stoink
0 Points
9 Posts
Re: Uploaded a file in chunks... now how to join them?
Apr 30, 2012 01:59 PM|LINK
I agree, but I'm providing a file encryption service (using persits AspUpload and AspEncrypt), so it needs to be http based as the files are actually encrypted as they're being uploaded (all file types are allowed), so should someone upload a virus it would be encrypted before being saved to the server.
I've used to the uploader to upload a 9GB file (in chunks) successfully. I don't expect many people to use the service for files of that magnitude (pricing varies depending on how large the file is, by default anything 100mb or less is free), but it's a good USP to offer 10GB uploads.
I've found a solution in the form of FFSJ.
I can join files through command line Parameters. The only problem is it's very, very slow and uses a lot of memory. If anyone knows of a similar, more efficient app, I'd greatly appreciate it.
Stoink
0 Points
9 Posts
Re: Uploaded a file in chunks... now how to join them?
May 02, 2012 10:55 AM|LINK
So there's no need to use a component or exe like FFSJ to join files. Windows can do it using a CMD command prompt.
For example:
cd c:\websites\mywebsite\data\encrypted\ && copy/b encrypted.001 +encrypted.002 +encrypted.003 +encrypted.004 +encrypted.005 +encrypted.006 +encrypted.007 +encrypted.008 +encrypted.009 +encrypted.010 encrypted.rar
This will join encrypted.001 to encrypted.010 and save it as encrypted.rar, it's fairly fast and not CPU or Memory heavy at all.
This works fine when run in the standard CMD, but when trying to run it from code using WScript.Shell I get a '80070002' permission denied error.
User permissions to the folder containing the files are all correct (to my knowledge), any suggestions?
Thanks
Stoink
0 Points
9 Posts
Re: Uploaded a file in chunks... now how to join them?
May 02, 2012 11:06 AM|LINK
Sorry, did a bit more googling and found the answer :)