Last post Mar 01, 2019 07:52 PM by deepalgorithm
Aug 30, 2018 12:39 PM|JY_2013|LINK
I'm designing a Web based application that needs to be available as a RESTful Web service to clients over the Internet. The size of the data to be sent as a response can be very huge ranging from 1 GB to about 50 GB. I believe it is not possible to send
such a huge response over as a Web response.
What are some possible ways in which I can send such data across? Are there any industry-standard approaches to sending such data across? Or do I need to share it on some FTP site once such a request is received?
Appreciate any inputs on this.
Sep 21, 2018 06:09 AM|chandradev1|LINK
You can do like this
what type of data are you going to send as response ?
Mar 01, 2019 07:52 PM|deepalgorithm|LINK
APIs that move heavy amounts of data consume a lot of memory to materialize results which can lead to high memory usage, paging, and sluggish performance. With this in mind, you want to implement a streaming approach. Assuming your datasource can stream
results, you can keep more memory available on the server by moving data with
IQueryable to the client.
The projects I'm on that require massive data ingestion leverage services like
Azure Event Hubs. What you decide to do comes down to the architecture and overall system requirements.