We are going to develop web page to download large amount of data from 3rd party api. what are the main limitations/areas we need to think about.
we are going to display data in the web page as well as allow client to download data as file. at the moment api support download 1000 records at a time.
Have you tried to see if this API have support for "paging" ? Usually you try to show a user what matters and it's not uncommon for an API to serve a "page" with possibly links to the previous and next "page" of data.
If not just talk directly with those providing this API. They may also have a "term of use" to avoid API abuse etc....
Thanks for your reply Patrick!!. Yes we are planning to do paging. We need to download more than 50,000 records. We are going to display 1000 record per page. When we are downloading file can we download it as single file?
Member
2 Points
27 Posts
Download large amount of data
Jan 16, 2018 03:41 AM|aish|LINK
Hi,
We are going to develop web page to download large amount of data from 3rd party api. what are the main limitations/areas we need to think about.
we are going to display data in the web page as well as allow client to download data as file. at the moment api support download 1000 records at a time.
any reason why we can't increase that number.
Thanks for the help
All-Star
48270 Points
17981 Posts
Re: Download large amount of data
Jan 16, 2018 09:54 AM|PatriceSc|LINK
Hi,
Have you tried to see if this API have support for "paging" ? Usually you try to show a user what matters and it's not uncommon for an API to serve a "page" with possibly links to the previous and next "page" of data.
If not just talk directly with those providing this API. They may also have a "term of use" to avoid API abuse etc....
Member
2 Points
27 Posts
Re: Download large amount of data
Jan 16, 2018 09:17 PM|aish|LINK
Participant
1644 Points
792 Posts
Re: Download large amount of data
Jan 17, 2018 11:14 AM|rajesh93180|LINK
Hello aish,
Yes sure you can download file with all the pages records in single click.
Thanks,
Rajesh
Mark as answer if you find this post helpful.