I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries likeanglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic
website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).
I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries likeanglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic
website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).
If you need to scrape a website you can use ScrapySharp scraping framework. You can add it to a project as a nuget.
IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
Member
4 Points
9 Posts
Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 17, 2020 08:36 PM|armanes|LINK
I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries like
anglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).Does C # have a good library for this purpose?
Participant
1660 Points
952 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 12:35 AM|PaulTheSmith|LINK
What does "not working dynamic website" mean? What problem do you have?
Contributor
3370 Points
1409 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 04:02 AM|samwu|LINK
Hi aemanes,
If you need to scrape a website you can use ScrapySharp scraping framework. You can add it to a project as a nuget.
https://www.nuget.org/packages/ScrapySharp/
You can also try to use WebScraping Library. WebScraper provides a powerful framework to extract data and files from websites using C# code.
https://ironsoftware.com/csharp/webscraper/
Best regards,
Sam
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 07:48 AM|armanes|LINK
IronWebscraper has a license , scrapySharp working on the dynamic website?
Contributor
3370 Points
1409 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 02:25 AM|samwu|LINK
Hi armanes,
Here some demo about how to Scrape Data from Websites in C#.
https://ironsoftware.com/csharp/webscraper/tutorials/webscraping-in-c-sharp/#helpscout-support
Best regards,
Sam
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 08:13 AM|armanes|LINK
thank you for giving me information. I want scrap youtube Channels, can IronWebScraper click on elements or page scroll down (for extract all videos)?
All-Star
53631 Points
23985 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 11:11 AM|mgebhard|LINK
Why are you screen scraping? Why don't you simply create a developer account and take advantage if the youTube API? What is your goal?
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:24 PM|armanes|LINK
My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)
All-Star
53631 Points
23985 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:30 PM|mgebhard|LINK
Then I recommend learning how to use the youTube REST API. https://developers.google.com/youtube/v3
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:47 PM|armanes|LINK
Thank you very much