I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries likeanglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic
website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).
I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries likeanglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic
website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).
If you need to scrape a website you can use ScrapySharp scraping framework. You can add it to a project as a nuget.
MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue.
If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.
MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue.
If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.
Member
4 Points
9 Posts
Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 17, 2020 08:36 PM|armanes|LINK
I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries like
anglesharp/anglesharp.js(give an error for .net core 3.1),Html Agility Pack(not working dynamic website) and pseudo-Specialized libraryselenium(but this library for UI testing- I don't want to use this library).Does C # have a good library for this purpose?
Participant
1420 Points
816 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 12:35 AM|PaulTheSmith|LINK
What does "not working dynamic website" mean? What problem do you have?
Contributor
3340 Points
1409 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 04:02 AM|samwu|LINK
Hi aemanes,
If you need to scrape a website you can use ScrapySharp scraping framework. You can add it to a project as a nuget.
https://www.nuget.org/packages/ScrapySharp/
You can also try to use WebScraping Library. WebScraper provides a powerful framework to extract data and files from websites using C# code.
https://ironsoftware.com/csharp/webscraper/
Best regards,
Sam
Please remember to click "Mark as Answer" the responses that resolved your issue.
If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 18, 2020 07:48 AM|armanes|LINK
IronWebscraper has a license , scrapySharp working on the dynamic website?
Contributor
3340 Points
1409 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 02:25 AM|samwu|LINK
Hi armanes,
Here some demo about how to Scrape Data from Websites in C#.
https://ironsoftware.com/csharp/webscraper/tutorials/webscraping-in-c-sharp/#helpscout-support
Best regards,
Sam
Please remember to click "Mark as Answer" the responses that resolved your issue.
If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 08:13 AM|armanes|LINK
thank you for giving me information. I want scrap youtube Channels, can IronWebScraper click on elements or page scroll down (for extract all videos)?
All-Star
49781 Points
22106 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 11:11 AM|mgebhard|LINK
Why are you screen scraping? Why don't you simply create a developer account and take advantage if the youTube API? What is your goal?
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:24 PM|armanes|LINK
My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)
All-Star
49781 Points
22106 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:30 PM|mgebhard|LINK
Then I recommend learning how to use the youTube REST API. https://developers.google.com/youtube/v3
Member
4 Points
9 Posts
Re: Can anyone give me a specialized library about scrap dynamic website (js) in C#
Apr 20, 2020 03:47 PM|armanes|LINK
Thank you very much