Why you should consider using the .net environment for your next scraping task
Many dot net programmers may before of heard of the practice of web scraping, a few may have even used it in other programming ecosystems, however I would guess that few have ever done any kind of web scraping in a dot net language. I have come to this conclusion because of the lack of resources out there for web scraping with c# or vb, and an abundance of them for languages like python or R. I can’t help but feel that dot net is being sold short here…
The reason for the popularity of web scraping in python is largely down to the number of high quality libraries which exist to help with the task and the large amount of documentation and help out there for it already. When you look up how to scrape, python tends to be the first language you stumble across, for this reason I believe for most this will make it the scraping language of choice and possibly wrongly so!
Inversely c# and vb can be hard to find any good scraping content for. There are however a number of very strong and mature libraries which will help you achieve your goal. So I can only conclude that its lack of popularity is due to the lack of documentation online about the subject.
I would like to put forward to you that when you look towards performing your next web scraping task that you should consider using c# or vb. The dot net framework is arguably the most robust set of libraries out there. Combine that with some good tooling to help with scraping and you have one of the best platforms to work off of.
If you need any help getting started have a look at this introductory c# web scraping tutorial. It covers the basic tooling and techniques needed.
