[Video] What 15 data analysts have to say about web data

More than fifteen data scientists, data engineers, MLOps engineers, and data analysts recently came together to talk about the latest challenges and opportunities in web data and why Bright Data’s solutions are a real game changer.
Noah Kalson
Noah Kalson | Director of Brand Marketing

One especially noteworthy data town hall discussion we’re featuring here, led by Ken Jee, centered around the data influencers’ perspectives on web data, along with an enthusiastic Q&A taken live from Youtube viewers.

Mark Freeman, a startup data scientist, started the discussion by delving into what he called the “murkiness” of retrieving good web data without using a data collection tool like Bright Data.

“I’ve had projects where we’re trying to build the MVP of something and just dealing with where we find the data is a huge challenge”

Internet data influencer and engineer Shashank Kalanithi continued on data’s common pain points:

“For me, the hardest part of [any project] is getting good data. I tried to scrape a number of job listing sites for real time data and I got blocked almost every time.  It’s a shame that public web data isn’t always easily accessible.” 

Notable MLOps Engineer, Miki Bazeley, continued to explain the technical challenges that exist in web data extraction today, and why solutions like Bright Data are so exciting. “Especially if you’re an enterprise company or you’re a startup and you’re trying to move fast, one of the best ways to move fast is to not have to move slow because you’re constantly fixing things.”

Open Discussion on Web Data Collection
Open Discussion on Web Data Collection #BrightDataHouse

Zach Wilson, of Youtube channel @Datawithzack, likewise described the benefits of an automated tool like Bright Data with great excitement.

One of the very common things that companies can do with web data is they want to get better insights into their competitors. If you have a competitive platform… it’s not like your competitors are going to give you that data, one call away. They’re going to make it a pain, and that’s where web data can be really really powerful. Because of the nature of the contracts, your scraper is probably going to break. That’s what I really like about Bright Data. It can validate your scrape.”

The group, in agreement, then further delved into Bright Data’s nearly-impeccable solutions. “It can be really valuable to outsource… and again that’s one of those really cool things about Bright Data… They use an incredible proxy system,” spoke Ken Jee. “If you have a company and your product or your models are reliant upon outside data, that can be really unreliable. It’s very hard to build infrastructure around that if it’s constantly changing. Obviously Bright Data is the best solution because they’re maintaining it and you’re essentially outsourcing it to them.”

To which Ken then asked the group, “Is that the best solution?” 

The group of fifteen-plus top data influencers in the world agreed. Indeed, Bright Data is today’s most exciting data solution. Be sure to watch the full video to see the entire “data avengers” discussion, along with their exciting take on Bright Data’s toolbox of offerings!

Noah Kalson
Noah Kalson | Director of Brand Marketing

Noah oversees the brand marketing strategy at Bright Data ensuring that all marketing initiatives reflect the brand's core values. Noah has a strong background in organic marketing which helped him develop a holistic approach to marketing, branding and communication.

You might also be interested in

What is data aggregation

Data Aggregation – Definition, Use Cases, and Challenges

This blog post will teach you everything you need to know about data aggregation. Here, you will see what data aggregation is, where it is used, what benefits it can bring, and what obstacles it involves.
What is a data parser featured image

What Is Data Parsing? Definition, Benefits, and Challenges

In this article, you will learn everything you need to know about data parsing. In detail, you will learn what data parsing is, why it is so important, and what is the best way to approach it.
What is a web crawler featured image

What is a Web Crawler?

Web crawlers are a critical part of the infrastructure of the Internet. In this article, we will discuss: Web Crawler Definition A web crawler is a software robot that scans the internet and downloads the data it finds. Most web crawlers are operated by search engines like Google, Bing, Baidu, and DuckDuckGo. Search engines apply […]

A Hands-On Guide to Web Scraping in R

In this tutorial, we’ll go through all the steps involved in web scraping in R with rvest with the goal of extracting product reviews from one publicly accessible URL from Amazon’s website.

The Ultimate Web Scraping With C# Guide

In this tutorial, you will learn how to build a web scraper in C#. In detail, you will see how to perform an HTTP request to download the web page you want to scrape, select HTML elements from its DOM tree, and extract data from them.
Javascript and node.js web scraping guide image

Web Scraping With JavaScript and Node.JS

We will cover why frontend JavaScript isn’t the best option for web scraping and will teach you how to build a Node.js scraper from scratch.
Web scraping with JSoup

Web Scraping in Java With Jsoup: A Step-By-Step Guide

Learn to perform web scraping with Jsoup in Java to automatically extract all data from an entire website.
Static vs. Rotating Proxies

Static vs Rotating Proxies: Detailed Comparison

Proxies play an important role in enabling businesses to conduct critical web research.