The VP/C-suite guide to web data

Understand what web data is and how it can enable a newfound competitive advantage
The VP-C-suite guide to web data
Omri Orgad
Omri Orgad | Managing Director, North America
27-Jan-2022
Share:

Web data now answering strategic questions 

Someone in a senior position gets paid to make high-quality decisions that produce beneficial business outcomes, reflecting positively on the company and the individual’s professional trajectory. 

But smart decisions which usually come in the form of questions can only be made based on the most reliable, up-to-date information. 

This is where web data steps in. 

If we use the digital retail space as an example, some of the biggest day-to-day operational questions can be answered using web data.

Such as: 

What are my competitor’s best selling products?’

The data points that can best help answer this question are:

  • Competitor inventory monitoring, such as collecting data on which Stock Keeping Units (SKUs) have the highest Sell-Through Rates (STRs). SKUs include additional product attributes such as color, manufacturer, model, and warranty terms. This means that businesses can also look for correlations between one of these attributes, and above-average STRs.
  • Positive customer ratings, and consumer sentiment on reviews indicating that the item’s quality, and delivery was satisfactory as far as consumers are concerned.
  • High positioning in marketplace search results indicating that the marketplace’s algorithm identifies this product as being in high demand, and the seller as reliable.

What dynamic pricing strategy will help us corner our market?’

Dynamic pricing doesn’t necessarily mean lowering your pricing but rather customizing it to target consumers. Relevant data points in this respect include:

  • Pricing items based on the zip code a prospect is accessing your site from. Serving higher prices to people living in more well-to-do areas, and lower prices to folks who live in less well-off places.
  • Supply, and demand data indicating when competitors’ inventory levels are low, and demand is high (this is when prices should be raised) or when market inventory is abundant across-the-board and demand is low/moderate (this is when prices should be lowered). 

How do I perform product matching for each individual product?’

Relevant data points include: 

  • Cross-referencing item specifics in order to confirm two item’s compatibility.
  • Comparing listing titles – specifically looking at title length, keywords included, and sentence structure to determine if you are comparing oranges to oranges.  
  • Seeing how item visuals match up in terms of angles, styling, and quantity of images.

Competitors intentionally mesh-up these product identifiers in order to make it harder to compete, once data is used to dispel these smoke clouds, competing becomes much more straightforward. 

Which categories, and brands are trending in specific geolocations?’

Relevant data points in this regard would be:

  • Shipping data indicating what percentage of competitor sales are being sent to which regions, countries, and cities. 
  • Language detection including scanning the web for localization efforts by competitors, e.g. identifying an English language listing translated into French, German, and Japanese.

Where are there customer satisfaction gaps we can capitalize on?’

Using Natural Language Processing (NLP), text-heavy data points can be mined for consumer sentiment on specific items or broader solutions using data points such as:

  • How people are reacting to an item on social media either on a video product review on YouTube, a Facebook post with personal experiences or a Reddit thread discussing a product category.
  • Customer reviews either on or off the marketplace where items are being sold. This is especially true of customers’ reactions to competitor products. Typically one or more specific points of contention can be picked up, and capitalized on. E.g. Slow shipping, lack of customer support, missing parts etc. 

How do I keep track of competitor product offers that are constantly changing? ’

Relevant data points include:

  • Scanning competitor stores, websites, and other Points of Sale (PoS) in real-time, discovering new products and categories that your company can start competing with/in. 
  • Collecting information on publicly available product catalogs so that your business can receive alerts when a new product is introduced into the inventory pool.  

But obtaining web data is not so easy 

The sad truth however is that actually being able to obtain customer/competitor web data in real-time, and in a structured format that can be utilized by algorithms immediately is not as simple as it sounds. 

To accomplish this, professionals will need:

  • Infrastructure (hardware and software) such as cloud servers, networks, Application Programming Interfaces (APIs), and regular code changes based on target site architecture changes. 
  • Technical teams including an extensive group of engineers as well as IT, and DevOps personnel.
  • Data preparation as information is typically collected in its most basic, unstructured form. It typically needs further processing including removing duplicates, cleaning corrupted files, enriching datasets with missing information, and structuring data into a usable CSV or JSON file format.

And yet now there is hope 

In recent months new autonomous data collection tools have been released. These are fully automated Data Collectors that do not require any in-house infrastructure. They build on previously developed data technology including:

  • Artificial Intelligence that takes care  of cleaning, matching, synthesizing, and processing data sets. 
  • Structuring target web data, and formatting it (JSON, CSV, HTML, or XSLS) so that it’s ready-to-use, and can be analyzed. 
  • Machine Learning identifies target site structure changes on an ongoing basis to help avoid blocks, and smoothly retrieve data

All of this means that executives can start answering their business questions using user-generated data collected in real-time, empowering them to make smarter decisions, while creating more meaningful corporate value. 

Omri Orgad
Omri Orgad | Managing Director, North America

A tech-savvy business leader with extended experience working in the network, software, and security industries. Deep familiarization with multiple global markets, serving in several senior leadership technology-based roles. Vast experience in business negotiation, forming strategic profitable partnerships with large-scale enterprises as well as working with start-ups and SMEs accelerating their growth and entering new markets.

Share:

You might also be interested in

If your company has even ONE developer dedicated to web data collection, you are wasting precious resources

The state of the economy in general, and of tech in particular, is leading many CEOs to put budget cut pressure on Information Technology execs. This article aims to help IT leaders improve their bottom lines by offering a more strategic approach to operational web data collection outsourcing

Shooting ourselves in the foot? Why we willingly killed 10% of our network

Bright Data believes in transparent and ethical practices, especially when it comes to dealing with users who make up its Residential peer network. To ensure compliance, we use advanced monitoring protocols and partner with top anti-virus companies. Sometimes, we make decisions which might seem a little crazy, like hurting our own network. That is what this post is about.
Web Data powering e-commerce

Mystery shoppers are so 2000 and late. Web data is the future of e-commerce.

We sat down with Charmagne Cruz from Shopee, the leading e-commerce platform in Southeast Asia, to discuss how the online conglomerate uses public web data to drive forward the company’s success as well as carve out a large section of the Asian e-commerce market.
Qualitative data collection methods

Qualitative data collection methods

Quantitative pertains to numbers such as competitor product fluctuations, while qualitative pertains to the ‘narrative’ such as audience social sentiment regarding a particular brand. This article explains all the key differences between the two, as well as offering tools to quickly and easily obtain target data points