How web data has led to companies capturing a disproportionate market share

From hedge funds using social sentiment data for more strategic investment decisions to cosmetics brands identifying, and capitalizing on organic traffic gaps utilizing search engine data. Companies across the board are leveraging web data in order to gain a significant informational advantage
5 min read
How web data has led to companies capturing a disproportionate market share

Executive index:

Web data gives businesses an informational advantage 

This happens when companies need to deal with real-world occurrences that have a big impact on their industry, here are 3 examples:

Social sentiment data helps hedge funds increase returns 

Reddit’s Wall Street Bets (WSB) ‘Big-Short Squeeze’, served as a major wake-up call for those in the financial sector. Melvin Capital Management (MCM), was a prime example of an investment management firm that did not utilize social sentiment data, leading to huge losses on GameStop (NYSE: GME). MCM took major ‘short positions’ based on ‘classic corporate financial datasets’ (e.g. debt-to-earnings ratio, and projected annual sales). 

But what they failed to take into account was the social sentiment on the stock. While MCM was busy hedging against GME, WSB investors were publicly discussing their positive outlook on this position. In fact, for a moment in time, they were able to drive the price up almost 200 times above the stock’s historic average valuation. This would be fine if it were not for the fact that MCM portfolio managers were so confident in themselves that they placed hedges in excess of 40% of existing stock volume. What this means is that they were forced to sell at certain price points otherwise their fund would not physically have enough cash to cover their losses and would need to file for bankruptcy. 

The bottom line here is firms that collected social media data were able to identify positive sentiment towards GME enabling them to take preemptive (i.e. selling their shorts) and proactive measures (i.e. ‘buying long’). They crossed the finish line as winners ‘in the green’.  Those that didn’t lost their pants! 

Timber data gives Real Estate Investment Trusts (REITs) a leg up 

Real Estate Investment Trusts were able to act strategically during a recent bout of consumer price inflation in the US. The cost of labor, and building materials shot up, driving the price of property higher as well. Some funds took necessary actions based on data, while others did not have access, and were seriously damaged.  A good example of a data-driven informational advantage in the real estate industry is having access to timber data. 

Lumber massively affects the cost of building in the US so companies that had early access to data that indicated:

  • Sawmills closing down due to COVID
  • Incremental lumber futures increases
  • As well as lumber theft (an early indication of wood scarcity based on geography, quantities, and frequency) 

Were able to pivot in time. 

The bottom line here is companies that tracked web data, were able to easily time purchases at the optimal point. They used all of their available liquid cash to purchase many new assets, loaded up on timber, and/or focused on purchasing mills that closed down. They were able to ‘capitalize on this black swan’. Those that didn’t, could not build new projects, are now stuck with cash that can’t be correctly invested as properties are massively overpriced. And to top it all off, they lost core investor capital to competing REITs.

Market research carried out by Terakeet in the beauty industry showed that publishers and informational sites seriously outperformed big beauty brands and retailers:

  • 60% of informational and product-focused search queries are dominated by digital beauty magazines that do not sell any products
  • Publications like ‘Allure’, and ‘NY Magazine’ consistently appear higher in search results than retailers like ‘Sephora’, and ‘Amazon’
  • ‘Byrdie’, and ‘Allure’ (beauty publications with zero online sales) alone account for roughly 6 million organic monthly search-enabled visitors. 

Meaning most monetizable traffic in the beauty industry is lost to digital magazines. 

But according to the research cosmetics brands are not helpless – they can reroute that traffic to their online points of sale by scanning the web for:

  • Organic search trends
  • Keywords
  • And specific content that is currently driving traffic

Brands like L’oreal, Estee Lauder, and Clinique can build a Search Engine Optimization (SEO) strategy, enabling them to create content that will rank higher on Search Engine Results Pages (SERPs) than mere ‘interest pieces’. 

The bottom line here is that those beauty companies that optimize both AdWords, as well as content based on search engine data, can capitalize on non-branded queries, as well as new early-stage prospective customers who can be nurtured, and eased down marketing funnels. 

Takeaways, and insights 

Now is as good a time as ever to revisit Everett M. Rogers’ ‘Diffusion of Innovations Curve’:


This bell curve chart specifically addresses Investment management firms but its takeaways are relevant to business people from a wide range of industries. The companies on the left side of the curve are ‘innovators’, ‘early adopters’, and an ‘early majority’ –  they alone will reap the major rewards of utilizing web data (increased market share, and profit margins). 

They are the winners.

 Everyone to the right, the ‘large majority’, and ‘laggards’ are left with the crumbs, and leftovers. 

They are the losers

More from Bright Data

Datasets Icon
Get immediately structured data
Access reliable public web data for any use case. The datasets can be downloaded or delivered in a variety of formats. Subscribe to get fresh records of your preferred dataset based on a pre-defined schedule.
Web scraper IDE Icon
Build reliable web scrapers. Fast.
Build scrapers in a cloud environment with code templates and functions that speed up the development. This solution is based on Bright Data’s Web Unlocker and proxy infrastructure making it easy to scale and never get blocked.
Web Unlocker Icon
Implement an automated unlocking solution
Boost the unblocking process with fingerprint management, CAPTCHA-solving, and IP rotation. Any scraper, written in any language, can integrate it via a regular proxy interface.

Ready to get started?