Some 93% of businesses believe that web data is either “very important” or “crucial” to supporting operations and decision-making across their organizations, according to a recent study conducted by Bright Data and the independent research firm Vanson Bourne.
The survey results of 500 IT, technology and data analytics professionals across the US, UK and France, revealed the sheer expansion of web data usage within the retail, travel and banking sectors — indicating that the market reliance on web data across these varying business sectors has significantly increased over the past year.
According to the survey, close to nine-out-of-ten (87%) of respondents stated that the need for web data within their organizations has grown over the last year, and 90% reported that their web data budgets increased within the same time period.
This is despite the fact that companies worldwide are cutting expenses across the board due to the macro global economic situation. Where many other departmental budgets have been sized down, this is not being reflected in the data acquisition market, which businesses are seeing as crucial to the success of their organizations.
Web Data Usage and Sourcing
To backtrack for a second, in general, businesses collect web data from various online sources to make better informed decisions for their organizations.
The primary uses of this information go towards operations such as product research and development (47%), competitor analysis (45%) and testing and/or training operational systems (43%) — pulling web data from online sources such as social media (58%), search engines (56%) and competitor websites (47%) to perform such actions.
“It’s not surprising to see that web data has become the most used source of data in businesses today across the board,” said Amir Yazdi, Marketing Analyst at McDonald’s Canada. “As the survey indicates, the uses of web data range from competitive strategy building to pricing strategy, from measuring public sentiments to training operational systems.
“All this is done to ensure that customers or consumers get the most relevant services, offers or products. After all, that is the goal of every business and organization.”
Web Data Collection
Focusing on sourcing, almost all (97%) of those surveyed stated it important to gain access to multiple web data sources or datasets, which aside from the use cases stated above, allow them to better gauge consumer sentiment, set dynamic pricing strategies, gain increased visibility over the market landscape and so on.
Therefore, to support their web data consumption, some 90% of organizations are currently using web scraping applications or technologies to automatically and systematically collect web data at scale from varying online sources of information.
Accordingly, nearly half (48%) of respondents stated they are collecting this data themselves, while a third (32%) are focused on tailored pre-made data sets — large sets of data or information, that focus on a single subject, collected from either single or various sources.
These datasets are purchased “off-the-shelf” from companies that specialize in collecting and structuring ready-made public data sets for immediate use and purchase.
With an increased interest in pre-made datasets, the survey now indicates a diversified approach to collecting web data, with over half (55%) of respondents stating that they are now searching to make acquisitions or find external partners to support their web data efforts — more than a 25% increase compared to two years prior — as 56% of respondents claim they currently can’t cope with their internal data demands.
Conclusion
In conclusion, the findings of the survey point to the emerging trend of organizations placing more resources into their web data efforts, as well as future plans to add to their budgets to consume additional forms of data in the years to come.
Considering these efforts require significant investment in teams to collect, clean, structure and analyze data, many organizations are involving external partners, who provide the full service and deliver data on demand — allowing them to still be able to use data without investing the time and resources it takes to collect it.
“It is clear that in the past year, the market reliance on web data has significantly increased,” said Or Lenchner, Bright Data CEO. “More and more market leaders across every sector are looking at on-demand data solutions, such as ready-made data sets to make their data operation and data management that much more efficient and cost-effective.”
“Both tailored and pre-made data sets are becoming significantly more attractive for organizations of all sizes,” Lenchner continued. “The smartest players are using a combination of both offerings, whether it is specifically tailored data sets or ready-made to address their most essential questions and provide the immediate answers that drive their organizations further.”
