The survey, which included 150 IT, technology, and data analytics experts from various retail, technology, and non-profit organizations across the US, found companies and non-profits alike all rely on publicly available web data to efficiently and effectively carry out their missions, with most (94%) doing so on a daily basis. Evidently, 79% of respondents stated they would be unable to operate effectively without access to public web data, with 73% claiming they would struggle to achieve their most important goals without it.
“Whether used for business purposes or to save lives, results of this survey show that publicly available web data is critical to many organizations’ missions,” said Or Lenchner, CEO of Bright Data. “Denying access to public web data undermines people’s right to freedom of information and severely cripples projects working for the benefit of humanity.”
Identifying hate speech and early signs of violence
Survey results found that two in five (40%) organizations use public web data to identify harmful content online, showing that while some companies collect public web data for business use, some use it to literally save lives. Non-profit organizations rely on public web data to conduct research, flag sensitive content, or identify harmful trends.
For example, The Network Contagion Research Institute (NCRI) partnered with The Bright Initiative to search public social media channels, to identify cyber-social threats and the spread of hostile ideologies, political deception, and hate and manipulation on the internet that can develop into actual violence.
In addition, Human Trafficking Initiative (HTI Labs), another Bright Initiative partner, works to fight human trafficking in the commercial sex industry through data-driven research. It uses web-based commercial sex advertisements, data science, and network analysis to identify potential trafficking networks within the commercial sex industry and then passes on these potential threats to law enforcement agencies. As an example, in its early days, the organization successfully brought down a large sex trafficking network involving nearly 70 women from across the U.S., which then helped bring a dozen human traffickers to justice and face their crimes.
Infinite use cases
From using up-to-date public web data to implement dynamic pricing strategies to setting better travel accommodation offers to display better booking packages to keeping brands accountable, public web data has many uses in the commercial sector, and many companies have long-embedded data strategies as a vital part of their operations.
According to the survey, 61% collect publicly available web data to compare pricing and monitor the inventories of competitors, and 17% use it for market research. For example, a local department store can gather public data from social media channels to better identify the target audience, determine their needs based on online reviews, offer more specific merchandise, and better control their inventory. Or rather, a travel agency analyzing public web data can provide better packages, highlight favorite destinations and monitor competitor prices.
Public data extracted from the internet helps enterprises function as much as it powers critical innovations for the benefit of the public. Denying access to publicly available web data would violate people’s right to information and deal a devastating blow to projects working on improving and sometimes saving people’s lives on a global scale. As reflected in the survey results, industry leaders had their say – public data needs to remain publicly accessible to all.
