Real life lessons on how public web data can improve HR

From diversity hiring over to retention, leveraging public web data alongside HR data analytics allows industry professionals to be more informed and inherently more strategic within their people decision making processes – providing them with the insights necessary to reach their organizational growth goals.
9 min read
Real life lessons on how public web data can improve HR

Breeding Organizational Success Using Public Web Data Strategies:

The talent supply is a “key concern” that has only been “exacerbated” by the current health crisis, according to Deloitte’s 2021 Human Capital Trends study, and today, many HR agencies and professionals are incorporating public web data strategies as the first inline resource to address the depletion of talent.

Many industry professionals have found that incorporating web data strategies greatly enhance processes tied to recruitment, development, performance and compensation – inevitably allowing them to make the highest quality people decisions, while doing it fairly and consistently.

To look deeper into the matter of how public web data insights can attribute to the growth and success of an organization, we spoke to two HR professionals: Co-founder and CEO of Mathison David Walsh, whose organization promotes diversity in the workforce; and CTO of HRog Vivek Amilkanthawar, whose company assists in recruiting strategies for HR managers. 

The Hiring Process:

Public web data helps HR professionals make thousands of people decisions a year: Who should we hire? Who should we promote? How much should we offer candidates? How much should we pay our best employees?

To elevate this process, HR professionals are looking towards public web data to allow them to peer into the looking glass of how workers seek employment and how organizations access, acquire and retain them. With that regard, you need to know who is hiring and how.

Companies such as Amilkanthawar’s Japan-based HRog, who use Bright Data’s residential proxy network, have developed their own software to help streamline recruiting processes by compiling web data points found openly on the internet.

The company’s solution focuses on recruiting processes that help identify organizations who are actively hiring or investing in marketing – meaning they are highly motivated to hire and to do so immediately.

“We check what jobs are being posted as advertisements, which ones are being promoted to be on the top of the search results, etc. and then feed that data back to our HR clients,” Amilkanthawar explained.

To perform its operations, HRog gathers relevant public web data – such as job title, location, salary range, company details, etc. – from company job sites and other job portals, which allow their clients to better gauge the market and adjust their strategies.

“Collecting web data from job listing sites gives us great insights,” said HRog CTO Vivek Amilkanthawar. “For example, how many engineering jobs are open in Tokyo, what the average salary is in a particular location compared to other locations, among many other examples.”

Amilkanthawar’s organization also uses public web data to identify the latest news and trends within the HR industry as well as the job market itself, and releases useful information daily for those linked to the industry – from how to choose a recruitment tool to best ways to conduct interviews and over to industry practices.

The information presented by HRog’s platform helps HR professionals identify: 

  • Who’s hiring? Their details.
  • Which sourcing channels are the most successful candidates being secured from? 
  • What types of jobs are currently on the market within a specific vertical? 
  • What are the typical compensation packages for like-positions? 
  • As well as the health of the talent supply.

More precisely, regarding recruitment, public web data can indicate:

  • The amount of applicants applying for a specific type of position
  • The skills of the candidates applying for like-positions
  • As well as the general job postings themselves:
    • How are they formatted?
    • How are they advertised and on what channels?
    • What experiences are listed as pertinent?

Diversifying the Workforce:

Diversity in the workforce is a core value in the success of any organization, creating both an ethical and competitive advantage for companies who invoke this approach. 

For many organizations, diverse workforces have notoriously outperformed their counterparts. This is due to the fact that within a diverse workforce there are a broader range of skills, experiences and perspectives – different cultures and backgrounds, etc. – providing the company the tools to deal with the nuances of the global marketplace. 

Mathison, who created a first-of-its-kind innovative all-in-one system managing diversity recruiting, strategy measurement and reporting, facilitates workplace diversity using public web data. 

“In 2020, social justice initiatives and the shift to managing culture and diversity in remote settings led to greater interest in our services,” said Mathison’s Walsh. “Our mission is to bridge the gap between underrepresented job seekers and employers committed to inclusion by helping employers cast a much wider net in their hiring.”

Mathison’s platform itself centralizes hundreds of inclusive talent networks – such as LinkedIn – and uses AI to help employers find diverse candidates for their most important roles. 

This includes sourcing diverse candidates, reducing bias in job descriptions, candidate selection and interviews and mobilizing team members in diversity hiring efforts with tools to build awareness and change behavior.

In order to collect publicly available online profile data, Mathison uses Bright Data’s Web Scraper IDE through its partnership with The Bright Initiative. This has enabled them to access and collect massive amounts of the specific public online profile data it needs, which is used as a critical building block to create its diverse platform and talent network.

To describe the specifics in which Mathison’s platform operates, the company combines public web data and proprietary machine learning to compile and analyze the resumes and profiles of candidates that may be typically overlooked by HR managers using traditional sourcing methods.

Its Equal Hiring Index additionally leverages internal metrics to audit an organization’s existing hiring and advancement processes to uncover opportunities that breed a more inclusive work environment.

To promote diversity, the platform uncovers hidden and unconscious bias during the hiring process by flagging exclusionary language in job descriptions and other written communication, such as terms that are gender-biased as well as other keywords that promote exclusion.

Additionally, to reduce bias, the platform anonymizes LinkedIn profiles gathered by Mathison – hiding the names and faces of candidates to help HR managers stay focused on skill set and experience when making their determining decisions, as opposed to being influenced by other stimulating factors.

As Mathison’s platform shows, the perfect HR analysis cocktail uses an even mix of internal as well as external web data to formulate the best insights towards creating a more inclusive workforce.

However, as stated, that only provides a minor understanding of how to promote diversity. To gain a full understanding companies also need public web data to learn from the best to have an idea of where to improve.

Public web data allows HR professionals to: 

  • Monitor the outreach activities of both themselves and their competitors
  • Determine what channels diverse companies are using to attract their candidates
  • Decipher how competitors advertise their offerings
  • Calculate the industry averages for its open positions
  • As well as find where they stand against their competitors in the quest to secure more diverse candidates

Basically benchmarking themselves against their competition, allowing them to tailor their acquisition strategy to identify and convert qualified diverse talent into employees.

Data Analytics for Candidate Sourcing:

With regard to moving the HR industry forward, many professionals within the vertical have invoked use of people analytics to secure talent for their organization – allowing them to find uncut gems lying in wait.

Today, it’s becoming all the more common for companies to look for candidates containing highly specialized skill sets in order to help them fill their ranks.

As discussed in previous blogs, using tools like Bright Data’s LinkedIn Datasets, HR professionals can incorporate public web data to shed light on the personality and make-up of certain candidates, allowing HR professionals to pinpoint qualities which they find to be indicative of a candidate’s success within the role they intend to fill. 

This could be the typical web data points found on a resume like:

  • Languages, skills or experience
  • To the more complex identifying factors found on social media such as verbal communication skills or certain hobbies, passions or activities a candidate takes part in.

Additionally, HR professionals can utilize insights from public web data to discover and map out which companies are directly competing for candidates by looking at businesses that are:

  • Active on job boards
  • Have a large employee base
  • Or large engagement rates with their brand

HR professionals could also determine what educational background and experience members of these organizations possess by searching for resume similarities in promising competing organizations.

Furthermore, public web data could allow employers to identify attractive employees that work for competing organizations that should be up for promotion, but have not yet, and offer them a more enticing position within their own organization.

Prepped for Success Using Public Web Data:

While the HR industry has been turned upside down following the onset of the coronavirus pandemic, leveraging the use of public web data could significantly help turn the tides and enhance operations that would attract and retain the best talent for an organization.

From the hiring process, to promoting diversity and back over to retention, HR managers have a lot to digest when assembling teams that could make or break a company’s growth or success, especially in today’s ever-changing landscape.

According to a Deloitte survey, only 3% of corporate respondents said that they have a “mature” understanding of people analytics and “have the [necessary] information needed to make sound people decisions.” That leaves around 97%, a well-determined majority, of HR professionals who are not incorporating data strategies into their talent acquisition processes. 

Naturally these professionals are making unstructured and uninformed decisions, meaning those who invoke these web data protocols are in effect miles ahead of the pack – giving them a competitive advantage to secure the best quality candidates and retain them for many years to come.

More from Bright Data

Datasets Icon

Get immediately structured data

Access reliable public web data for any use case. The datasets can be downloaded or delivered in a variety of formats. Subscribe to get fresh records of your preferred dataset based on a pre-defined schedule.

Web scraper IDE Icon

Build reliable web scrapers. Fast.

Build scrapers in a cloud environment with code templates and functions that speed up the development. This solution is based on Bright Data's Web Unlocker and proxy infrastructure making it easy to scale and never get blocked.

Web Unlocker Icon

Implement an automated unlocking solution

Boost the unblocking process with fingerprint management, CAPTCHA-solving, and IP rotation. Any scraper, written in any language, can integrate it via a regular proxy interface.

Ready to get started?