A Peek Inside the Development of the Elevated Careers Matching Models

The Backstory:

eHarmony has a long history of conducting research on compatibility, and providing a product that helps people find the right partner for a happy, lasting relationship. Since launching in August of 2000, eHarmony has created matches that have resulted in millions of users getting married, and research has shown that these marriages are happier and less prone to divorce than couples that meet without eHarmony’s help. In 2013, after nearly 15 years of refining research and matching systems for romantic relationships, eHarmony decided to look at what other relationships might benefit from compatibility matching. This quickly led us to turn our attention to a second domain: A matching model for companies and employees to improve job satisfaction and engagement.

Tweet this: What other relationships can benefit from compatibility matching? Careers, of course!

The Premise, Decoded:

In developing this new focus, we quickly realized there are important differences between the data available to measure and create romantic compatibility, and the data relevant to workplace compatibility. There’s a big distinction, and in order to make this matching system work for….well, work, we had to identify those differences and the reasons behind them.
“There are important differences between the data available to measure and create romantic compatibility, and the data relevant to workplace compatibility.”
Arguably the biggest difference between romantic compatibility and work compatibility is that, in the romantic setting, you have two individuals who can self-report their information, wants, and ideals.
Companies are not people capable of self-reporting their own information. This information must be collected from people who comprise the company. Workforce and cultural data needs to come from the people who set the company’s strategies and execute on tactics. It needs to represent the views and experiences of the people working within the company. Not surprisingly, working with this kind of data creates a host of new statistical and methodological challenges that need to be tackled.

Tweet this: Workforce and cultural data needs to come from the people who set the company’s strategies and execute on tactics.

Solving for X:

Generally speaking, each person’s rating of their company is heavily influenced by their own opinions of that company (see, Kristof-Brown et al., 2005; Cable & Judge 1996). While their rating is a good indicator of how they feel about the company, it won’t generalize well to someone else being placed into that environment. Elevated Careers instead uses aggregate ratings from many employees within the same company. While company ratings from individuals all suffer from a degree of bias, aggregating those ratings over many people presumably averages that bias to zero, creating a usable company score that will generalize better.

Road Work Ahead:

The challenges don’t stop here, either. Aggregate ratings for companies have a set of less well-known psychometric properties that need to be tested in addition to the more common forms like Chronbach’s alpha or test-retest reliability (see, Biemann, Cole, & Voelpel, 2012). We need scales that are consistent and specific enough across companies to make aggregate ratings meaningful.
In other words, we need to be able to differentiate between companies on our scales, while maintaining adequate reliability. This is a challenge because high reliability can lead to poor differentiation under certain circumstances, and vice versa. The two must be balanced carefully. This requires testing and re-testing our measures until we narrow down appropriate survey items.
There are still further considerations with using aggregated scores. Well-formed scales can be meaningless without the appropriate level of aggregation. Let’s take Google, for example. Will a Google employee in Michigan rate Google similarly to a Google employee in California,  or New York City? To a certain degree, the answer is yes. But even a company like Google will have important regional and managerial differences that add error to the aggregated values. It depends entirely on the consistency in values across locations in the company.
“Even a company like Google will have important regional and managerial differences that add error to the aggregated values.”

Solutions Revealed:

The best aggregate scores tend to come at the most specific level: employees in the same building rating their specific location. Collecting this volume of data at each specific location can add up quickly if a company has multiple offices. This makes sense to those working in Human Resources or managerial work of any kind. It’s very common to see different cultures in different locations of the same company.
These methodological and statistical considerations have given us a lot to work on. Fortunately for us, there have been decades of theoretical (Ostroff & Judge, 2007; Chatman, 1989;  Kristof, 1996; Schneider, 1987), empirical (Kristof-Brown et al., 2005; Harter, Schmidt, & Hayes, 2002), and methodological research (Edwards & Parry, 1993; Edwards, 1994; Jansen & Kristof-Brown, 2005) on organizational fit.  This academic research has been absolutely vital to the construction of our own research, and the Elevated Careers product, and we hope to contribute in kind by both publishing our research, and offering companies and candidates the opportunity to benefit from the use of Elevated Careers.
Posted in Employers, HR TechTAGS: , ,