Many companies tout machine learning methods as eliminating bias, as many studies have shown humans’ implicit bias in race and gender. However, if machine learning models are trained on skewed data, they will also be biased. These companies are essentially assigning value to humans via machine learning algorithms that have a high probability of being designed with biased algorithms.

Companies such as stella.ai​, HireVue​, Ideal, and Entelo all boast machine learning and big data in the hiring platforms that they have created for companies such as HBO, Unilever, Jetblue, Mount Sinai, Paypal, Target, and Oracle. Google’s own CloudJobs is used by big players such as Johnson and Johnson and Fedex.

It is no secret that the recruitment process in tech is riddled with biases at just about every stage and throughout the industry. Women and people of color are less likely to have their resumes approved for interviews than their similarly qualified white male counterparts. The algorithms that companies tout to eliminate this bias are trained on past data. However, that data are the past decisions which were themselves biased. These algorithms are therefore likely to be trained on biased datasets, but we can’t know for sure because these companies refuse to publish information about their datasets. Tech companies have a longstanding track record of using skewed datasets in AI (see HP’s facial movement tracking software which was unable to do so for black faces or 23AndMe’s performance with Korean Americans), so we shouldn’t just trust these companies to be responsible and do the right thing. In areas that would directly affect job searching and candidate vetting, bias has also been proven to exist. Google search is 5 times more likely to show ads for high paying jobs to men than to women, for example. Word2Vec, a cornerstone of natural language processing which may be used in anything from resume screening to job search results, is biased in gender roles.

For many products, there is a lack of transparency showing exactly where machine learning is used, and we have no way of knowing if these algorithms are biased or not.

However, solutions do exist. It is possible to analyze the training and test sets for skew towards a particular race or gender. It is possible to incorporate basic techniques to combat bias that don’t use machine learning. Lastly, it is possible to de-bias algorithms with better algorithms, but more research is needed in this area.

Because of this, we demand companies do the due-diligence needed. We demand increased transparency in the form of published statistics on training and test set data in the cases of machine learning models that are used to evaluate humans.