Predicting skin reactions with machine learning

Bayer Digital Campus Challenge · Germany · 2019

Bayer’s Digital Campus Challenge invites students from across Europe to work on real problems from the company. I chose the machine learning and deep learning track, where the question was simple to ask and hard to answer: will this skin product give this person a rash?

In numbers

• Time: 6 weeks from data to recommendation, 60% of it spent cleaning the data.
• Money: a volunteer skin test costs around €150. Screening out the riskiest 10% of combinations first would save about €75,000 per 5,000 tests.
• Errors: an area under the curve of 0.87 for the model we recommended and 0.89 for the trees, varying by less than 0.02 between skin types.
• Adoption: the explainable model was the one we recommended to a jury of 6 Bayer experts.
• Scale: 5,000 product tests, 1,400 ingredient names in 3 languages matched to 380 ingredients, 4 skin types and 5-fold cross-validation.

The approach

• The data: about 5,000 anonymised product tests, each with the product’s ingredients, the tester’s skin type and whether a reaction followed.
• Cleaning: the same ingredient appeared under a dozen names in 3 languages. Matching them took longer than any model.
• Models: logistic regression first, because every prediction can be explained, then gradient-boosted trees to see how much accuracy that explanation cost.
• Testing: 5-fold cross-validation, and a check that the model worked as well for every skin type, not just the most common one.

What I learned

The trees were a little more accurate, with an area under the curve of 0.89 against 0.87. We still recommended logistic regression. A dermatologist who hears “62% chance of a rash” needs to know why, and the simpler model could say: the fragrance, the alcohol and a history of reactions.

It is the same choice I make with AI today: a model people can understand beats a slightly better one they can’t.