Recommender systems are like tech helpers on websites. They do jobs people used to do. In news, these systems pick what stories you see online, aiming to get more clicks. But not many studies have checked how well they do compared to humans.
Carnegie Mellon did a study on an online news site in Germany. They looked at how people reacted to computer-generated suggestions versus picks by human editors. The computer usually did better, but humans did better in certain situations. The study suggests that the best way might be a mix of human choices and computer help.
The right mix
“Our work highlights a critical tension between detailed yet potentially narrow information available to algorithms and broad but often unscalable information available to humans,” the researchers say. “Algorithmic recommendations personalize at scale using information that tends to be detailed but is often temporally narrow and context-specific, while human experts base recommendations on broad knowledge accumulated over a professional career but cannot make individual recommendations at scale.”
To figure out the best way companies should use algorithmic recommendations versus human choices, researchers checked how users reacted to both at a big online news site in Germany. This happened from December 2017 to May 2018. The site has lots of visitors and gets a ton of page views every month because of ads.
On average, the computer suggestions got more clicks from users compared to choices made by human editors. But there were conditions: more experienced human editors did better, the computer needed a lot of data to do well, and on days with more interesting news, humans did better.
The findings say that going back to human choices can help fix the problems of personalized computer suggestions. They also suggest that when there’s not enough personal data, platforms should rely on human expertise. The best mix of human choices and computer help can boost clicks by up to 13%.
“Based on our experiment, we suggest that managers leverage humans and automatic recommendations together rather than looking at curation as an issue that pits human experts against algorithms,” the authors conclude.
