Facebook automatically scans posts and chat logs for criminal activity, using big data processing techniques similar to those used in targeting advertising to determine the most vulnerable users, according to a new Reuters report that explores combating pedophilia in social media. The social network's scanning tools use factors such as mutual friends, past interaction, distance and age difference — alongside simple phrase searches — to flag potentially nefarious conversations for human moderators. They also rely on archives of previous conversations that are known to have led to sexual assaults, identifying patterns and searching for similar ones.
Apparently keen to pre-empt the sorts of privacy concerns that have dogged Facebook in recent months, Chief Security Officer Joe Sullivan tells Reuters that "it's really important that we use technology that has a very low false-positive rate." He explains that "[w]e've never wanted to set up an environment where we have employees looking at private communications," stressing that the company's systems attempt to avoid flagging up long-standing personal relationships. A lot of the activity that Facebook refers to law enforcement is identified through its user reporting system, detailed in a recent infographic.