A former Meta employee talks about the actual situation of ``XCheck'' that excludes specific accounts on Facebook and Instagram from moderation targets
Facebook and Instagram have automated moderation systems that check content and remove it if there are issues. Its automatic moderation system is powerful, and even legitimate users may have their posts deleted if malicious reports are repeated. There is a program called Mr. Krishna, a system engineer who worked at Meta's
XCheck at Meta: Why it exists and how it works
https://blog.nindalf.com/posts/xcheck/
Facebook and Instagram have a feature that allows users to report content, which allows them to report offensive posts, posts that violate individual rights, etc. as 'bad content'. The number of reports can reach millions per day, and it is difficult for human moderators to check everything and determine 'whether or not a post should be deleted according to the report', so the reported content And a system that automatically checks the contents of posts has been introduced. Our automated moderation system automatically hides posts after a certain number of repeated 'bad content' reports.
According to Krishna, there are two main problems with the automatic check system. First of all, the system judges behaviors such as 'sending messages to people who are not friends, but the response rate is extremely low' as 'fraudulent account behavior', but a small number of legitimate users have similar behaviors. I sometimes move. Due to such 'outlier-looking behavior', even legitimate users may be subject to penalties by the automatic moderation system. Secondly, by using multiple accounts to make malicious reports, it is possible to abuse the system that ``automatically hides posts when there are many reports''. In particular, accounts of celebrities, such as politicians and athletes, are increasingly being attacked.
In addition, the integrity team, which manages the moderation system of posts on Meta, has hundreds of teams spanning multiple time zones, and it seems that there are thousands of people in charge overall. As a result, even if each team made good changes to the automatic moderation system, each team did not cooperate well, and some 'loopholes' were always created, Krishna said.
Build a ``fail-safe'' that says ``This user is a verified good user. makes sense, Krishna explains. This failsafe is 'XCheck', a system whose purpose is to 'prevent moderation systems from impacting accounts that are known to be good'.
XCheck-protected accounts will not have their content removed, even if there are many users who dislike the account and file many reports. Also, when soccer player Neymar posted a nude photo on Instagram in 2019, it is usually automatically deleted after a certain number of reports, but Neymar's account was protected by XCheck. , there have been instances where posts remained until XCheck staff manually deleted them.
Facebook has 5.8 million VIP users, applying its own 'XCheck' program outside of moderation monitoring - GIGAZINE
XCheck was sometimes questioned by users as a system that treated some users as VIPs, but according to Mr. Krishna, it had a bad reputation internally. XCheck contains millions of programmatic instances , but I've never heard of them working so well that they 'prevented erroneous penalties on the right accounts,' Krishna said. I'm talking
Platforms can be abused by malicious users, and integrity systems that try to fix them can also be abused by malicious users. XCheck exists to fix that, Krishna concludes, but that XCheck can also be abused. Krishna's explanation of XCheck is accurate as of early 2021 when Mr. Krishna worked for Meta (then Facebook).
Related Posts:
in Web Service, Posted by log1e_dh