Instagram to test a feature that automatically detects and blurs nude images in teen DMs and displays a warning screen for senders and recipients

Meta has announced that it will test a feature on Instagram that will detect and blur nude images sent in direct messages and also display a warning screen, in an effort to combat 'sextortion,' a practice that targets minors on social media.

New Tools to Help Protect Against Sextortion and Intimate Image Abuse | Meta

https://about.fb.com/news/2024/04/new-tools-to-help-protect-against-sextortion-and-intimate-image-abuse/

Sextortion means 'sexual blackmail' in which the perpetrator builds an intimate relationship with the victim through social media, demands that the victim send private images such as nude photos or videos, or forces sexual acts online, and then blackmails the victim using the images or videos obtained to demand money or further sexual acts. Sextortion is a serious problem, especially among young people, and has a significant impact on the mental health of victims, sometimes driving them to suicide.

The new feature announced by Meta uses machine learning algorithms to analyze whether images sent in Instagram DMs contain nudity, and because the machine learning algorithms run on the user's device, they work even in end-to-end encrypted chats.

And if the algorithm determines that the image you're sending contains nudity, it will automatically be blurred and you'll see a warning screen saying, 'Please be careful when sharing sensitive images.'

If you receive an image that contains nudity, it will also be automatically blurred and a warning screen will appear giving you the choice of whether to view the image or not.

If you try to forward a nude image you received to someone else, a warning screen will appear.

You can also block and report the other person from the warning screen.

Meta says it plans to turn the feature on by default for users under the age of 18. It also plans to develop a system to detect sextortion before it happens, based on a variety of signals that may indicate sextortion.

Meta is one of the founding members of a program called ' Lantern ,' which shares accounts and behaviors that violate child safety policies. Meta said that through Lantern, it will share information related to sextortion with other companies to help other social media sites identify accounts suspected of being sextortion perpetrators and take appropriate action.