How Project PI, an AI that automatically checks whether products are damaged or missing before being shipped from Amazon warehouses, works

Amazon has announced a system called ' Project PI ' that uses generative AI and image recognition to check whether products are damaged or the wrong color or size before being shipped. Project PI has already been introduced in some of its fulfillment centers in the United States.

Amazon: AI spots product defects, reduces waste
https://www.aboutamazon.com/news/innovation-at-amazon/amazon-ai-sustainability-carbon-footprint-product-defects

How Project PI helps Amazon remove imperfect products - Amazon Science
https://www.amazon.science/news-and-features/how-project-pi-helps-amazon-remove-imperfect-products

Amazon's fulfillment centers used an OCR model to check the label information when inventory arrived and compare it to information in Amazon's database. This system allows them to quarantine expired products that don't match the dates in the database. Projet PI incorporates image recognition into the traditional system to allow for product status management.

The 'PI' in Project PI stands for Private Investigator. Project PI uses an image recognition system to scan products and check whether there are any problems with the products. The CV (computer vision) model that performs this image recognition scrutinizes color and monochrome images of reference images from product catalogs and actual product images to detect damage such as bent book covers.

In addition, the system also uses generative AI capabilities to process information multimodally by synthesizing product images from images taken during the fulfillment process and combining them with customer feedback. For example, if a customer makes a complaint, the system analyzes the image data of the product and learns from the problem.

With conventional CV models, it is necessary to train a model for each tear in the box or break in the seal, resulting in dozens to hundreds of models, but with a multimodal model, one model can deal with multiple problems.

If a problem is found with a product, it is automatically quarantined from being shipped to a customer and other products are checked for similar issues. Products quarantined by Project PI are reviewed by a human staff to determine whether they can be resold at a discount, donated, or put to another use.

According to Amazon, a proof of concept has been conducted in some fulfillment centers in the United States since May 2022, and it has been possible to accurately isolate expired items and items of the wrong color or size. In the future, the company aims to implement near real-time defect detection using local image processing, so that a robot can remove problematic items from the conveyor and automatically order a replacement, preventing the fulfillment process from stalling.

'Ultimately, we want to be behind the scenes, and customers don't need to know that this is happening,' said Keiko Akashi, senior manager of product management at Amazon. 'They should receive perfect orders, and shouldn't even know that expired or damaged items exist.'