Amazon Harnesses Generative AI For Seamless Palm Recognition Shopping Experience

Generative AI may have made headlines for its ability to write text and create art, but its uses extend beyond simply generating content. Amazon has been using the technology to recognize customer palm scans for its Amazon One payment system. Let’s explore how the technology behind ChatGPT is helping Amazon to reimagine the shopping and payment experience.

What Is Amazon One?

Forgot your wallet? No problem. That’s the idea behind Amazon One, a contactless payment system that lets customers pay, not with their wallet or phone, but with the palm of their hand. Customers show their palm to an Amazon One scanner, which then uses infrared light to recognize the palm’s unique features like lines, grooves, ridges, and veins. It’s a palm signature that is effectively connected to the customer’s credit card or Amazon account.

Sounds great, right? However, the problem with an innovative system like this is that it needs to be super-accurate for it to work. This is where generative AI comes into play.

The Problem

The scheme is being rolled out at retailers, grocery stores (including Whole Foods), and entertainment venues. But Amazon One isn’t just a payment method – it can also be used to verify a customer’s age, as an entry system, or in place of a loyalty card.

When you’re training any deep learning model, if you want it to be highly accurate (which of course, Amazon does), you need quality data. And lots of it. In this case, lots and lots of images of palms. Which was a problem for Amazon, since the company only had a small amount of palm data to train their system.

So the company turned to generative AI to generate millions of synthetic images of palms to train their model. As Amazon describes it, they used generative AI to create a “palm factory.”

Known as “synthetic data,” AI-generated data like this can prove useful in several ways. For one thing, it can be used to create additional data when available real-world data is limited. It can be used to create a more balanced, diverse dataset – thereby helping to reduce bias in AI models. It can also help preserve privacy by using synthetic data instead of the real, personal data of millions of people. In other words, synthetic data can replicate the breadth and variety of real data without the complications that often come with real-world data.

So, How Did This Work In Practice?

One of the things Amazon did was use generative AI to quickly generate hands reflecting all sorts of subtle changes, such as different hand poses, different lighting conditions, and even the presence of rings, scars, and Band-Aids. Generative AI was also used to create images not just of the palm itself but of the veins underneath the skin. And this played a major role in making the system more accurate. The system was also trained to recognize the difference between a live palm and, say, someone holding a fake silicon hand over the scanner. Amazon even tested the system with 3D-printed palms, and it was able to tell the difference.

Thanks to AI, palm images could also be automatically annotated, with labels telling the system that it was looking at, for example, a hand wearing a wedding band. This saved time and allowed Amazon to accelerate the system’s training.

As Amazon explains, the problem of identifying palm scans is more complex than, say, your smartphone recognizing your face and automatically unlocking it. Your smartphone already knows who you are, so all it has to do is verify that the face in front of the screen is indeed yours. It’s a one-to-one scenario; in other words – your phone is basically working out, “Is this you or not.” Whereas an Amazon One scanner doesn’t know who is putting their hand over the scanner. It has to work out the palm signature, differentiating it from all the other Amazon One palm signatures and quickly. It must also be able to identify palms that aren’t enrolled in the Amazon One program – when a person isn’t an Amazon One customer but is trying to use the scanner anyway.

The Results Are Impressive

Amazon set out to create a highly accurate system that could recognize palm scans, and that’s exactly what it achieved. According to Amazon data, in the more than 3 million times the system has already been used, it has proven to be 99.9999% accurate. In fact, the Amazon One system is 100 times more accurate than scanning two irises, marking a significant breakthrough in biometric identification.

And as a bonus, Amazon says its system is designed to protect customer privacy. Amazon One doesn’t use palm information to identify a person – it is purely matching the palm signature to the payment associated with it. Amazon also says its system is race and gender agnostic, meaning it cannot accurately perceive sex or skin tone.