PepsiCo

Touchless screens are competent at recognizing that there’s a human hand in front of them. However, our particular use case required more nuanced gesture recognition, turning them into reliable, readable intentions. 

 

Beyond selecting items, we needed to empower customers to explore menus, select and customize options, and finalize their orders, all without touching the screen. Customers also needed to intuitively understand what to do with their hands to place their orders effortlessly – and have a delightful experience along the way.

To address the first challenge of gesture recognition, we deepened our technological understanding of the opportunities that touchless screens offer. We experimented and researched a variety of interaction scenarios to develop an advanced model.

 

We then took an existing kiosk form factor and applied this to our model, building a prototype in Unity, the world’s leading platform for creating and operating interactive, real-time 3D (RT3D) content.

 

Before field-testing the prototype at a KFC in Europe, we also needed to understand the typical customer journey. Combining camera and LiDAR data, we created a 1:1 scale CAD model of the restaurant to identify high-priority touchpoints and the optimum location for the kiosk.

 

In parallel, we integrated the prototype with the restaurant’s content management system (CMS) and point-of-sale (POS) systems, built the hardware kiosk, and developed the user interface (UI) flow to support ordering from the full menu.

Our team first tested the experience with KFC customers in a controlled lab environment to identify any issues with the interaction model and interface, iteratively refining the prototype as we went.

 

In our first iteration, we implemented body tracking to increase the stability and robustness of the kiosk interaction. We learned that customers wanted to interact with the kiosk the same way they might use a touchscreen due to their previous experiences.

 

Next, we updated the model to reflect this insight and lean into customers’ expectations. Re-testing resulted in significant improvement in customer satisfaction.

 

During our final lab session, we tested different onboarding strategies to expedite customers’ familiarity and comfort with the experience. We then ran a nine-day field test at a KFC in Poland, allowing real customers to interact with the experience.

To understand the results of this research phase, we combined qualitative observations and quantitative surveys, leading to a deeper understanding of how users respond to the kiosk in context and greater insights into their experience.

 

We also incorporated interaction and transactional data analysis. This allowed us to understand how customers physically interacted with the kiosk, helped compare basket size, and understand the breakdown of transactions by food and beverage across all the location’s kiosks (touchscreen and touchless) and the counter.