Checkers Little Shop AR app unlocks even more fun!

27 June 2017 ‐ 6 min read

Following the overwhelming success of Shoprite Checkers Little Shop campaign in 2016, South Africa’s most loved retailer approached us to create Little Shop 2 - the “MSQRD” of FMCG brands!

Checkers Little Shop mini products were such a hit with consumers that the retail giant wanted to expand on it with a second phase in the form of a specially developed augmented reality app. The app enables users to unlock funny face filters for each of the 30 participating brands. A fun platform designed for kids, (but anyone can use it), the campaign app is set to drive customers in store and encourage consumer spend.

How it works

When customers shop at Checkers or Checkers Hyper Stores, they receive a mini product for every R150 spent in-store. Users simply download the app for free via the App Store or Google Play, scan their minis to access fun filters, snap away and share! Users can post their snaps to the “Buzz Wall” via Instagram, Twitter and Facebook using the #CheckersLittleShop hashtag.

 

We built a face-tracking tool and optimised it for smooth and accurate tracking of facial contours

Creating the software

Licenses for existing face tracking technology are costly, so we built our own face-tracking tool which had to be optimised for smooth and accurate tracking of facial contours.

To show 3D and 2D models with animations, the user’s face is masked with the models while tracking their facial landmarks from the phone camera feed. We used openCV (open computer vision) with dLib facial landmark detection to help us achieve this, however the technology is a bit jumpy and the performance is not as good as we wanted it to be. We created a work-around by building in filtering logic with Kalman filters, and used optical flow with the Lucas-Kanade method to improve the smoothness of the detection. This was done on top of the Unity 3D platform.

We also created animations that can be triggered by smiling, pouting or opening your mouth, making the filters even more engaging.  

The models and animation work was done by Wolf & Halo, who also had to track each model to a unique rigging code.

 

 

Development insights

Battery usage for the users was a consideration, so we conducted performance testing and compared it to industry giant, Snapchat. We introduced clever methods and timing to reduce battery usage without compromising on quality, and in the end produced an even more battery-efficient app than major competitors.

Understanding the world of 3D was definitely a learning curve, particularly working with shaders and lighting and occlusion so that you don’t see the back of a person’s head through their face. We spent a fair amount of time doing competitor research and reaching out to the open source community to make sure we built a product of the highest quality.

We knew it was going to be a challenge with lots of unknowns, but there was nothing that we weren’t able to figure out.

Working with Checkers

Originally there was supposed to be an equal split between 3D and 2D models, but the client loved the 3D so much that they increased the budget midway through the project and asked us to build more 3D models and animations.

We enjoyed the process so much that we created additional filters that we really loved, and when the client saw the potential of what we were able to do, they got even more excited and we got to work closer together to create a really awesome product! We guided the brands in terms of what was possible, and came up with ideas for how the filters could work really well to promote their products in a fun way.

We guided the brands in terms of what was possible, and came up with ideas for how the filters could work

Launching Little Shop 2

The end product exceeded our client’s expectations, and we celebrated with champagne all around when the app launched on time to coincide with the nationwide Little Shop 2 campaign.

The campaign is currently live and runs for 8 weeks nationwide, or until stocks last. With the support of ATL advertising and social media campaign messaging, the app stores are already seeing staggering numbers of downloads.

Haven’t got it yet? Download the app today on via the App Store or Google Play.