Given that many retail operations require visual feedback and generate large amounts of data, the growing interest in computer vision among retail companies shouldn’t be surprising. According to the 29th Annual Retail Technology Study by RIS, only 3% of retailers have already implemented computer vision technology, with 40% planning to implement it within the next two years.
Turning to computer vision software development, companies can tackle many retail store pain points and can potentially transform both customer and employee experiences. For example, customer journeys can be redefined by making store layout improvement decisions based on real data rather than intuition.
Today’s consumers expect as much personalization and convenience in retail stores as they experience online. This is also the reason why applications for virtual reality retail are becoming more popular. With computer vision cameras, self-checkout experience becomes frictionless, while virtual mirrors enable the previously unseen level of personalization. Routine tasks can be automated with autonomous robots, leaving employees more time for more customer-oriented tasks.
A holistic approach to implementing machine learning and computer vision technologies helps make digital transformation in retail much more attainable. Let’s look at the top five applications of computer vision in retail.Â
The most popular applications of computer vision in retail
Self-checkout
Self-checkout has already solidified its importance for brick-and-mortar stores. Given that customer service automation is becoming a priority, companies need to update their processes to make them more efficient.
Currently, in the majority of stores, self-checkout implies that customers manually scan barcodes of individual items, which is a notably daunting experience. Computer vision-powered cameras can recognize products without relying on barcode scanning, significantly improving customer experience and security while also speeding up the checkout process overall.Â
In the last few years, with the proliferation of machine learning in retail and automated visual inspection, software companies have recognized the demand for computer vision-enabled self-checkout systems and now offer a range of variations of this concept. For example, Australian startup Tiliter developed an automated product recognition system poised to free both cashiers and customers of manually entering price look-up numbers of fresh produce. The system has proven to be especially effective for stores that deal with high theft rates. Besides increasing security, Tiliter’s Vision System also eliminates the need for wrapping fruits and vegetables.
Abto Software, in another example, has developed a full-fledged cashier-less technology, enabling each computer-vision-powered unit to process more than 100 items of any category per minute. One of the limitations of Abto’s solution is that it can ensure a 100% accuracy only if the products are not overlapping while transported over the checkout conveyor.
Lastly, a review of cashier-less systems can rarely go without mentioning Amazon’s Just Walk Out system. By utilizing a combination of cameras, sensors, and deep learning, customers can simply pick up the items they need and walk out of the store without waiting in line to pay. Notably, customers don’t need an Amazon account or an app. Computer vision camera track a person’s movements, and shelf sensors detect item removal or return. As the customer leaves the store, their card is automatically charged for items they picked up.
For this retail automation system enabling retail automation to work as intended, sensitive information including credit card data is required for collection. However, in today’s privacy context, Just Walk Out is yet another trade-off between privacy and convenience. Nevertheless, Philadelphia and San Francisco, potentially huge markets for Amazon’s technology, both banned cashier-less stores last year. Interestingly enough, in March 2020 Amazon started selling its Just Walk Out technology to other retailers.
Inventory management
Computer vision has also found its way into retail inventory management. According to the 30th Annual Retail Technology Study conducted by RIS, 64% of retailers are looking to deploy various data-driven solutions, computer vision included, to optimize inventory in the next two years.
For example, Shelfie uses computer vision cameras mounted on top of standard retail shelves to alert staff about incorrectly placed products and gaps on shelves. This frees up shopping floor staff to shift their focus to customer service more. At the same time, fully-stocked shelves dramatically increase consumers’ propensity to purchase. Moreover, with real-time data analytics, retail stores can also use Shelfie to dynamically reposition products and foster data-driven decision-making on the spot.
Other companies take a somewhat different approach and build all-in-one inspection systems. For example, Tally, a mobile robot designed by Simbe Robotics, captures visual data from more than 12 high-resolution cameras. Besides notifying staff about out-of-stock products, Tally can also detect damaged packaging and inaccurate pricing.
Unlike other similar solutions, Tally doesn’t require any infrastructure changes. At the end of 2020, Simbe updated Tally with a new optical system, claiming that now the robot’s recognition accuracy is nearly 99%. Simbe claims that by reducing out-of-stock items by 20% and ensuring perfect product layout, retailers can return their investments in one month.
In 2022, the company announced its partnership with Wakefern, the largest retailer-owned cooperative in the U.S. Currently, Tally has been adopted by many top retailers in the US, Europe, and the Middle East and has surpassed cumulative 200,000 miles of autonomous in-store scanning.
Leverage the power of computer vision in retail
Store layout improvement
By installing computer vision cameras, retailers can track customers’ movements across the store to identify their purchase patterns, ‘hot areas’ and behavior in relation to certain products. With this information at hand, retailers can take informed decisions about product placement, store layout, and staffing.
For example, Samsung used computer vision and retail predictive analytics to optimize its pop-up store dedicated to the pre-launch of its flagship mobile phone Galaxy S9. When it comes to such short-term marketing campaigns, being able to optimize performance on the fly is critical. With a number of in-store computer vision cameras installed across the store, Samsung gathered footprint, dwell time, product interaction and demographic data in different store zones.
The end-goal of this marketing campaign was to convert each visitor to be a potential buyer and social advocate of the product through engaging them in different zones within the store. By gathering vast amounts of data, Samsung managed to identify what zones had the most impact on the conversion, when the store was under-staffed, and what messaging was the most effective for attracting passers-by. Such insights enabled Samsung to adjust store layout in real time, deriving maximum value from its marketing campaign.
In a similar example, a high-end Serbian fashion retailer, Legend World Wide, collaborated with Deloitte to build a ‘connected store’. The company installed computer vision-equipped cameras and sensors across its physical store to track customer journeys, better understand how products move around the store, and gain better product-related insights overall.
By analyzing customer movement heatmaps, Legend discovered that store layout was its biggest bottleneck. When entering the store, most men quickly skimmed through the clothes on the left, immediately realizing that they are browsing women’s clothes. This made most male customers to turn around and leave the store. Legend solved this problem by simply installing clear navigation signs indicating that the men’s section is upstairs. While an experienced retail analyst would most likely manually detect this bottleneck, computer vision software allows to make such conclusions with greater certainty and much faster.
Virtual mirrors and recommendation engines
In retail, virtual mirrors may become the next frontier of personalization and customer experience enhancement. A virtual mirror is a conventional mirror with a display behind the glass. Equipped with computer vision cameras and AR, virtual mirrors can show a range of contextual information, which helps customers connect with the brand better.
For instance, as part of its array of retail marketing services, FindMine offers an in-store virtual fitting room solution ‘Complete the Look’. The ML-powered engine behind FindMine’s flagship product provides its users with real-time fashion recommendations based on their current outfit. For example, if a person tries on a hat in the fitting room, the virtual mirror will recognize the hat and recommend a bag, accessories, jewelry, and other items that will complement the look.
To make this happen, a retailer feeds FindMine’s algorithm with the data on each product from their catalog. The system then considers a multitude of items’ characteristics including color, description, price, and gender targets to recommend other relevant products at the store.
Besides its in-store use, FindMine’s algorithm can also significantly impact the ecommerce sector. For example, John Varvatos, a US-based luxury menswear brand, turned to FindMine to elevate their brand experience and create the experience of a virtual store. By utilizing ‘Complete the Look’ on their product description pages, John Varvatos saw a 74% increase in the average order value, an 83% increase in conversions, and a 107% increase in time spent on the website.
In a similar fashion (no pun intended), eyeglasses manufacturer, Warby Parker, uses a virtual try-on feature available exclusively on its iOS app. With the help of an AR-based app, customers can now see how exactly any Warby Parker frames will look on their faces. Importantly, the company has managed to achieve impeccable accuracy in terms of matching the real-life position of glasses, making customers much more confident in their purchases.
Barcode scanning smartphone apps
One of the most frequently cited appeals of online shopping is the ability to research product reviews in detail. According to TrustPilot’s 2020 study on the role of reviews in online trust, 89% of consumers read reviews before making a purchase decision.
With computer vision, online reviews and ratings can finally be brought to the physical store. For example, Scandit’s barcode scanning application allows customers to access product information and reviews by scanning this product’s barcode with the smartphone’s camera. The Scandit’s Product Information Lookup application overlays information on the device’s screen with the help of AR.
Guitar Center, one of the largest musical equipment stores in the world, utilizes Scandit’s barcode scanning application to leverage the well-established features of its online store in physical outlets, too. Customers can now use the mobile app to look up videos, reviews, ratings, similar products, and alternative colors of scanned products, which helps Guitar Center to provide a much-demanded omnichannel experience.
Currently, Scandit has over 1700 customers worldwide including industry giants like Levi’s Straus and Sephora. At the beginning of 2022, the company raised $150 million, which indicates the immense potential of computer vision-enabled barcode scanning apps in retail.
Similarly, the largest grocery retailer in Denmark, Coop, has turned to Scandit to boost efficiency of its store operations. With a dedicated smartphone app, Coop’s customers can now scan products, add items to the shopping cart, and pay in the app by card, avoiding queues. Coop reports 250,000 customer using the app daily. Moreover, Coop also deployed Scandit to help its 2,000 employees to complete in-store tasks, such as markdown. This provides more in-store efficiency and long-term economic benefits, as now employees don’t need to share scanners between them.
We build robust AI-based tools for retail professionals
Taking a holistic approach
Far too often, retailers adopt emerging technologies to optimize just a certain part of store operations. Such a vision is usually driven by niche software vendors, who are often focused on solving a particular business problem. At the same time, the success of Amazon Go can be largely attributed to an integral vision of transforming the retail experience in its entirety.
While this scenario might be too extreme or economically unfeasible for many companies, considering long-term transformational potential is critically important. Take small steps and ensure that your organization explores new tools and takes a data-driven approach to business. This should foster experimentation and discovery of new use cases for the technology.
In the times of rapidly digitized and continuously changing consumer lifestyles, adapting to consumer needs and behaviors on the fly becomes crucial. With computer vision and AI, companies can measure customers’ experience and engagement while making informed decisions about improving store operations holistically.
