Checkout Visual Compliance Detection       


Jonathan H. Connell photo Quanfu Fan photoSharathchandra U. (Sharath) Pankanti photo

Checkout Visual Compliance Detection - overview

In many business sectors, there is a great need to ensure that the activities of employees and customers comply with established procedures. For instance, in a construction site, it may be required that building supplies be placed in designated areas to ensure safety. There are many examples of this need when it comes to casinos, which have an obvious need to verify that its customers are not cheating. This verification process is also very applicable to retailers in various contexts.

Human surveillance has long been used to perform this verification. However, this approach is not generally very effective, due to scalability issues and the ability of a human to maintain sustained attention. In some scenarios, such as a retail checkout station, a data stream (in retail, a TLOG) might be available that can be mined to infer non-compliance based on statistical analysis techniques. However, these statistical anomalies may not be very strongly correlated with fraudulent activity.

With advances in video analytics, it is possible to visually monitor activities for compliance. This technique is referred to as Visual Compliance. As video is captured by cameras installed at a site of interest, video analytics algorithms automatically determine whether or not compliance is satisfied.

The Exploratory Computer Vision Group at IBM Research has been focusing specifically on developing solutions that implement Visual Compliance in retail contexts. We apply intelligent video analytic technology to understand normal and abnormal casher behavior during transactions. Our solution provides unique and distinct capabilities for retailers to resolve the shrink problem effectively and efficiently.

alternate textA breakdown of retail shrink in 2005 (Source: National Supermarket Research Group)

Retail Shrink is one of the topmost concerns on the minds of retailers. According to recent studies, the shrink in stores is on the order of 90B USD in the US and Europe alone. A significant portion of this shrink is mediated by employees (50%) and occurs around the point of sale (33%). It makes a lot of sense to reduce this shrink because it significantly improves profitability for retailers. According to one estimate (AMR 2004), decreasing shrink from 2% to 1% will improve profitability by as much as 40%!

A lot of checkout shrink is related to how well cashiers are complying with the checkout process. It is widely understood that a number of incidents related to employee theft are fraudulent exceptions executed by cashiers, supervisors or both. Generally speaking, retail shrink is largely attributed to the following types of checkout fraud:

  • Non-Empty Cart: the cashier or shopper may not completely empty all the items on the belt so that they can be scanned for pricing.
  • Fake-Scan: the checker avoids scanning one or more items intentionally in a transaction (sweethearting).
  • Tampered Scan AKA Ticket Switching: the shopper or cashier (in the case of sweethearting) tampers with the barcode label on the item and scans a different barcode than the appropriate one.
  • Invalid Override: the cashier fraudulently refunds, voids or otherwise invalidates a completed transaction and then pockets the money.

As shown by the diagram above, our solution consists of three principle components. The front end of the solution captures video and other transaction data. The middle piece of the technology processes the video and collates it with the transaction data (e.g., TLOG) based on the semantics of the video content. Finally, the back end of the solution permits the end-user (e.g., retailer) to interact with the processed data via a number of applications.