Marty the Robot Rolls out AI in the Supermarket

By John P. Desmond, AI Trends Editor 

When six-foot-four inch Marty first rolled into Stop & Shop, the robot walked into history. Social robot experts say it is among the first instance of a robot deployed in a customer environment, namely supermarkets in the Northeast. 

Marty rolls around the store looking for spills with its three cameras. It does take the place of the human worker, called an associate, that did the same thing, but it means the associate can do something else. Doing the walk-around of the store is seen as a mundane task. 

Marty does not talk or tell jokes. Unlike Alexa, who many children in the store undoubtedly interact with at home, Marty will not respond. The robot does notify associates when it sees with its computer vision that something on the floor needs to be cleaned up, through the public address system. An associate comes over to clean it up, and presses a button on Marty that it’s done. Marty takes a picture of the cleaned-up aisle.  

Badger Technologies Rolled Out 500 Martys in December 

The AI in Marty is concentrated on the machine vision and the collision-avoidance navigation features, according to Tim Rowland, CEO of Badger Technologies, makers of Marty. Last December, after a year of trials, Badger rolled out 500 multi-purpose robots into Stop & Shop and Giant/Martin’s grocery stores on the East Coast. Each Marty is equipped with navigation systems, high-resolution cameras, many sensors and its software systems.  

When Marty gets to the store, it takes a trip in a shopping cart for 45 minutes to an hour to map the floor. “It produces a 3D map that it stores in its memory as its reference point,” Rowland said in an interview with AI Trends. The robot is instructed to “pose” at the top and bottom of each aisle as it makes its run around the store. At each pose, the three cameras on the right side of the robot are turned to look down the aisle and take pictures. It can see up for 70 feet, with its three cameras focused on close, medium and far distances.  

A primary design goal for Marty was to travel safely in the stores. “We were one of the first to introduce a robot into such a public setting,” Rowland said. “We would rather be working in a warehouse than out in the middle of many shoppers, so the primary goal was to operate safely with a lot of foot traffic around and to not bump into anything.” 

Beyond that, the robot was designed to scan the floor to see if any spills need to be cleaned up. “The imaging function is the second major area of AI on the robot,” Rowland said.  

The designers wanted the robot to be “in the background” as much as possible, so they colored it a muted gray. It gives off an audible beep to let shoppers know it’s in the area, and it has traveling lights. It weighs 140 pounds and costs $35,000. 

“The application of the ‘googly eye’ was somewhat of an accident,” Rowland said. One of the Stop & Shop associates took it upon herself to try to make the robot look more friendly, so she put some eyes on it. Now Marty has a social media following. On Saturday, Jan. 12, 325 Stop & Shop locations celebrated a one-year birthday for Marty, inviting kids in for refreshments and coloring books. Stories about Marty parties ran in many local newspapers and on TV news. Many videos of Marty are available on YouTube. 

Marty Anthropomorphized 

Anthropomorphizing is assigning of human traits, emotions or intentions to non-human entities. It is considered to be an innate tendency of human psychology. So by virtue of the Googly Eyes, voila! Marty is anthropomorphized. This has been a learning experience for Badger, which has not so far employed any industrial anthropologists to guide the introduction of Marty. 

Advertisement

“I’d love to tell you it was intentional and that we did studies, but it was a reaction on our part. It happened accidentally. And we have learned from that. It takes the edge off an upright piece of technology moving around a supermarket,” Rowland said. 

The public embracing of Marty is making a difference. “In hindsight, it did achieve the acceptance of a piece of high tech equipment more than it might have been. It softened the change of dynamics,” Rowland said. 

The robot’s height and camera placement enables it to scan upper shelves to spot out-of-stock or mis-priced products. Six stores were testing this capability when we spoke in early May.  

Marty A First 

Dr. John Sullins of Sonoma State University, Calif., an expert on computer ethics and the philosophical implications of technologies including Robotics and AI, has a range of experience with robotics in the workplace. “Marty is one of the first robotic systems we have seen that is operating autonomously in the same environment with customers,” Prof. Sullins said in an interview with AI Trends.  “On an assembly line, robots are commonly fenced off or otherwise physically separated from other workers for safety purposes. Systems like Marty have to be designed to move around a crowded aisle plotting a path to avoid customers and not rudely entering their physical space.  As humans, we do that sort of thing unconsciously, but for a machine to do something similar takes a lot of programming skill and years of product trials before it can be done safely.” 

The release of Marty into the “wild” of a supermarket is a green field for social robot researchers. Dr. Sullins, for example, is very involved with developing robot ethical standards within IEEE. One area of study is the setting of expectations between humans and the robots. “Robot ethics requires transparency in human robot interactions,” Dr. Sullins said. Marty’s job in the supermarket is not necessarily spelled out to shoppers, leading to speculation about what the robot is doing. “We need to convey that kind of information quickly to the general public as it interacts with machines like Marty,” Sullins suggested. 

Some of Marty’s behavior Dr. Sullins would classify as “nudging,” actions of intelligent, robotic systems designed to influence behavior, such as moving pedestrian traffic away from spills and directing human workers to attend to the spills. “More subtle nudges are the googly eyes, the Easter Bunny decorations and Covid face mask, which are designed to make people feel more comfortable around the machine,” he said. 

[Ed. Note: Dr. Sullins and his compatriots in the IEEE are looking for AI professionals interested in crafting standards for the ethical use of intelligent autonomous systems. Dr. Sullins is co-chair of P7008 – Standard for Ethically Driven Nudging for Robotic, Intelligent and Autonomous Systems (https://standards.ieee.org/project/7008.html). Dr. Sullins sees Marty demonstrating some nudging behavior.]  

Badger: an Entrepreneurial Success Story 

The early version of Marty had a tablet computer mounted on the side that the robot designers thought could help customers locate products, and essentially open the door to customers communicating directly with the supermarket robot. But the timing was not right and that plan was shelved. “The early conclusion was that it distracted it from the main job,” Rowland said. “A future application could be verbal; that makes sense. It could be a future requirement.”  

Marty has been good for Badger Technologies, which Rowland founded in 2017. He had spent 28 years in various engineering capacities at Lexmark, the printer company located in Lexington, Kentucky. He led an innovation project team focused on retail robotics; the team selected Jabil to do the advanced manufacturing required. The first live demo was conducted in 2016. Soon after, Lexmark management decided to continue its focus on printers and imaging products, so Rowland and a 12-person team formed Badger. 

Advertisement

Jabil helped Badger in a joint development effort, enabling the team to move into retail in the spring of 2017. By late summer of that year, Jabil acquired Badger Technologies as an independent product division of the global manufacturing company’s operations. Jabil describes itself as a “manufacturing solutions provider that delivers comprehensive design, manufacturing, supply chain and product management services.” Jabil was founded in 1966 and is based in St. Petersburg, Fla.   

In early 2018, Badger began a pilot robotics rollout with Ahold Delhaize USA Brands, owners of Stop & Shop and Giant supermarkets. The supermarket business is tough, with margins hovering in the one to two percent range.  Ahold Delhaize of Zaandam, Netherlands, was formed with a merger of two companies in 2016, with the idea of squeezing out some savings and diversifying from conventional retail brands. The Ahold Delhaize USA division oversees Food Lion, Giant Food, Giant/Martin’s, Hannaford, and Stop & Shop, as well as e-grocer Peapod; Retail Business Services, a U.S. support services company providing services to the brands; and Peapod Digital Labs, its e-commerce engine. Operating more than 2,000 stores across 23 states, Ahold Delhaize USA is No. 4 on Progressive Grocer’s 2019 Super 50 list of the top grocers in the United States.  

Labor issues hit Stop & Shop in 2019, with 30,000 workers going on strike for 11 days, costing the company between $90 million and $100 million, according to an account in The New Food Economy The strike was resolved, the workers came back, and Marty still patrols the aisles, having become well-established and friendlier. 

Source