Big data

Walmart started using big data even before the term big data became known in the industry and in 2012 they moved from an experiential 10-node Hadoop cluster to a 250-node Hadoop cluster. At the same time they developed new tools to migrate their existing data on Oracle, Netezza and Greenplum hardware to their own systems.

The objective was to consolidate 10 different websites into one website and store all incoming data in the new Hadoop cluster. Since then they have made big steps in integrating big data into the DNA of Walmart.

Social big data solutions

Many of the big data tools have been developed at the Walmart Labs, which was created after Walmart took over Kosmix in 2011. Some of the products that were developed at Walmart Labs are ‘Social Genome’, ‘ShoppyCat’ and Get on the Shelf.

The Social Genome product allows Walmart to reach customers, or friends of customers, who have mentioned something online to inform them about that exact product and include a discount. In order to do this they combine public data from the web, social data and proprietary data such as customer purchasing data and contact information.

 This has resulted in a vast, constantly changing, up-to-date knowledge base with hundreds of millions of entities and relationships. It helps Walmart to better understand the context of what their customers are saying online.

An example mentioned by Walmart Labs shows a woman tweeting regularly about movies. When she tweets “I love Salt”, Walmart is able to understand that she is talking about the movie Salt and not the condiment.

Walmart came across several technical difficulties when developing the Social Genome, among others the quantity and velocity the data pours into their Hadoop clusters. As the regular Map-Reduce/Hadoop framework was not able to cope with the amount and speed the data was coming in, they have developed their own tool called Muppet. 

This, now open-source, tool processes the data in real-time over all clusters and can perform several analysis at the same time.

The Shoppycat product that was developed by Walmart is able to recommend suitable products to Facebook users based on the hobbies and interests of their friends. It uses the Social Genome technology among others to help customers with presents for their friends. An interesting aspect of this Facebook App is that Walmart will direct the Facebook users to a different store in case the product is sold out at a nearby Walmart store.

QUANTUM Global Academy is famous for providing training in Hadoop & Big Data, ITIL, PMP, Six Sigma, event management, retail management and logistics & supply chain in Gurgaon, Delhi. We are here to help for improving company’s performance and productivity. For more information visit our website quantumglobal.org/ or can call us at 01244609530, 08527092037, or click on here.