M-Dot Network Enables Retailers to Reliably Deliver Coupons without the Clipping
The couponing industry is moving into the Internet age as devices such as smartphones, tablets and gaming systems, in addition to traditional desktops, deliver digital content to users. However, there is no industry standard solution for integrating digital content with retailer point of sale (POS) systems. Designed to remedy the problem of numerous retail integration points, M-Dot Network is an open platform that sits between the hundreds of companies producing digital content and the retailer POS. The M-Dot Network solution provides a single point of integration that connects the coupon issuers to the retailers and their clearing house.
“As an architect of multiple large-scale performance testing efforts on behalf of Fortune 500 companies, I am exceedingly pleased that we contracted with SOASTA.” – Jim Giddings, Director of Quality Assurance
The M-Dot Network platform enables consumers to receive digital coupons via a retailer’s website or through a mobile device. M-Dot realized that their new platform had to respond quickly when consumers redeem digital coupons or gift cards upon checkout. Retailers are very protective of their POS systems and M-Dot needed to reassure potential partners that the network could reliably scale to accommodate thousands if not millions of concurrent transactions from multiple retailers and locations.
M-Dot started researching testing solutions with an eye for a fast and affordable solution. They needed a solution that could test an environment hosted entirely in the cloud through Amazon Web Services. M-Dot chose SOASTA for its cutting edge technology, industry experience, cost and the ability to meet M-Dot’s accelerated timeframe. This was particularly key as alternate offerings estimated far longer turnaround when compared to SOASTA’s ability to quickly certify their performance, with a goal of maintaining less than 100ms response time at full scale.
“As a start-up I have only one person in testing and limited funds so I needed to use smart money,” said Mike Kavis, CTO, M-Dot Network. “We had to test very, very quickly because we had already built our system and were in the sales process with an aggressive go-to-market plan. We just needed to give the retailers more confidence that the system could deliver the performance levels they needed. By working with SOASTA, we were able to do so almost immediately.”
Validating Transactions vs. Users
The M-Dot network is a “headless system”, meaning there aren’t large numbers of users hitting a website. Instead, M-Dot processes massive amounts of POS generated transactions through its system. The traffic comes from many stores across the globe; each sending multiple and concurrent shopping transactions to their AWS hosted servers. M-Dot had to ensure that they did not have any latency while handling mass distribution of offers. Their goal: 100,000 concurrent transactions (or, the equivalent of 20,000 stores with 5 checkout lanes each).
“Performance is critical for us. A second lost at a store’s POS can cause a loss in the millions of dollars, as well as consumer loyalty — they won’t wait around for a redemption that takes more than seconds to complete. We work on behalf of the retailers and if we slow them down, they kick us out. It’s just that simple,” said Kavis.
Using SOASTA’s Performance Test Methodology, M-Dot designed a test with 5 scenarios, based on the number of items in a shopping cart and the spread of locations, with a relatively aggressive ramp time. 50% of the carts were checking between 3 and 14 items and 40% simulated 15-50 items. The remaining 10% represented outliers on either end. The test simulated 1.5 seconds between items and 5 seconds between carts and generated just under 2800 requests per second. In the final soak test, SOASTA certified the M-Dot system could maintain a response time less than 100ms with 10,000 concurrent lanes of traffic over a 9-hour period.
SOASTA Performance Engineer Jeremy Whitener worked closely with M-Dot. “I have worked with multiple Silk, Compuware, and Mercury performance test engineers. During proofs of concept, I have been privileged to work with some of the best in the industry. Jeremy ranks at the top with the best,” said Jim Giddings, Director of Quality Assurance at M-Dot.
This initial test was enough to demonstrate scalability to retailers. In order to better understand how well they could scale when pushed, SOASTA also executed a 1M-transaction test that spanned two sessions, each lasting approximately 4 hours. M-Dot leveraged the scalability inherent in EC2 to deploy additional servers to handle the increased traffic.
On the first day the scenarios used in the initial M-Dot testing were validated at peak transaction rates, which also uncovered significant issues in the system, particularly with the load balancer configuration and proxy server when the transaction level hit 800K. The 1M goal was reached early on day two, using an aggressive ramp to stress the system. Average response time remained at less than 1/10 of second, achieving the speed required for a user redeeming coupons at checkout and proving the scalability of the application in AWS EC2.
First Impressions are Everything: Adopting Performance Testing
The test results report that SOASTA provides each customer has proven vital to M-Dot to show that they can deliver high performance numbers and it’s become a key part of their presentation on sales calls.
Due to the event-based, massively parallel, multi-threaded architecture of CloudTest, only 587 load and result servers were required to reach the target transaction level. Using the SOASTA Global Test Cloud, all of the servers were provisioned from Amazon EC2 and ready to test in less than 10 minutes. The efficiency of the platform dramatically reduced the challenges associated with the preparation and execution of a test this large.
M-Dot has integrated SOASTA CloudTest into its testing environment. Anytime the company makes changes their architecture or needs to hurdle a new barrier, they look to SOASTA for help.
