To some pundits cloud computing is just the next buzz word, the next marketing hype, the next fad, and nothing new. For every pundit, there is an innovator out there taking advantage of on demand resources at a price point that can’t be matched, with fewer resources and quicker to market. In this post I will show how my company used cloud computing to build an impressive high speed transaction network on a bootstrapped budget and went from unknowns to leaders in the market in just 2 years.
Marketers and technologists were building and deploying innovative solutions to get mobile and web content in the hands of consumers, but there was no standard, secure way to execute at the point-of-sale (POS). POS integrated solutions were expensive, invasive, and often required new hardware or required users to have smart phones thus alienating a majority of consumers.
The following presentations explain the problem M-Dot Network set out to solve two years ago to the day.
Build a standard and secure high speed network with seamless integration at the POS with no additional hardware. Consumers select digital incentives (coupons, web and mobile ads, digital gift cards, etc.) on the web or their mobile device and simply identify themselves at the POS via their loyalty card or mobile phone and automatically receive discounts without ever printing out anything.
Much like an ATM machine, the system must be highly reliable, scalable, and secure. In addition, the implementation of this high speed network should have a zero footprint (no hardware, no additional communication lines, etc) at the retail stores and headquarters and must respond in subseconds. The system must also be able to achieve high scale and handle large spikes in traffic to handle heavy shopping patterns such as special events, holidays, peak hours/days, new retailers joining the network, etc. The network must be up 24x7x365 and needs to have a robust business continuity and disaster recovery plan. Money is only made when transactions are occurring. Therefore, outage = lost revenue and angry consumers and retailers.
All of this must be done on a bootstrap budget with limited capital and a very small IT team.
Build a high speed cloud based network connected to a digital incentive Platform as a Service (PaaS). On this platform, channel partners can build marketing and analytical solutions to execute these campaigns in real time across a large network of POS systems for immediate redemption. M-Dot supplies the “rails” so marketers can focus on the consumers and not integration.
Step 1: Build a business architecture
In this post a year ago I talked about how we leveraged the E2AF framework to first define what our business architecture was before we chose what our technology was (Real time transactions coming soon to a cloud near you). We used the E2AF Cheatsheet that I created to answer the important who, what, when, how, where, and with what questions.
Step 2: Gather additional requirements
After processing all of the information in the E2AF cheat sheet, it was clear to us that we needed to build a highly scalable transaction network that required the security and compliance of an ATM machine. So we hired an expert in that field (ZSquad) to define all of the security and regulatory requirements that we would need to adhere to when processing digital incentives and consumer data. The E2AF cheat sheet information also helped us decide that the elasticity, low CAPEX, speed to market, and computing capacity of cloud computing was a no brainer for being our choice of our target technology platform. Since we had high speed performance requirements, we chose IaaS (infrastructure as a service) over PaaS (platform as a service) because we wanted total control over the operating system, application server, database, etc.
Step 3 – Evaluate IaaS vendors
Once we made that decision to use an IaaS provider, we hired another industry expert (David Linthicum) to help us with the IaaS vendor selection process and to review our proposed architecture to point out any risks or issues that we may have missed. Based on these factors: global presence, technical maturity, financial stability/longevity, thought leadership, available tools/services…our choice of Amazon as our IaaS provider was a no brainer.
Step 4 – Prototype
One of the best things about the cloud is that you can fire up instances in five minutes and try things out. My architect, Greg Rapp, who is an expert at high speed transaction processing, was able to try many permutations of virtual machines with different mixes of CPU and memory. He was also able to test different database storage engines like MyIsam and Innodb. He could do all this without the need of buying any hardware, requesting change requests to system administrators or DBAs. It was simply Greg and a self service platform, experimenting, testing, researching over and over until he was able to achieve subsecond response time. I summed this up in September 2009 in an article called using cloud computing to build better software.
Step 5 – Test scale
Once we built it, I hit the road with my CEO to sell the platform to grocery retailers. Many retailers doubted our ability to take each transaction out of their building, go to the cloud to process all of the business rules, and return to the store in subseconds across their hundreds of stores. No matter how much I talked about our backgrounds where we managed a network of 30K+ stores to deliver paper coupons on printers at the POS, they were just not convinced. I knew we needed to do a major performance test to prove our worth. In the technical assessment that Dave Lintihicum did for us, he recommended SOASTA as a testing as a service vendor. I called up the SOASTA guys and paid for a 10K concurrent user test. We were able to prove that our transaction network could handle 10K concurrent transactions with an average response time of 98ms sustained over 9 hours. At the same time, SOASTA was trying to find a customer that could handle a one million concurrent user test, something that had never been done before. SOASTA graciously offered to fund the effort for us because they believed our architecture could achieve this milestone. Sure enough, our transaction network passed with flying colors as we successfully delivered a million concurrent transactions with no errors over 1 hour sustained with an average time of 108ms (see press release). After that test, we were never doubted again by any retailer.
Step 6: Build the platform
Now that we had built our high speed transaction network and filed our patents, it was time to build our PaaS on top of Amazon. It is important to note that we have never spent a dime on actual hardware/servers or on software. Our entire development environment is open source (LAMP stack). All of our 3rd party tools are SaaS (Basecamp, Unfuddle, Sugar CRM, Google Apps, etc.). For more on the tools go here. We were able to get to this point with a total IT infrastructure spend of about $1K per month with just two developers, no system administrators, no DBAs, no nothing! We brought on a great offshore team to build our B2B and B2C applications on our platform. The key to this effort was our approach to dataservices. Architect Jack Hickman implemented our DJango framework and built a suite of RESTful web services so that our offshore team (and channel partners in the future) can simply connect to our platform without any knowledge of the underlying database structures and physical location of the database. I wrote an article called reaping the benefits of data services that goes into detail on our approach.
Step 7: Planning for production
The next step was to prepare both the real time transaction network and the PaaS for production. This is where the cloud really shines. We architected a fully redundant, self healing, auto-scaling environment that replicates all of our data and virtual servers across multiple zones. In essence, we have real time disaster recovery with all of our assets being used all the time. The following article (Taking the disaster out of disaster recovery) does a deeper dive into how the cloud makes business continuity and disaster recovery both feasible and reliable.
Step 8 – Launch
We just officially launched our first chain yesterday (press release). I am on cloud 9 right now – pun intended!
Things to think about
This has been a journey, almost 2 years in the making. I think it is important that I point out some things so that my readers can really understand the value of cloud computing. Here are some facts about our solution and path to implementation.
- We do not own a single server
- We do not have a datacenter
- We do not have a system administrator (although we will hire one in the future)
- We do not have a DBA (although we will hire one in the future)
- We have a fully redundant production environment across multiple zones
- Our solution is faster than our competitors who have dedicated servers in the stores
- Our monthly infrastructure bill including redundant prod, stage, and test environments in roughly $2500/month (this is insane!)
- Our production environment autoscales to meet the demands of fluctuating traffic
- We have not spent a penny on software licenses
- We have built a robust enterprise solution on a bootstrap budget
And most importantly, we are now the market leader in our industry with more retailer contracts than our competition and we did not exist 2 years ago. Although a lot of this can be attributed to the heroic efforts of our people and our leadership, none of this would be possible without the cloud. We would not even be in business if we had to raise the money to do this on premise. How many companies do you know that have built a high speed transaction network capable of delivering 1M concurrent transactions per minute in 1/10 second on $2k/month worth of infrastructure?
So the next time you see or here someone say the cloud is just hype or declare that “it is nothing new, we have been doing this for years”, send them this article and tell them to do their homework. ‘Nuff said!