Whether you are deploying applications or services in the cloud or on-premise, you can still leverage the cloud to build better software. Two areas where the cloud gives you huge advantages over on-premise computing are prototyping and testing. Let’s explore.
|From Cloud Computing|
On-premise model: A typical approach for on-premise is to perform calculations based on estimated bandwidth requirements, transaction volumes, capacity planning, etc. The architect will weigh a number of these factors and build a prototype on whatever hardware is available to them and make projections of potential hardware needs based on the prototype.
Off-premise (cloud) model: The architect fires up different size instances and runs the prototype against each to find the best performance. The architect has access to virtually unlimited computing resources at marginal costs (80 cents an hour for an extra large instance on Amazon) and can fire up tens or even hundreds of instances to run tests of worst case scenarios and tear them down when finished. The pay as you go model of cloud computing allows architects to experiment with different computing configurations against different types of payloads that is neither feasible or timely in an on-premise setting.
My findings: At M-Dot Network, we were able to experiment with various configurations across various different size virtual images. We could have a new environment up and running in about 5 minutes by running some simple scripts. These environments included a full functioning LAMP (Linux, Apache, MySQL, Python) stack running on an Amazon EC2 instance without procuring any hardware, depending on a system administrator, or competing for any IT resources (people or infrastructure) at any time. When we completed our testing we tore down the instances and our costs for the day would be about $100. The benefits I saw were as follows:
- Speed to market - the prototype phase was reduced by weeks or even months
- Cost - Some of the scenarios we tested were not even feasible in the on-premise model.
- Quality - Due to the feasibility, we were able to run more scenarios and run extreme stress tests to prove out our architecture.
- Confidence - Because we could stress test our architecture at extreme worst case scenarios, we learned about our bottlenecks and roadblocks before we went to production. We were able to test for projected volumes several years out and validated our platform could handle it.
On-premise model: Performance testing is limited by the amount of computing resources and software licenses for testing tools like Load Runner and others. Some companies have huge budgets and can simulate massive volume tests. Other companies do the best performance testing they can do within the budget constraints and licensing limitations they have.
Off-premise (cloud) model: Architects can leverage the pay as you go cloud model to simulate huge loads and only pay for the compute cycles they use during the hours that they test. An even better model is to leverage a Testing-as-a-Service model. In this model, architects can leverage a technology partner to offload the processing and reduce the costs of computing power and software licensing.
My findings: At M-Dot Network, we used SOASTA as our Testing-as-a-Service provider. We were able to simulate 1 million concurrent transactions sustained over a period of time for an extremely low cost. We worked with their engineers for a few days so they could develop the appropriate testing scripts to drive the volume and gather the metrics. We were up and running this test in days and quickly found and eliminated our bottlenecks. On their side, they fired up over 500 virtual machines. On our side we fired up over 50 virtual servers based on our estimates. According to Tal Broda, VP of Engineering at SOASTA, a test of this size would have required over 7,500 Load Runner licenses had we used the traditional on-premise methods. What we found out is that our servers were only at 40% capacity and that we actually needed less horsepower than we expected (glad we didn’t have to buy!). We also were able to prove out that our architecture could support over 200,000 stores sending us transactions concurrently. This number easily exceeds the number of stores we anticipate having 3-5 years from now. In essence, this test told us that we could stop tweaking and fine tuning our transaction processing engine and move on to other tasks. But most importantly, it proved to us that our architecture can support our long term business needs.
To conclude, we leveraged the cloud to enable us to perform extensive prototyping and performance testing. We were able to validate our architecture at low cost and figure out exactly what our target infrastructure should look like. In my past experiences, we would do the best we could with the limited resources and budget we had. Sometimes we got it right, sometimes we didn’t. Many times we encountered bottlenecks down the road when volumes increased because we were never able to stress the system high enough to see the issues first hand.
The real point of this story though, is you can use the cloud for prototyping and performance testing even if you have no plans on deploying in the cloud. Testing-as-a-Service companies like SOASTA can test performance of both on-premise and off-premise solutions. So even if your company doesn’t have the balls to deploy in the cloud, they would be doing their business an injustice if they don’t consider the value proposition of the cloud for prototyping and performance testing.