Author Archive
How the cloud has changed testing forever
While many companies still debate whether it makes sense for them to look to the cloud for their application and data center needs, there should be no debate that the cloud is the place to go for testing. In this post I discuss three examples of how my company (M-Dot Network) has leveraged the cloud for testing in ways that are not even feasible in the on-premise world.
Prototyping/Discovery
Problem: We were building a high speed transaction processing system and did not know how much memory and CPU would be optimal for the architecture.
Pre Cloud – In most of the previous projects I worked on in other companies, we typically had to purchase our hardware early in the life-cycle due to long procurement cycles and setup time. Usually we had to order this way before we started prototyping which means we had to take our best guess. The consequences would normally be that we would have to tailor our solution around the hardware that we purchased or existing hardware that we were forced to use. The end result was usually a less than optimal performing solution due to hardware constraints. Instant legacy, how nice!
Cloud - The procurement and setup issues go away because resources can be provisioned in minutes in the cloud. This allowed us to get deep into the prototyping phase and then test our design against numerous combinations of CPU, memory, and database subsystems. The result was the complete opposite of the old days. Instead of rigging our design to meet the constraints of hardware, we were able to provision the virtual servers that met our performance requirements based on actual test results. That is nirvana!
Performance Testing
Problem: It is very time consuming and expensive to create performance tests that can simulate millions of transactions from multiple touchpoints.
Pre Cloud – Unless you were testing on a soon to be production system, you had no way to do true performance testing without having a replica of your production environment, plus enough hardware/infrastructure to simulate loads that far exceed your current projections. After all, it makes no sense to only test the amount of traffic that you expect to see in production without testing spikes and testing for future growth. I would also argue that one should test the system to see at what load the system starts to break or perform sub-optimally. In the on-premise world, buying additional capacity just to do performance tests coupled with the incredibly high costs of software licensing from tools like Loadrunner is just not feasible. So companies rarely are able to simulate with large enough loads to test scale and simply can’t afford to provision test systems that are equivalent to their production environments.
Cloud - We leveraged Testing as a Service. We spent 2 weeks with our technology partner SOASTA and were able to script and deploy a 1 million concurrent user performance test. SOASTA fired up 587 servers and we fired up over 50, that is 600+ servers “procured” and installed in an hour. We ran our simulation through out the day testing bursts and sustaining a load of 1 million concurrent transactions and were able to prove that our network could handle transactions for over 200,000 stores concurrently! After we finished the test we tore down all 600+ servers and only paid for the time we used them. Imagine trying that on-premise! A 2 week setup and test of 1 million concurrent transactions per minute leveraging 600+ servers at a very low cost without adding headcount, infrastructure, software licenses, etc. Without a solution like SOASTA’s Cloudtest, we would not even had attempted a test of this magnitude. We actually discovered and fixed a few bottlenecks along the way. Without this test, those bottlenecks would have been discovered in production and would have impacted our customers.
Resources and Mobile Devices
Problem:
We recently completed our social media manager tool to allow our retailers to manage their SMS groups, Twitter, & Facebook updates from one place (including issuing coupons). To test this we needed to have access to numerous types of mobile devices, carriers, geographical locations, and an army of temporary testing resources.
Pre-Cloud:
In the old days, I would have to purchase, lease, or borrow a large collection of mobile devices across a wide variety of carriers. I would also need to put people in various areas across the country to truly test the experience as the retailers’ customers would experience it. After all, I would rather encounter coverage/carrier issues before consumers do. I could also outsource this to a technology provider for some heavy costs. Either way, this is an expensive undertaking using traditional methods.
Cloud:
Instead, we partnered with cloud based crowd sourcing testing partner, UTest, who has testers scattered throughout the world with every handset/carrier combination imaginable. They also have a web based defect tracking and collaboration platform. So with a simple phone call and a very low cost, we set up and executed a three day test that covered most of the major phones, from smart phones to basic mobile devices, and across numerous carriers. In addition, we loaded up on testers who lived in the areas close to the retailers who would be using this service. This gave us mass coverage of devices and carriers for an extremely low cost and over a very short window of time. Agile, cost effective, and efficient!
Summary
These are just three examples of how the cloud is changing testing forever. All three of these scenarios can be tested in the cloud regardless if the actual solution to be tested was built in the cloud or on-premise. Even if you are building your solution on-premise, you can still do the initial prototyping on the cloud to figure our how much CPU and memory you need to buy. Many of both SOASTA’s and UTest’s customers have on-premise solutions. So even if your company fears the cloud as a solution for deploying applications and building data centers, there is no excuse for continuing to spend way too much time and money testing things the “way we have always done it”.
What corporate IT should learn from Startups: Part 2 – Roadmaps
As I have mentioned in numerous posts over the last several months, I am finding that things like process, governance, architecture, SOA, cloud computing, and others are much easier in my new startup world than in my old corporate world that I battled in since the 80’s. Even though I never intend to return to the corporate world I feel obligated to share with my colleagues in the corporate world because I know how hard it can be innovate and promote change in established cultures. In part one on process, I recommended creating a startup atmosphere by building a small team free from the constraints of the corporate setting.
In this post I will focus on roadmaps. Whether you are building a roadmap for you overall architecture, for a portfolio of projects within a given domain of your architecture, or for reengineering business processes, roadmapping can be a challenge because of large amounts of legacy systems and ingrained behavior. Many roadmapping exercises start with a long process of capturing the current state. Often this leads to analysis paralysis and lots of time and money is spent while nobody is building the future state.
WWSD?
What would a startup do? Well, a startup’s current state is that they have a blank sheet of paper and an opportunity to build the best possible solution with no legacy constraints. Hmm, doesn’t that sound attractive? I wrote a post back in 2007 called Getting to Future State where I recommended designing the future state first and then capture the current state later. The reason is simple, if you start with the current state you immediately constrain the innovation process for future state. Why not start with the perfect world and work back instead of starting with an imperfect world and adding to it?
Starting with current state can create undesirable results
Starting with future state can increase your chances for a desirable outcome
WWSD? A good startup would map out what it wants to be when it grows up first and then work towards that goal while carefully managing its precious resources and capital. A startup will also deliver early and often because it has to generate revenue, customer interest, and investor enthusiasm before it goes broke. That is exactly what a corporation should do! Deliver early and often making incremental improvements and proving its value to the executives (corporate equivalent of investors).
I have seen and been involved in too many promising projects where the next new technology, process, or organizational change was going to solve all of the world’s problems. Each time these initiatives fell short of expectations and each time these new solutions were just another layer on top of the last solution. Each layer added a new layer of complexity and legacy on top of the previous layer. The reason for this is these teams started with what existed and figured out how to “wire in” the next technology, instead of figuring out how to move off or abstract parts of the legacy systems in order to take advantage of the newer technologies.
So the next time you have a roadmapping exercise, don’t start by analyzing what exists. Start with a blank sheet of paper and ask “If we were a startup and were starting business today, what would the future state look like?” Once you define what the perfect world looks like, then figure out how to get there. You will likely find that you will have to make some sacrifices here and there but at least your innovative thinking was unconstrained when you envisioned the future state!
What corporate IT should learn from Startups: Part 1 – Process
I have spent well over 20 years in corporate IT environments, most of it working for IT shops in the 100-300 person range. In every company that I worked for, IT was seen as a bottleneck and IT struggled to satisfy the needs of the business. In many companies, this is the standard. The reasons for these struggles can be boiled down into the following categories:
1) Process – Too much, too little, or the wrong process for the organization
2) Architecture – No focus on EA (”Wild West”), vendor driven, or Ivory Tower Syndrome
3) Culture – Silos, change resistant, IT thought as a cost center, wrong or unmotivated people
4) Priorities – No portfolio type thinking, decisions made at the wrong level, lack of accountability/justification
In the last 18 months, I have worked in a startup with a team of 10+ employees advisors, partners, and consultants. Many of the above issues are not a problem in our environment for the following reasons:
1) Process – Teams are small, easy to communicate, everybody depends on everyone else
2) Architecture – We see architecture as both a necessity and a competitive advantage
3) Culture – Survival depends on alignment, agility, and change
4) Priorities – Every penny spent better contribute in some way towards a penny earned
Established organizations can usually survive and sometimes thrive even despite their deficiencies in IT because their core business is bringing in sufficient revenue. For startups, I have seen some great business models fail because of the company’s inability to execute. Startups, especially those in early stages, rarely succeed if they are inefficient. So how can larger, established companies create that entrepreneurial environment that brings the same motivation, accountability, and alignment that comes naturally in startups?
What does success look like?
Often, companies like to create a profile of their most successful employees so they can use this profile as a hiring tool for identifying talent that matches the best people in the company. I recommend that any company who recognizes that it needs to make changes to improve delivery and business/IT alignment create a profile for what a successful startup looks like. Notice I did not say what a successful well established company looks like. The reason is simple. Startups typically deliver frequently, with little capital, with just enough features, and with more modern solutions. Isn’t that what established companies really want at the end of the day?
So why do startups tend to move faster and innovate more?
1) Smaller teams
2) Better communication/alignment
3) Smaller budgets = less features and shorter deadlines
4) Employee incentives (survival) are directly tied to results
5) Everybody matters, everybody contributes
6) Not married to legacy systems, processes, cultures
7) Everybody sees the big picture
So how can leaders in IT create this kind of environment that fits the profile of so many successful startups? Transforming an entire organization can be an ominous task. We have seen many companies fail implementing new technologies because of the inability to change the culture. It takes buy in at the highest levels and great transformational leadership to change a company’s culture and motivation. Maybe the solution is simpler. Create a “startup” within your organization and ask yourself, “What would a startup do?”
WWSD?
A good startup would build a business plan that shows investors what it will build, how it will generate revenue, how it will keep its burn rate to a minimum, how it will deliver quickly, and why it is better than the competition. Then the startup would build a small team with its limited funds and create extremely aggressive goals and targets that are back loaded with incentives for employees pending on the outcome of their delivery and future funding. Then the team would be empowered to do whatever it takes to meet those deliverables within the limits of the existing funds and resources. In a startup, it takes a certain mentality for an IT person to survive and thrive in this type of environment. Make sure that only those types of people are included in this internal startup. It only takes one corporate attitude to destroy a startup.
To meet the commitments, team members will have to heavily rely on one another. They will also need to innovate to figure out how to balance features, functionality, and quality. Decisions will need to be made quickly and documentation will have to be just enough. In my startup we call this JEJIT - Just Enough, Just In Time. Our requirements and design documents are more visual than textual. You won’t see any 150 page requirements documents because nobody has the time to create it. Instead you will see an iterative process where we get just enough requirements to throw a prototype together. Then we show the prototype to the product owner and iterate through changes until the requirements are good enough for a demo. The first slide of the following presentation shows how we iterate through each phase.
Startups, cloud computing, and the freedom to innovate
I was the guest writer for this week’s Zapflash article for the guys at Zapthink. I wrote a piece talking about how startups have been the ones innovating with cloud computing while the bigger, established companies are still trying to understand the pros and cons of the cloud.
In this article I discuss my own personal experience where the well established paper coupon industry is in a mad dash to transform itself to paperless. The market leaders in this space are bogged down with legacy systems and executing against their current business models. In the mean time, agile startups with no legacy systems, no data centers, and a entrepreneurial cultures are quickly advancing this space and effectively becoming the service providers for the paper giants at a price point that can’t be matched with on-premise computing solutions.
Read the article and you will see how cloud computing is changing the landscape of business and how those that resist this change may be hurting their business in the long run. If you know of similar examples in other industries, please share.
Zynga launches Cloud Wars on Facebook
Zynga , the #1 social gaming company on the web and makers of popular games Farmville and Mafia Wars has just released its newest game called Cloud Wars that will go live on Facebook at 5pm today. Sarah Featherstone, spokesperson for the new game says that the recent struggles of the IT community to define what cloud computing is coupled with the deceiving marketing messages that traditional hardware and software vendors have been spreading all over conferences, blogs, and magazines have given them the idea. Cloud Wars, based on the popular game, Mafia Wars, allows gamers to start off as a lowly mainframe programmer and work their way up the ranks to VB programmer, security specialist, Rails developer, and even blogger. The goal is to reach the CEO level of a multi-billion dollar cloud computing company. The ultimate ranking is the King of Cloud CEO. At this level, the gamer can buy any company they wish and give keynote presentations declaring that the cloud is a bunch of hogwash.
Like the famous Mafia Wars game, gamers are given tasks where they earn experience points and coins that help them advance to higher levels. There is also the fight section where technologists argue whether there is really such thing as a private cloud, whether cloud computing is new or something we have been doing for 30 years, or who should get kicked off of American Idol. Gamers can add friends to their project team and exchange gifts or gang up on other project teams or vendors. As gamers gain status and climb the corporate ladder the tasks get harder. As a programmer, gamers can easily crank out tasks and gain experience points, but the coins always seem to go to the higher ranked players. As players reach the security or architect roles, it becomes increasingly difficult to get anybody to implement any of the Visio diagrams and UML models that the gamer spent many days and lots of coins delivering. But things get easier when the gamers reach the management level. Golf outings, delegation tasks, and 3 day Las Vegas conferences are easy tasks to accomplish and large amounts of experience points and coins are earned along with some vendor t-shirts.
At the end of each level, the gamer must complete a deployment task. The higher the rank, the more friends the gamer will need in his/her project team to get the deployment done. Deployments in the early startup ranks are much easier but there is very little coin involved. The higher ranks can simply buy up the startups after they have successfully deployed and gain major experience points and coins. If the team members from the startup organizations do not shower the higher ranking players with gifts, they may find themselves demoted back to a mainframe programmer and their red stapler will be taken away.
Ms. Featherstone states that this is the first of their new series of “reality strategy games” that they have created.
“Gamers are starting to get bored with planting crops on farms, tending zoos, and playing boring card games”, she says. Instead gamers want to “escape the monotony of their real world jobs, actually have a chance to become successful, get annual raises that exceed 2%, and tell their boss to take a hike. This game provides all that and more”.
I interviewed a few beta testers of the game. Most of them loved the realistic game play as it pertained to climbing the corporate ladder but the best part of all, as one gamer told me is “in the gaming world, I actually get to deploy a real enterprise application in the cloud!”. Apparently, that is a goal that is unrealistic in the real world according to many people.
Ms. Featherstone says the team is busy on the next reality strategy game called “Paling with Palin”. The goal of this game is to tour the country in a bus to get as many people as possible to hate both the Democrats and Republicans and make a powerful third party called the Tea Baggers. The top gamer gets to own Texas after they force the state to succeed from the union.
Happy April 1st! ……with apologies to Zynga and Facebook
Lawson CEO eats crow and deploys on Amazon
Do you remember this article in 2008 when Lawson CEO Harry Debes made a bold prediction that the SaaS market would collapse in 2 years? He also made some classic quotes that he most likely regrets like these:
ZDNet: Won’t people avoid the mistakes of “previous” SaaS incarnations, as you mentioned?
Debes: People are stupid. History has shown it repeats itself, and people make the same mistakes.Debes: …”Getting signed up as a SaaS customer is fast, but getting out is just as fast. Whereas traditional software is like cocaine–you’re hooked.”
Debes: One day Salesforce.com will not deliver its growth projections, and its stock price will tumble in a big hurry. Then, the rest of the [SaaS] industry will collapse.
That article set the blog-o-sphere on fire for several days and the article on ZDNet had tons of comments, most of them very negative towards Debes.
So today Amazon’s CTO Werner Voguls tweeted this:
Lawson Announces Full-Function ERP on Amazon Web Services Infrastructure
The first thing that came to mind is that they must have a new CEO. So I went to their website and found that Debes is still there. So I read the press release from Lawson’s site and look what they are saying now:
“Commodity software may be good for the vendor but not necessarily the customer”
“This should be great news for CFOs and CIOs who worry about lengthy and complex on-premise installations, the cost and inefficiency of their data centers, the best way to allocate IT staff, and the complexity and difficulty of maintaining software versions and upgrades.”
Wow, what a change of heart! Looks like SaaS is not dead after all. Maybe SaaS is the new cocaine! What I would be interested to know is if Lawson’s move to the cloud was a port of legacy code with some tweaks or did they totally rearchitect their system to take advantage of the AWS. I ask this because I managed a team that maintained Lawson a few years back and its architecture was archaic and outdated at best. I’ll spare you the gory details. The press release did mention the use of S3 so there has to be some changes on the backend.
I would love to see ZDNet reinterview Mr. Debes and see what he thinks about SaaS now. Does the phrase “Eat some crow” sound relevant?
OSBR April edition on Cloud Computing
I had the privilege of being the guest editor for the April edition of the Open Source Business Resource. The topic for April is cloud computing. In my opinion, a big part of the success of cloud computing is the extensive use of open source software, especially the LAMP stack, which lowers the overall costs of the cloud services. With cloud computing, the operating system, the application server, and the underlying hardware has become a commodity.
I was fortunate enough to get some well known writers to contribute articles on cloud computing from many different perspectives. Read my editor’s notes to see my thoughts on open source and the cloud and to see a summary of the articles. Here are some excerpts:
Cloud computing may be the biggest game changer within the enterprise since the adoption of the Internet in the 1990s and the personal computer in the 1980s.
The LAMP stack has become widely adopted as the standard engine running much of the cloud services. With the exception of Microsoft’s Azure cloud platform, most cloud service providers have embraced open source software, allowing them to drive costs down while providing reliable services for their customers.
Pay-as-you-go is the new economic model for IT as we enter a new decade. Gone will be the days of making large purchases of commercial software with huge maintenance costs.
Open source software will play a huge role in making the shift to cloud computing economically feasible. At the same time, commercial software companies are racing to the cloud and are struggling to replace their expensive software licensing models with a pay-as-you-go model in order to make them an attractive alternative to open source software in the cloud.
Here is a list of the authors and their topics:
David Linthicum – The value of cloud computing
Fred Waldner – Cloud Computing: What is it, and How Will it Affect Organizations?
Daniel Crenna – Re-evaluating Open Source for Sustaining Competitive Advantage for Hosted Applications
Ronald Schmelzer – Private Clouds: Reality or Fog?
Tom Lounibos – Performance Testing From the Cloud
John Crupi & Chris Warner – Enterprise Mashups: Cloud-Based, Cloud-Driven and Cloud-Derived Applications
I thought all of the authors wrote excellent articles. Probably the most thought provoking article was from Daniel Crenna. I highly recommend you read his article and comment on it. It really got me thinking about how the shift to the cloud will change how we view commercial software forever.
I hope you enjoy reading the content as much as I did! A special thanks goes out to editor Dru Lavigne for organizing and editing all of this content and for patiently waiting for me to finish my part! You can follow her on Twitter at @osbr
Cloud innovators and cloud pretenders
Randy Bias wrote a good post today discussing which vendors are leading the cloud race. I totally agree with his assessment that pure play cloud service providers are leading the way with more innovation and speed to market for new features.
In contrast, traditional hosting companies moving into cloud computing are hobbled by running two teams: development and operations. Expect the gap to widen as more hosting companies continue to misunderstand that this race isn’t about technology; it’s about people, software, and discipline.
I think these results are driven by the reason why these vendors are in the cloud to begin with. The thought leaders like Amazon, Google, and Salesforce are early pioneers in this space and have built sustainable business models for offering cloud services to enterprises. The pretenders, also known as hardware and software vendors, are running to the cloud because they see a future where it no longer makes sense for enterprises to spend huge sums of money on infrastructure and packaged on-premise software. Unfortunately for the pretenders, their organizations are not built to compete in the world of cloud computing. It’s the man behind the curtain syndrome for the pretenders.
So these vendors hit the road and attend conference after conference filling up panels and keynotes with their own cloud “experts”. Meanwhile, while the pretenders continue to rebrand their existing offerings as cloud-ware, the innovators just continue to deliver more robust APIs, expand data centers to other parts of the world, and lower prices.
Can the pretenders make it?
Some will, some won’t. I believe that the software vendors have a legitimate chance to compete in the SaaS and PaaS markets, especially Microsoft. The biggest challenge for the software vendors is how to price their services so they can bring in the same revenue streams that they are used to in the on-premise world. It will be a greater challenge for the hardware vendors. Their strategy seems to be to convince enterprises to build their own private clouds, something I strongly oppose with few exceptions. The following image represents my view of what you really get when you build your own cloud in your data center.
It will be interesting to see how this plays out over the next few years. I think that some of these big hardware vendors will eventually concede after a while and simply start buying up some of the smaller pure players who have the ability to innovate. They will always be able to persuade their current customer base to buy, but that does not equate to significant incremental revenue or new customers. One hardware vendor I do think will succeed and even thrive is Cisco. Both cloud innovators and cloud pretenders will continue to buy large amounts of Cisco products. It is the server companies that will struggle.
I would love to hear your thoughts on how you think this will play out.
NoSQL vs. RDBMS: Apples and Oranges?
The last few weeks I have witnessed a ton of passionate debate about what is better, NoSQL or RDBMS. It almost sounds like a religious battle between Windows and Mac fan-boys. To me this is like arguing that a hammer is better than a screw driver. If you need to pound a nail, I’ll take the hammer. If you want to turn a screw, I’ll take the screw driver.
First, here is a little history on how we got here. As telecommunications and storage costs dropped due to advancements in those technologies, it became more feasible to store larger amounts of data than ever before. This led to marketers investing heavily in targeted, one-to-one marketing in order to better understand and influence potential consumers of goods and services. As the databases grew larger, the queries started taking longer and longer. Eventually, engineers started looking for better solutions because relational databases (RDBMS) where not able to keep up with the demand.
RDBMS
Relational databases are great for enforcing data integrity. They are the tool of choice for online transaction processing (OLTP) applications like data entry systems or on-line ordering applications. RDBMS requires that data is normalized so that it can provide quality results and prevent orphan records and duplicates. It uses primary and secondary keys and indexes to allow queries to quickly retrieve data. But all of the good intentions that the RDBMS has for ensuring data integrity comes with a cost. Normalizing data requires more tables, which requires more table joins, thus requiring more keys and indexes. As databases start to grow into the terabytes, performance starts to significantly fall off. Often, hardware is thrown at the problem which can be expensive both from a capital standpoint and from an ongoing maintenance and support standpoint.
So many data architects moved to denormalized data structures to solve the problem. With denormalized data structures, extremely large databases are extracted and flattened and put into structures like a star schema for faster retrieval (see diagram below).
In this diagram, you can see that a RDBMS is still used to capture input as it happens from various systems. The transactional history goes through an ETL (extract, transform, and load) process which denormalizes (or flattens) the fact data (orders) and extracts the supporting dimensional data (date/time, customers, products, etc). The ETL process then loads this new data into the data-warehouse where it becomes read-only. Some companies trickle data into their warehouse while others do it nightly or weekly because it requires a lock on the tables. The star schema does not require all of the overhead of indexes and commit/rollback functionality since it is read-only. This design is built for speed but still has its drawbacks. The drawbacks are it can’t be updated in real-time and it still has problems with large joins if there is a need to join multiple large fact tables. Star schemas will out perform a RDBMS when it comes to extremely large databases but it is expensive, maintenance heavy, and usually requires big iron.
The answer is distributed and in-memory computing on cheap nodes. The answer to extremely large databases is NoSQL.
NoSQL
NoSQL takes the best of the RDBMS and Star Schema approaches and takes it one step further. NoSQL uses a sparse multi dimensional data structure and groups relevant data closely together to reduce the i/o time required to return results. NoSQL also distributes the work across multiple locations (often deployed on a grid) so that many threads are working simultaneously and independently. Instead of indexes, NoSQL uses the concept of maps which holds multiple index values allowing for a single map to handle a dynamic set of queries based on many attributes. NoSQL also allows for versioning of records. By time-stamping changes, new records are added to the database without the overhead that updates and deletes have in a RDBMS. The new way of handling large databases may look like this in some enterprises.
NoSQL provides a suite of APIs so developers can add data to the data structures on the fly without having to pass through a batch ETL process. An ETL process can still be used if desired but is no longer necessary because of the commit/rollback capabilities of NoSQL
When to use the hammer
Going back to my opening statement, it is not really a question of whether NoSQL is better than RDBMS. The question is when should I use NoSQL? The answer (IMO) is when you have a requirement to query extremely large datasets with fast query speeds. That is when you bring out the hammer. Extremely large datasets are often event based transactions that occur in chronological order. Examples are weblogs, shopping transactions, manufacturing data from assembly line devices, scientific data collections, etc. These types of data accumulate in large numbers every second and can take a RDBMS with all of its overhead to its knees. But for OLTP processing, nothing beats the combination of data quality and performance of a well designed RDBMS.
Now I know some people will say that they know of very large database that are built with an RDBMS with Oracle or the like. You can do it, but you can also use a hammer to nail in a screw. To sum it up, both NoSQL and RDBMS are great tools for certain jobs. Just make sure you pull the right tool out of the toolbox when the job comes calling.
Private Clouds: Are they good for business or just cloud washing?
There is a huge debate going on about private clouds and whether they are really clouds or just a buzzword for modern day on-premise data-centers. An article called Are Private Clouds Hogwash? does a great job of capturing the debate that has been raging on for over a year now.
Before I give my opinion, let’s look at the definition of a private cloud as put forth by NIST (National Institute of Standards and Technology):
Private cloud. The cloud infrastructure is operated solely for an organization. It may be managed by the organization or a third party and may exist on premise or off premise.
See entire cloud computing definition here
Based on this definition, one could argue that an on-premise private cloud is real and exists. I won’t argue that. What I will argue is this:
Does an on-premise private cloud make sense for any businesses other than vendors selling the hardware and software that allow private clouds to be built?
In my opinion, with a few exceptions, the answer is a resounding NO! (Yes, I just shouted). Before I get into my reasoning let me explain why I chose to use the word on-premise before the word private cloud. That was to differentiate between a private cloud contained within the walls of an enterprise, and a virtual private cloud, which is a private cloud provided by a third party. Virtual private clouds (VPC) make sense to me because you are still outsourcing your infrastructure needs to a third party and greatly reducing your capex. Of course, the term VPC has many different definitions too (that’s for another post some other day).
Now for my reasoning. One of the biggest benefits to the business for cloud computing is the reduction of capital expenditures brought about by outsourcing hardware and data center costs to a third party provider. Choosing to build your own cloud on-site is like building your own refrigerator. Sure you can do it and you can have total control over it, but it is way more expensive, labor intensive, and will take you forever to get it done. Wouldn’t it be simpler to just buy one with the latest and greatest technologies and energy efficiencies and just plug it in? Now I am not saying that going to the public cloud is as easy as plugging in, but it is a heck of lot easier than building your own.
Here are some exceptions where I think an on-premise private cloud might make sense.
Government: In my mind, it makes sense for a government to create it’s own private cloud to provide low cost computing and security to its many agencies and divisions. With organizations as big as governments with huge budgets and armies of resources, it may make sense to build a private on-premise cloud to service a variety of locations and greatly reduce the costs by eliminating the redundancy of each agency having its own primary and secondary data centers.
Large multi-located conglomerates: Extremely large organizations made up of numerous locations and numerous types of businesses can be served from an on-premise private cloud. Just like with government organizations, a huge organization can actually reduce the number of data centers if they were to build a private cloud in a centralized data center and scale up and down to meet the needs of entire organization and its user base.
These types of private clouds can be argued to be a community cloud (here is the NIST definition):
Community cloud. The cloud infrastructure is shared by several organizations and supports a specific community that has shared concerns (e.g., mission, security requirements, policy, and compliance considerations). It may be managed by the organizations or a third party and may exist on premise or off premise.
What about the rest of us?
If your organization does not fit into one of those two categories, I question why you think you need a private cloud. My belief is that most who think they do are simply afraid of giving up control of their hardware, data, and network. Now some people may argue that a hybrid model, public and private cloud combination, is the answer. The concept of a hybrid cloud makes perfect sense where you keep your data in the private cloud and push as much processing as possible to the public cloud to get the benefits of cheap processing cycles. However, is that really a private cloud or should we just be solving that with a service-oriented architecture, specifically a data services layer, where we leverage the cloud but access certain data elements on-premise?
Summary
Private clouds are a vendor’s dream. As Dave Linthicum has been saying, many vendors are simply “cloud washing” their products and creating hype to make buyers think they need to build their own clouds. Companies that buy this snake oil may consume their precious capital and human resources on long expensive projects just so they can declare that they have built their own cloud. There is nothing wrong with not putting certain data or applications in the public cloud and simply leverage good architecture and virtualization internally to modernize the data center. Just don’t call it a private cloud.
