Burton Group Analyst Anne Thomas Manes wrote a great article yesterday called SOA is Dead, Long Live Services. If you just read headlines, you might think that she is denouncing SOA as a failed architecture. But if you read the entire article you will find that what Anne is really denouncing is the term SOA and the failed approaches that most organizations have used to try to implement it.
Although the word “SOA” is dead, the requirement for service-oriented architecture is stronger than ever.
I wrote a post on CIO.com several months ago called the Top 10 Reasons Why People Make SOA Fail. My message was that SOA is disruptive and transformational. It requires vision, leadership, organizational change management, and exceptionally talented architects and engineers. Dave Linthicum summed it up in his post when he stated:
So what went wrong?
First, there are not enough qualified architects to go around, and you’ll find that most of the core mistakes were made by people calling themselves “architects,” who lack the key skills for moving an enterprise towards SOA. They did not engage consultants or get the training they needed, and ran around in circles for a few years until somebody pulled their budgets.
Second, the big consulting firms drove many SOA projects into the ground by focusing more on tactics and billable hours than results and short- and long-term value.
Third, the vendors focused too much on selling and not enough on the solution. They put forth the notion that SOA is something they have to sell, not something you do.
Finally, the hype was just too much for those charged with SOA to resist. Projects selected the technology first, then the approach and architecture. That’s completely backwards.
As much as I agree with Anne’s article, I fear that the attention will now shift towards the next exciting technologies like mashups, cloud computing, Saas, and others and we will repeat our mistakes. Two years from now we will be blogging about the death of cloud computing and mashups if we don’t take a step back and review our lessons learned. When you bowl it down, what is the real problem here?
In my opinion, we suck at architecture! Why do we suck at architecture? Let me count the ways.
- We think process is a bad thing and it slows us down. Well bad processes are bad, but good processes can guide us through the proper steps that lead us to successful implementations. But there is a balance between process and agility.
- We are impatient. Many executives, especially in publicly traded companies don’t have the stomach to wait a year or two or more to reap the long term benefits of a service-oriented approach. They want results and they want them now. So we skip steps like identifying the organizational impacts, implementing governance, and acquiring the proper talent and wind up with a hodge podge of services that are built wrong and are unmanaged.
- We don’t understand what an architect really is. How many people do you know that have the title of architect? How many of these same people have a clue on how to architect an enterprise system. I laugh when I see job openings for a .Net architect. What does that mean? Is this person capable of architecting a .Net framework to be used by the .Net community? No, this person knows .Net real well. That’s not an architect, that’s a “senior, sr. developer”.
- We don’t understand what architecture really is. I am a member of many LinkedIn groups that focus on architecture. There is so much discussion on what the value of EA is and hundreds of different answers. If we can’t agree on the value of EA amongst ourselves, how in the world is the CEO or CFO going to support an EA initiative? There are so many good frameworks out there (Zachman, TOGAF, E2AF, etc.) yet it seems that we are trying to reinvent the wheel. Pick one, tweak it to your organization’s needs, and follow it. It is not rocket science.
- We lose sight of the value and argue semantics. I am also in many yahoo groups (SOA, Cloud computing, etc.) and the discussions are so discouraging that I don’t even participate. There are so many religious wars like Rest vs. Web Services, this vendor vs that vendor, and myths being discussed as facts. What we really need is to discuss what has worked an what has not. What are the best practices, the gotchas, the case studies, etc. We should be helping each other succeed instead of fighting over semantics. No wonder the business often takes matters into their own hands. How did we become a Dilbert cartoon anyways?
- We lack leadership skills and emotional intelligence. When was the last time you saw IT leadership who excelled in Technology, Business, and People Skills? You are lucky if they are good at two of these but to deliver transformational technology based initiatives they need to be skilled in all three (see The Missing Link in IT).
But these are not SOA specific issues. IT has historically struggled with enterprise wide initiatives because they require more than just technology to implement. Look at IT’s track record delivering ERP, CRM, new software development lifecycles (SDLC), agile, six sigma initiatives, etc. All of these require an organization, not just IT, to change behavior, standardize, and work in harmony to achieve a common set of goals. To make matters harder, we still need to keep the business running in the mean time.
The Cop Out
So now that we have declared SOA dead, we can forge ahead and start creating all kinds of ungoverned services, mashups, and cloud applications. But guess what, like SOA we will wind up spending more money on solutions that don’t solve our problems unless we invest in architecture. It reminds me of day trading and the the housing bubble. People fall for the get rich quick scams and think they are winning the game until it all comes crumbling down. Like a good long term mutual fund, SOA is a long term investment that lays the foundation for long term success. The day traders and house buyers where living high on the hog early on but when the market corrected itself many where left broke. The day trading bars disappeared, the real estate agents closed shop, and the hype died. That’s what I see in IT. SOA was the stock that went up 150 points in one day or the condo that you could flip for $50K in 3 months. So we bought ESBs, hired consultants, and wrote a ton of web services. There were some quick wins. But then we needed to make changes to these services and things broke. Worse yet, because we didn’t take the time to architect the data layer, a business request caused us to make changes to 35 services instead of making a configuration change in the data services layer. Even worse, we lost track of which version of the service we changed and who it impacted. We made the change and broke other applications. You get the picture. We day traded instead of architected. Guess what? We failed. But we blamed SOA, the vendors, the analysts. It couldn’t have been our fault could it?
So my question to the industry is when are we going to admit that we will continue to fail until we invest in enterprise architecture? EA frameworks do not only address technology. EA starts with the business and ultimately defines which people, processes, and technologies that are required to help the business meet its goals. Included in any good EA framework is the transition plan (I call it organization change management plan) required to successfully implement the appropriate solutions.
So the choice is ours. We can invest in EA or we can have the analysts declare our next strategy dead two years from now. I choose EA. What do you choose?
Excellent comments!
This just makes me believe that this is a great opportunity for companies specialized in transformation processes. Companies that do more than just IT.
Cheers,
Hugo Vega
[...] чего, почитайте и комменты тоже. Dan Foody ее поддержал. А MikeKavis так вапще всем напомнил про [...]
What annoys me is that you’re now listing the reasons why SOA is failing, when these exact same reasons were pointed out years ago. It didn’t stop the SOA Architect Astronauts from hogging into the budgets, screwing the business seven ways from Sunday, then bailing out to the next brand new fad.
The next problem is now going to be all those incompetent “not an architect” people now leveraging up on the new glossy brochures, burbling out a load of rubbish about the next brand new thing (PaaS?), and screwing the business seven ways from Sunday yet again.
Your mea culpa posting here is both a part of the original problem, and a tissue thin layer of excuse over your previous participation in being part of the original problem such that we’re even talking about “SOA is dead”. SOA never even bloody existed in the first place.
Even talking about this topic is part of the problem, because it makes it sound like that there are options, considerations, and various balances to be weighed before making a selection. When the options are vapourware (cloudware!), they are not valid in the first place and should have been completely and utterly ignored so as not to waste resources.
Until computer science and IT is overhauled with a new approach, we are still stuck with the problems and engineering tradeoffs of the previous approach, which is still von Neumann and mathematical mappings onto physical behaviour. Therefore all the solutions from the 1970′s are still valid for the problems we have now. Too bad, so sad. That’s reality.
It’s still the same old “mainframe vs PC” battle, with different terminology and new bunches of money being thrown at new monkeys with new sales pitches. People with no knowledge or understanding of computing history will always caper up and down, but people who have been around will yawn at the same old shit with the new blinkenlitchens.
Unfortunately, people with budgets and spending authority aren’t always the ones who have been around for a while, and that’s your real market there. Gullible idiots in the wrong area. Scam them while you can, and screw that business seven ways from Sunday!
@Steve,
Are you saying that there has been no innovation since the 70′s and we should all just revert back to the mainframe? Please clarify. I am trying to understand how you think I am part of the problem. I am promoting EA here and I don’t see how that is bad!
Yes, Mike. There has been no real innovation since the 1970′s. For every idea that is “new” now, I can find old videos and papers from the 1960′s or 1970′s that lay out the foundations. As an outstanding example, I can point to Douglas Englebart’s “mother of all demos”, or I can point to Alan Kay and Ivan Sutherland’s work with Smalltalk. Even John McCarthy’s work with LISP is making a comeback.
EA doesn’t really have any merits in itself, as it is just a term that covers the area that is the problem. In that respect, it is better than SOA or SaaS because those names don’t even describe the problem. However just using the term “EA” doesn’t solve any of the “we suck at architecture” issues or the shortage of people who can actually make stuff work.
To really direct attention at the problem, we can’t pretend to have a solution and then sell people on the new terminology. We need to admit that we in fact don’t actually have a solution, and then the problem is more clearly defined!
To clarify the specific point I’m making… pretend I’m a business that bought into SOA. You now want me to buy EA. What exactly are you proposing, other than changing the letters of the acronym?
@Steve,
Not sure I agree with that. That is like saying there is no innovation in television. Yes we still sit in front of a TV and watch programs today like we did 40 years ago. But the delivery format and the devices themselves are much more advanced and deliver content better than in the past (color, HD, interactive, etc.). The same applies to IT. I agree that in the 60′s on the mainframe people were using IMS to solve problems using patterns very similar to what we are trying to accomplish today with SOA. I worked on IMS DC which delivered transactions from one business process to the next much like we are trying to do today. But it took a lot more work to do this because everything had to be done by hand. The underlying infrastructure did not support real time cross company communications like we can today over the internet and with all of the standards. So I partially agree that the core problem solving concepts are not all that new, but the tools, standards, and infrastructure enhancements in the last 30 years have contributed to huge improvements in the business.
Steve,
I am not asking the business to buy EA. I am recommending that IT follows a framework of best practices with the intent of uncovering the who, what, when, where, how, and why questions from the perspective business, information, technology, and infrastructure. By answering these questions one can identify the core entities, services, processes, etc. to allow IT to align the proper technologies and solutions with the businesses goals and objectives. These frameworks are the direct result of decades of trial and error by some of the same great minds that have pioneered IT from its early beginnings. EA is not the solution, but following a framework can go a long way to helping IT deliver business value by identifying business problems first and solutions second. Unfortunately in today’s world, we tend to start with a solution (SOA) and then look for a problem to solve with it.
I agree that the infrastructure and tools have improved to the point where it is easier to implement a solution than it used to be. The difficulty is that the solution may not precisely match the problem, and therefore add its own problems.
I don’t really know which tools and platforms that you’re associating with EA, but Dave Linthicum did make a good point – which is that there are not enough qualified architects to go around. You have an advantage over the SOA crowd in that your EA label is a better description of the problem.
Where I lose faith is when you bring in the tools and platforms and frameworks, as that’s just another set of solutions for the same old problems with a different label on it. EA instead of SOA. To misquote Animal Farm, “meet the new solution, same as the old solution”.
I think it’s better to directly describe the problem by simply saying that there is a shortage of people who can make stuff work.
Fantastic tools with great amounts of snazzy output cannot create more people who can make stuff work, it can only create the impression that stuff is working. This is sufficient to create the “silver bullet” market, as there is a time lag from implementation to discovery where profit can be made.
Note that I’m on the other side of the fence from you. As a consultant, you want to know the most about a set of solutions that you can adapt to suit a range of problems, hopefully leading to a win-win situation. I have to consider when that doesn’t happen, and what consequences there are.
Even if you implemented EA and it worked, I still have the problem of solving the time lag until I know that it works, and whether the “EA” that you implemented is really “Mike can make stuff work” or a valid way of transforming numbers of people into “can make stuff work”.
If it is going to take time to find out if things are working, and if it turns out to be Magic Mike, then I may have solved a business problem at the risk of creating two larger ones. How do I reverse out of business processes being critically dependent on Magic Mikeism?
The ideal situation is to have a group of people who can make stuff work, and identifying those people is very difficult and the real problem to be solved. With the right people, real work can be done. Observe Lockheed Martin’s Skunkworks and the SR-71 Blackbird’s development.
@Steve,
I agree with your last statement. Like Dave, I also pointed out that people are the problem. I am a CTO for a startup and I have a small team that makes up a group of people who can solve the real problems. Most organizations are larger and have a mix of a few excellent people combined with many average and even poor people. So I get your point. However, even with a small group of excellent people, why should we reinvent the wheel? Why not follow a proven methodology (I use the extended enterprise architecture framework – E2AF) to allow the team to get gather all of the proper information we need to solve the business problem? But to your point, the EA is only as good as the people that use it.
Mike, I enjoyed reading your post and I also believe that the enterprise architecture discipline can help. But, EA is acronym-challenged too. Any time we get fixated on an acronym (e.g. SOA, WOA, EA), or any shiny new IT focused thing, we remove our focus from the important stuff – the business outcome and the transformation we are trying to enable.
I also commented in our Executive Advisory Program blog(http://eapblog.burtongroup.com/executive_advisory_progra/2009/01/soa-kill-the-acronym-and-focus-on-business.html).
Mike Rollings
Senior Analyst, Executive Advisory Program
Burton Group
@Mike Rollings,
Good point. From now on I should simply stress the need for applying architecture best practices and leave the acronym out. Thanks for the feedback.
[...] there are many, many identified technical reasons why most SOA projects don’t succeed, Mike Kavis in his post has articulated one perspective nicely at a high level (just summarizing his list [...]
[...] there are many, many identified technical reasons why most SOA projects don’t succeed, Mike Kavis in his post has articulated one perspective nicely at a high level (just summarizing his list [...]
[...] there are many, many identified technical reasons why most SOA projects don’t succeed, Mike Kavis in his post has articulated one perspective nicely at a high level (just summarizing his list [...]
[...] Mike Tavis, jumping into the “SOA-is-Dead” [...]
I’ve been in IT about 40 years. SOA and EA and SaaS and cloud computing all seem to be, in one way or another, just the latest attempt at centralization.
But the rise of decentralization changed all that. It allowed silos to form. You can’t get silo owners to give up their silos. You will have to pry the silos from thier cold dead hands.
I’ve seen it dozens of times. I’ve seen “new” systems that start back on the trail to centralization be retired or abandoned, while the silos survive.
Look at how long the federal government has been trying to do EA. They even call it FEA. And it seems they now believe it’s really SOA. But there are NO successes, just feeble attempts that did accomplish something but have nowhere to go.
I’ve talked to executives who agree that EA and all the other stuff is just a solution (maybe) in search of a problem. Plenty of them are content with working silos, and don’t even believe that SOA or anything else will provide any value they can’t get anyway. Many of them think (and rightly so) that the cost will far outweigh the (often vague) benefits.
IMO the organizational problems are the kiss of death.
[...] Did SOA die or do we just suck at architecture? (kavistechnology.com) – January 06, 2009When you bowl it down, what is the real problem here? In my opinion, we suck at architecture! Why do we suck at architecture? Let me count the ways. We think process is a bad thing and it slows us dow… [...]
@Ron
I agree with you. The problem lies with the people and organizational resistance to change. The technology, tools, frameworks, and best practices are there but the underlying issue is people and change.
[...] summary, making the best of the cloud requires that we take an architectural view, something that we’ve proven remarkably bad at over and over. Simply deploying an application unchanged to the cloud is unlikely to deliver much [...]
[...] deliver incremental milestones in your roadmap towards the future state. Without EA, we might just suck at architecture! VN:F [1.0.9_379]please wait…Rating: 0.0/5 (0 votes cast) enterprise [...]
[...] have all talked about the Death of SOA, the imminent failure of Cloud Computing, and countless other IT failures on blogs dedicated to [...]
[...] architected differently than on-premise solutions. Let’s learn our lesson from all of the SOA failures and focus on architecture. You don’t buy security, compliance, failover, performance, [...]