Reaping the benefits of Data Services
popping..I have written in the past about my company’s need to be able to implement our database layer in a variety of locations and in a variety of structures. Here are a few of the reasons why:
- Real time failover – we have our data replicated to multiple virtual data centers
- Multi-channel partners – we are an aggregator of digital incentives across multiple industries with multiple data structures
- Geographical requirements – certain customers and countries restrict moving data off of their premises.
- Regulatory – PCI, SOX, HIPAA and others require us to meet specific data requirements
- Security – We don’t allow anyone direct access to our databases. Internal structures are hidden from everyone in the ecosystem
The following presentation which I discussed at eBizQ’s webinar on the Convergence of Cloud Computing and SOA discusses M-Dot Network’s aggregation model and on slide 6 it shows an example of how we use a data services layer to meet the above mentioned requirements.
We recently had to rapidly build a web user interface to assist in the demonstration of our end to end solution from the creation of coupons down to the automatic redemption at the point-of-sale. One of my engineers has been focusing on building RESTful services using Django for the sake of ease of implementation and speed. What this enabled us to do is to outsource the development of the user interface to a team of experts in web development. The beauty of it is that they had no prior knowledge of our business and data structures and didn’t really need to know. We hid the complexity of our underlying data structures and granted them access to the data through the service calls. The services were very easy to understand and use as well as configure. During our testing we uncovered a few minor issues in the data that the web guys were displaying on the screen. With a simple change to some of the values they were passing in the URI, we were able to fix the bugs without ever touching SQL or the database itself.
So what did we learn from this exercise? By abstracting the data layer and creating configurable services as access points to the data, teams can quickly implement solutions in a controlled and standardized manner. So here are some thoughts about the benefits of this type of approach:
- Agile – teams can move quick due to the simplicity of the data access and the fact that they don’t need extensive knowledge of the underlying data. Also, the upper layers of the stack are loosely coupled from the data. If a certain customer requires a slightly different data structure or has location specific requirements, the implementation is hidden from upper layers (presentation, business processes, rules, etc.). Therefore, you make the changes in the data layer and/or in the data services and you are done.
- Cost effective – The further up the stack you go the cheaper the resources. Build the base with your experts and outsource the presentation layer. Makes for very cost effective user interfaces and makes change requests at the presentation layer much more feasible.
- Quality – Testing of CRUD functions are easier and should provide better results. Access to the data layer is controlled through these data services which tend to improve data quality due to single point of updates. Once those services are tested thoroughly, they only need to be regression tested if they remain unchanged for the next deployment.
There are many more benefits I could speak to but you get the point. I do want to point out that data services should be used when and where it makes sense. We have two major domains within our architecture. The first is a high speed, high volume transaction processing engine. We have stress tested the transaction processing engine up to a million concurrent transactions. Our response time is required to be in milliseconds. With these high speed performance requirements, we would not dream of using web services and data layers because the overhead would slow us down too much. For this we use old fashion tricks like caching, compression, and eliminating disk IO. But for the user facing systems like our B2B and B2C portals, RESTful services and data service layers make all of the sense in the world!
Did you enjoy this post? Why not leave a comment below and continue the conversation, or subscribe to my feed and get articles like this delivered automatically to your feed reader.
Comments
I have been involved in several similar efforts in a past life and have never seen this truly work. The “outsourcing” these efforts did were to other internal organizations which insisted that they needed to understand the data fully before performing any work. You have proven that with a properly architected solution and an effective data services layer that projects can be agile,the principles of SOA can be adhered to, and real business benefits achieved. *BRILLANT*
[...] This post was mentioned on Twitter by Mike Kavis, Mike Kavis. Mike Kavis said: Just posted "Reaping the benefits of Data Services | Kavis Technology Consulting" ( http://bit.ly/1pleCQ ) [...]