“When it comes to data access, you want to apply the same principles of reuse and flexibility that SOA offers,” explains John Goodson, VP & GM of DataDirect Technologies in this Q&A.
Goodson will be delivering the keynote address for the first Data Services World on June 24th in New York City, which is in the same location as SOA World 2008 East.
At Data Services World Goodson will show how the role that data plays in SOA is becoming increasingly important and how anyone responsible for data access are now being called upon to contribute to their organization’s SOA initiatives. You will learn about the technologies, best practices, and patterns that are shaping the way we use enterprise data.
Why data access in SOA environments? Why is that important?
Goodson: SOA has been around for a long time but it’s only in the past 2-3 years that it’s actually being implemented in production. What we’ve learned from this is that service-enabling application functionality leads to additional usage, which can result in scalability and performance issues. Services that are designed to be reusable aren’t necessarily designed to be scalable.
SOA guidelines are driven by SOA architects. The SOA experts do a great job at creating and managing useable services that constitute business or application logic, but they’re not experts at databases and the underlying mechanics of data access. So what we’re seeing and what we hear from customers is that services are being rolled out with inadequate data access, without the scalability and abstraction being built in upfront.
This also creates issues when it comes to deployment and maintenance because you have different groups creating different services using different methodologies and components. So the bottom line here is to plan ahead and consider using a data architect along with a SOA architect as you build your SOA strategy.
What is being done differently with data access in SOA?
Goodson: Unfortunately, the typical deployment is often the same. Before SOA, you had monolithic applications, for example accounting or CRM systems, which depended on technology like ODBC or JDBC to access data; they didn’t share any resources and weren’t built to work together.
What we’re seeing with SOA is that services are reusable by other applications but data access is still implemented in the same way as monolithic applications. So changing to a SOA architecture isn’t changing anything about your data access. You still have all these different data access components with different code all over the place.
Accessing data this way—directly inside every service—results in several bad side effects:
1. It forces your business logic experts to become data access experts
2. It results in extremely complicated deployment scenarios
3. It reduces scalability and performance
4. It produces inflexible services
Without flexibility at the data tier, the value you get from SOA—reuse and agility—quickly diminishes. When the data access is hard-wired into the service and the physical data source is altered, all the services that access that data now need to be updated. If you abstract the data access out into a data tier, the services will not be affected and the change can be managed (often automatically) in the data tier.
One of the reasons we adopted SOA was to share best-in-class services across a set of applications. And accessing data IS a best-in-class service. Designing a data services solution architected with superior data access technologies will ensure that the SOA implementation is scalable, easy to deploy, and reusable.
What trends are you seeing with companies that have to deal with multiple platforms, databases, and interfaces?
Goodson: What we’re seeing is that this heterogeneity is forcing some bad decisions in the SOA context. There are plenty of companies that have a lot of application types, languages, databases, and interfaces. And instinctively we’re taught to reduce that complexity. And in doing that we can reduce costs and increase agility.
And particularly with SOA, many enterprises are seeing their SOA strategy as a good time to standardize and consolidate. But what is actually happening is that people typically overvalue the results and undervalue the cost of doing it.
For example, mainframe migration is routinely a topic for discussion for SOA architects. That’s usually because the SOA architect understands distributed platforms and what isn’t understood is how mainframe assets can be used by SOA environments.
One of the main principles of SOA is reuse, so you don’t necessarily throw out what you have, including the mainframe. Legacy means “it works.” It’s a working system that runs your business. The key here is not to migrate a complex working system onto an alternative, but to implement a single simple interface to reuse those assets that have been working for 20-40 years.
What do most companies overlook in designing services?
Goodson: Two words: scalability and flexibility. When designing a database application, there is a process you go through, from designing the database schema, to deciding what architecture to use, to building, testing, and deploying individual services. And hopefully you load test the service so it’s not the users who are testing it. What happens next though is that people panic because the performance isn’t acceptable and they don’t know how to fix it.
There’s a lifecycle for solving scalability and performance problems. First, you look at your database and see if the bottleneck is there. Then the bottleneck moves to your application or service, which makes it harder to solve if the data access code and middleware is part of each service. Then, and this is what most people don’t look at, the bottleneck moves to your data access middleware.
Data access middleware can be tuned and this can make a very big difference in your application. A lot happens at the network layer with database access. There are a lot of performance tuning options that people don’t know about that affect how statements are processed at the network level by tuning those packets that are going across the wire. For example, just by changing the middleware your application can go from processing 50,000 rows per second to 500,000 rows per second. So the database driver you’re using makes a huge difference.
Flexibility in the data access design is often overlooked in SOA implementations, which is surprising since flexibility is a key goal in most SOA implementations. We have had customers that have come to us that were very thoughtful about how to organize their business logic into services, but they didn’t abstract the data access logic from those services. Once their physical data sources changed, they were forced to re-evaluate and change many of the services, vs. isolating those changes to a set of data services.
What about accessing data from several sources inside a service? What’s the best way to handle that?
Goodson: Good question because most SOA architectures begin with a simple service doing simple things. But as SOA matures what we’ve seen is that the complexities of your services increase such that one service may access a lot of different data sources. And since there is usually a combination of different data types, at some point data is usually marshaled to and from an XML format to a relational format like Oracle. The overhead involved to sequentially process this data inside services is extensive. Code must be written to do these transformations of data. This results in multiple invocations of data access services to fulfill a single request.
A better way to do this is to design the service using an XML-based integration approach, ideally with a solution that allows you to graphically construct the integration logic that can span different data source types. This allows a deployment pattern that can reduce the complexity involved in the sequential steps required to access different data stores.
With our XML-based integration products, we’ve seen a single XQuery replacing 1,000 lines of code because it encapsulates all the business logic and integration logic into a single step. And since you’re issuing a single XQuery, your performance is going to be much better.
Where do you see data access for SOA going in the future?
Goodson: As SOA continues to mature and become more pervasive, a lot of companies will end up with a ton of services that need to access enterprise data. Tightly coupled connections and data locked away in monolithic application silos is also an obstacle to building services quickly.
What we’re already seeing is that companies are starting to implement a tier that is used exclusively for creating data services. The consumers of those data services won’t have to know how to access the data or what format the data is in. That’s why people are starting to adopt a data services platform as part of their SOA infrastructure.
