Last time, I talked about Service Oriented Architecture (SOA) -- specifically, wrapping our systems (think databases) in services. (See: Not Your Daddy's SOA.) That way, when a table adds a new key or a column changes names, we don't have to re-test and rewrite everything. We can even add new services while maintaining backwards compatibility with the old ones, just by making sure the new service has a different signature or version number.
Externally, this allows us to partner with other companies. The cruise line Royal Carribean, for example, has an API to allow large travel agencies to search, reserve, and pay for cruises. This allows companies like VacationsToGo to build its own applications on top of Royal Carribean's, and tie directly to its customer relationship management software and other systems. But today I don't want to talk about the external API.
Today I want to talk about the back end.
The nasty back end
Yes, I know about the back end, that collection of systems that includes enterprise resource planning, human resources, CRM, finance, and the data warehouse; the thing that runs all the time, and that copies files which run in a batch to update something that is legacy and painful.
Look, legacy ain't bad. The word itself means a gift from another generation, usually finance, land, or a good family name. With software, it is usually a system that is handling millions (or billions) in transactions a year.
Day after day, legacy systems do the real work of the company: generating money. Our goofy web-based front ends are just pretty pictures to get the customer to press "Order," so the real system can take over and actually do the stock purchase, process the credit application, or fill the order. Legacy systems are good and make money.
The only thing is, they go down a lot.
Okay, not a lot. Maybe it's more like minutes at a time these days, perhaps a couple of hours if your company can get away with being offline on a Saturday between 1:00 a.m. and 3:00 a.m. Still, legacy systems need system maintenance, or run in batch, or upgrades.
And this is a problem with SOA, because your PlaceOrder call is going to hit a service that isn't up or isn't able to place the order. Now what?
Here's one fix for that: the message queue.
Handling Messages
Messaging queues can help organizations deal with legacy systems' unscheduled downtime.
Message queue or service bus?
One of the core ideas of the Enterprise Service Bus (ESB) was supposed to be that when an action happens, the bus would notify every sub-system subscribed to that event. So you'd have an order come through, and the ESB would tell the CRM system and the warehouse and the data warehouse about the new order, which was great... unless one of those systems was down.
Message queues fix that problem. The web server drops a message on the queue, and the backend service pulls messages off the queue when it needs them. Turn the legacy system off for two hours, turn it back on and wake up the service, and it can pull and process everything that has been queued up during its nap.
That means no race conditions. No adding to a table and maintaining a flag since last time the table was checked. System testing gets easier because you know where a sub-system ends. Did the message get dropped on the bus? And you know how to test the next level down; drop a message on the bus and see if it gets processed.
You don't have to pay a lot for messaging queues, either. RabbitMQ is free and open-source, though you may want to consider getting some support.
I'm not saying they are the solution to every problem, but as I see larger and larger IT infrastructures, the ability to create a seam, pull a system down, fix it, put it back, and have everything keep humming along is invaluable.
Somehow I thought that was worth writing about.
— Matt Heusser is principal consultant of Excelon Development.
@Mr. Roques wrote "What does it take for companies to decide to migrate their old legacy system into a new, state of the art one?" -- That's easy; they don't. The most common approach I have seen is wrapping up the endpoints where the systems interconnect, and having the communications happen via messaging. So instead of system A inserting into a database table and SQL B clearing it out, we use messages. Then, eventually, if you need to replace system A you can do it without impacting system B. (Sometimes this requires a little bit of refactoring, creating seams, etc). In my experience, you are right that failures and crisis is what forces the rewrite -- when you simple can not hire the COBOL programmers, or you realize that your RPG application is too resistant to change to allow your company to do what it needs to do when you need to do it. Royal Caribbean is in the middle of a process something like that right now - I think there management is the good guys, and they are well ahead of the crisis - http://www.cio.com/article/725114/How_Royal_Caribbean_Cruises_Manages_IT_on_a_Floating_City -
But really I bring this up because while it may sound absurd to build your messaging/queing application around an independent server, that is one of the designs that has long been part of the Windows WCF (Communications Foundation).
Microsoft has long has message queues for SQL server -- for the data purposes you describe. But also a IIS server can have MSMQ's added and integrate these into your .NET app. Now, having to run a IIS server (like running a sendmail server) on each and every workstation might sound cumbersome, but it's not unthinkable in a corporate environment (and in fact, the standard for most Linux distros is to install sendmail as both a local mail queue, and as a potential messaging/queuing utitlity!)
@jballo - Thank you for the comment. Actually, when it comes to messaging, I'm referring to tools like RabbitMQ. That's a programming API as simple as Drop_file(filename); for the sender and get_me_a_file(inout text) on the receiver. You can use whatever format you want. In fact, some of my clients that used to 'just' drop a file on a server and use a polling process replaced the drop/pickup with message queues using the exact same files! No muss, no fuss, no XML! :-)
Yes, message queing has been around since email in grand old products like Sendmail. In fact, sendmail is a Turing machine, did you know? Bonus points if you did.
But one thing Sendmail is also is a messaging mechanism with queues for both real time, and asynchonous, transfer of information...emails.
Now what SOA technologies like web services and WCF do is add messaging protocols for standardization like SOAP. SOAP is one of those good ideas, but it gets sticky when you start to deal with complex datatypes and objects. Initially you think it's all <lastname></lastname><firstname></firstname> stuff, but then it gets hairier!
JSON makes it less complicated and lighter but at the expense of less rigid syntax and hence greater capacity for error.
Part of the problem may be, is that many protocols were developed for human to machine communication. XML for example, has as a design point to be "human readable". For machine to machine, you may ask...why bother! For efficiency it would probably be something completely incomprehensible to a human but really fast, or even more complicated than a human could understand. Why not. That's why we pay big bucks for machines, right?
What does it take for companies to decide to migrate their old legacy system into a new, state of the art one?
I'm guessing they are trying to suck every little life is left of them but they should also consider the wasted time, not only due to failures but probably slower response times.
It's important to have the abilities to identify system outages and respond to them quickly, as well as take systems down proactively without threatening the entire enterprise.
Good thoughts.The Cost and efficiency challenges faced in maintaining legacy systems increase as the systems age. Read a whitepaper about this very topic "Y2k 12 Integrating next generation technology to transform your business" it offers good information on upgrading from legacy systems and integrating new technology.Readers will find it interesting @ bit.ly/S8p1W4
The ThinkerNet does not reflect the views of TechWeb. The ThinkerNet is an informal means of communication to members and visitors of the Internet Evolution site. Individual authors are chosen by Internet Evolution to blog. Neither Internet Evolution nor TechWeb assume responsibility for comments, claims, or opinions made by authors and ThinkerNet bloggers. They are no substitute for your own research and should not be relied upon for trading or any other purpose.
I've been writing about how the next evolution of the Internet might just be an advertising revolution, and how corporate IT can stay involved as the enablers and providers of the technologies that make this possible.
After the Internet ate the bookstore, music shop, and phone book, my previous article asked what the Internet would eat next
(see: Will the Internet Devour Your Job?).
In the late 1990s, banner ads were hit-or-miss. Pets.com would blast ads with no idea whether the viewer owned a pet. Drugstore.com threw up ads to people who didn’t yet trust the Internet.
With 24/7 processing and business continuation paramount, more organizations are considering having three datacenters, where primary and secondary datacenters are in their immediate region and a third is in a remote geography. Why? To avoid repercussions of a major disaster that could hit every IT resource in a specific region.
Big-data has become a big point of emphasis for many businesses. While the technology is available to deploy these applications, the needed personnel often is not. As a result, analytic engineers' salaries have blown past the six-figure mark, and hiring these experts has become a challenge for IT managers.
New tools like laptops, tablets, smartphone, and wireless connectivity let us work from San Diego to Katmandu, and anywhere in between. But time management remains a problem.
Showing results is the best way to win over social business doubters, according to Mary Maida, Medtronic lead information solutions manager. Internet Evolution's Mitch Wagner interviewed Maida at the E2 Innovate conference.
Software-defined networks, which deliver virtualization functions to enterprise networks, have the potential to dramatically change network design and significantly reduce costs and maintenance.
Companies need to take advantage of new technologies to simplify interfaces, improve capabilities, and enhance back-office processes. But they can't upgrade their Websites too often.
Wells Fargo uses social software to replace email chains and help its sales team collaborate more effectively to land deals, according to Kelli Carlson-Jagersma, VP Collaboration Strategy for Wells Fargo. Mitch Wagner spoke with Carlson-Jagersma at the E2Innovate conference
Cisco's rumored sale of Linksys suggests we may have problem with innovation and profit at the edge of our Internet, and that could be critical to the evolution of many Internet-delivered services.
New York's Metropolitan Transit Authority is conducting a pilot test of digital kiosks to guide subway users to where they want to go more efficiently and at lower cost.
The whole Amazon.reader debate is a double-stupid. It's stupid to think that there's any e-book buyer who doesn't know Amazon's URL, and it was stupider to let ICANN launch the whole free-form TLD initiative to start with.
While NFC's original goal was to enhance mobile commerce applications, it is finding its way into a number of other uses, which is creating both opportunity as well as challenges for IT departments.
Enterprises would like to move to cloud computing but are hesitant because they are concerned about providers’ ability to secure company data. Here are some tips that help to ensure that if breaches occur, the business is not left holding the bag.
Edmunds separates customers into segments based on the info it collects on its site and from partners, and uses that to push out custom content, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
The automotive website uses propensity modeling to target ads and customer registration forms, said Brian Baron, director of business analytics for Edmunds.com, at Predictive Analytics Innovation Summit.
Expert Integrated Systems: Changing the Experience & Economics of IT In this e-book, we take an in-depth look at these expert integrated systems -- what they are, how they work, and how they have the potential to help CIOs achieve dramatic savings while restoring IT's role as business innovator. READ THIS eBOOK
your weekly update of news, analysis, and
opinion from Internet Evolution - FREE! REGISTER HERE
Wanted! Site Moderators Internet Evolution is looking for a handful of readers to help moderate the message boards on our site – as well as engaging in high-IQ conversation with the industry mavens on our thinkerNet blogosphere. The job comes with various perks, bags of kudos, and GIANT bragging rights. Interested?
To save this item to your list of favorite Internet Evolution content so you can find it later in your Profile page, click the "Save It" button next to the item.
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
M2M: Rise of the Machines? Not Yet David Weldon In the 1970 science fiction thriller Colossus: The Forbin Project, two giant supercomputers from the United States and Soviet Union secretly join forces to take control of the collective nuclear might of the two countries. In the film, the two machines discover each other's existence, communicate back-and-forth, share their collective data, and cut their human creators out of the process. It is the ultimate example of machine-to-machine communications, or M2M. CLICK FOR MORE
Email This
