Got Legacy?

We at MAKE Technologies spend far too much time thinking and talking about mainframes, the crazy languages they run and legacy issues. These are the results of a mainframe-haiku-contest we held last week, in preparation for some awesome SWAG we’re planning for our Oracle OpenWorld booth at Intel’s Inside Innovation Showcase:

Sun rises in West
Birds fall from sky, hordes panic;
Hide from the mainframe.

Such an old system
It runs your business somehow
A constant worry

Flowers in spring,
Mainframe with empty meaning;
old is new again.

A buzzing mainframe:
sorts, moves, sorts, exports
Finds stillness.

JCL calls COBOL
looks up empty records in DB2
A lotus blooms.

Lonely language;
the last developer turns off the
green glow. Silence.

Batch can not sleep
MOVEs age into variable
sleeps again.

Process records at night
More every day, can’t keep up.
Batch window widens.

Without a lamp
the moonlight turns
my mainframe black

Garbage in:
code converts 4GL to OO;
garbage out.

Ancient, forgotten, cobweb
Unraveled, unscrambled, untangled, unknotted
Meet Tiger Mustang Dolphin.

Batches! JCLs!
How will they fix all this mess
When I have retired?

Mainframe costs soaring
Still the business asks for more.
We will get you off.

CIO demands
Service Orientation!
Show me the money.

Opened the door to
Modernization project.
Now the cat is gone.

Running on mainframe
Like beating a tired old horse.
We will get you off.

Still patching old code
Like a leaky old rooftop?
We will get you off.

Snow on silver hair
Old hands stiff under green screens
Y2K Redux

Hal, open the file
Hal, open the file, Hal
open the, please Hal

COBOL days of dust
Young minds never die
Old is new again

Batch Job override
Transaction code is long lost.
Where is that post-it?

require upgrade;
$time = now() || $never;
$modernize || die;

Sound of one hand clapping
The legacy system was
Enter MAKE and get the applause

Share on Facebook

Migration is not just for the birds

Data migration is like everything else – there is a right way and a wrong way to do it. Actually there is a right way for every scenario, and many, many wrong ways! This article will help you consider the alternatives.

What is data migration?

Just to make sure we’re comparing apples to apples, we had better define our terms. At times like this, I turn to Wikipedia:

Data migration is the process of transferring data between storage types, formats, or computer systems. Data migration is usually performed programmatically to achieve an automated migration, freeing up human resources from tedious tasks. It is required when organizations or individuals change computer systems or upgrade to new systems, or when systems merge (such as when the organizations that use them undergo a merger/takeover).

Can you imagine doing this by hand?? Nowhere in the world are there workers desperate enough to take on that task uncoerced. Hence the many automated tools on the market – computers will do anything.

What to look for

When you’re looking for a solution to your data migration problem, make sure that you consider a few key features that can make the difference between success and disaster.

1. Enables schema changes

At the end of the day, you want your mission-critical data organized in a rational way that makes it easy to get the data out in a form that serves your business. Trouble is, most legacy data stores are anything but rational. They have grown organically over the past 20 years like English Ivy next to a leaky nuclear reactor – not in an organized way, but with a certain craziness. They tend to exhibit “archeological layering”, so you can see where a certain field switched from use A to use B without actually changing names. The bottom line is that if you have a chance to fix your underlying data model, you should do it. It will make your life much easier in the future. Look for a toolset that allows a flexible and powerful assortment of transformations, including table splitting and merging, rationalization of foreign key relationships, extraction of value lists, etc.

2. Declarative

I hesitate to trot out the comp-sci jargon, but you want a data migration tool that stores your transforms as a set of rules, not as a script. Writing a bunch of scripts is kind of the old fashioned way to get the job done. And sometimes those scripts can get pretty huge and ugly. After all, it’s a programming task in itself, and not a pretty one. Do yourself a big favor and use a specialized tool that has some understanding of the fundamentals of the problem and stores your migration rules as rules, not scripting statements.

3. Iterative

Admit it – you will not get this right the first time. Trying to get it right the first time would be a huge waste of time, and probably lead you down the garden path to “analysis paralysis.” You need to get something done, see how it worked, learn from it, learn your lessons, make changes and do it again. Lather, rinse, repeat. You want a toolset that lets you run your data migration, inspect the results, and facilitate the iterative refinement of the script.

4. Auditable

In some environments this is not a nice-to-have, it’s the law. We can’t go losing our patient drug allergy records or our policy line items or (God forbid) our deposit and withdrawal records! And even in less regulated niches, it’s a good idea to be able to show where every record came from, and where every record went.

5. High performance

This is probably obvious, but when you’re cutting over from the old environment to the new, you will not have weeks and weeks go do it. You’ll be lucky to get 24 hours. And you may have hundreds of millions of records to read, cleanse, transform, re-insert, validate and audit in that all-too-brief period of time. Your tool had better be up to the task!

Integration: the killer feature while modernizing

It’s hard to overstate how useful a good data migration toolset is while you’re modernizing a large legacy system, especially if it’s integrated properly with your modernization toolset.

The first huge benefit is test data. While you’re developing, it makes a world of difference to be able to try out your new screens, batches and reports with actual (not just realistic, but actual) data. So the earlier it’s available in the life cycle, the better. Woe betide those who leave data migration to the end of the project, for they have missed a big opportunity.

Integration is a two-way street as well, because it’s extremely valuable to feed the inevitable screen and interfaces changes back into the modernized schema and the transformation rules, so that you can keep your test data happening with limited fuss.

Yes we built one

I’m sure you will not be shocked to discover that MAKE has just such a tool, designed from the ground-up with data migration in mind, integrated with the rest of our modernization suite. But don’t take my word for it – use your new found knowlege and perspective to evaluate all the data migration tools on the market before making any kind of decision.

- Tom Metzger

Share on Facebook

You got to know when to hold ‘em
Know when to fold ‘em
Know when to walk away
Know when to run
- Kenny Rogers

Today I am going to write something simply useful. And I might get in trouble for this, when the execs realize that I’m telling you how to do most of an SMR, a service that we charge for. Anyway here we go – let me set the scene.

At the Crossroads of Modernization

There comes a time in the life of every CIO and IT Manager when they realize that they have a bunch of legacy software applications, and they have a nagging suspicion that they need to do something about it. Often in this moment of truth, after a few rough nights dreaming of exploding mainframes and those horrible “bombshell” meeting when your outsource partner tells you it’s going to be much more expensive than they thought to add a trivial but time-critical enhancement, the CIO will hire an outside consultancy firm to conduct a big study that explains all the various options for modernization, the pros and cons of each, and to create a multi-year modernization roadmap in all its gory complexity.

So I think I can save you some big bucks here, because it’s not rocket science folks! Spend ten minutes running your legacy application portfolio through the following decision tree, and you will be most of the way to a roadmap.

When To Do Nothing

Ah, everyone’s favorite solution! You can do nothing if you still have time to react if something goes wrong. Keep in mind that doing something can be a multi-year process, so the decision to do nothing requires some considerable foresight. The litmus test for this one is simple: are you sleeping well?

When To Kill It

“Retire” is the euphemism here, but your system will not be heading off to the beach to read paperbacks, it will be gone. And good riddance, because if it fits into this category, it was just sucking up your money and giving nothing back. Freeloader.

When To Rehost It

Do this if the mainframe is the real problem, and you just need the same system, in the same language, running on a cheaper box. And even if the system itself is a nightmare, at least you’re not making it any worse. So you can solve your short-term mainframe problem, and deal with the more difficult software issue later.

When To Wrap It

Wrap or “SOA-Enable” your system if the biggest problem is integration. This can extend the life of your system by allowing other things to talk to it. Of course, you’re not getting rid of the system itself, so if the legacy system is a problem, it’s a problem you still have. And it’s also pretty clear that you have increased the complexity of the overall system, and introduced dependencies between the legacy system and the rest of your infrastructure. So it’s not a good way to address issues of complexity, maintenance cost, etc.

When To Port It

You port a system to a new language if the language is the issue. I think of porting like a meat-grinder – you take your old source code, run it through the meat-grinder, and you get the same system in a new language out the other end. Warts and all. You can’t make any material changes to the structure or design of the application, but at least you can get rid of your aging, expensive COBOL or PL/1 programmers and replace them with fresh college recruits.

Except they will all hate you afterwards. The guys you fired will hate you for firing them, and the new Java programmers will hate you for making them work on a Java system that looks like COBOL. Most of them would rather push bamboo under their fingernails. Pay them well, ’cause they won’t be sticking around because of the sexy tech.

When To Rearchitect It

This is what you do when the legacy system is in a sorry state, but still runs your business and has a lot of value buried deep inside. You want the baby, but no bathwater, thank you very much. This may be the most ambitious of the alternatives, but in many cases it’s the only one that gets you what you want out the other end – a nice looking system that doesn’t reflect the pathology of the legacy system it’s replacing.

With a proper rearchitecture, you can get a brand-new, relational data model, a modern tiered architecture, you can decompose the system into SOA-enabled components, you can revise the screens and the use cases to meet new business requirements, you can convert from batch-orientation to a real-time paradigm, you can choose a great new platform and a new architecture that can even include one or more off-the-shelf components, and you can do all that more cheaply and with far less risk if you leverage the legacy system rather than starting from scratch.

As you can see, for many organizations, rearchitecture is the Holy Grail of modernization. And it really hasn’t been possible until very recently, so you can forgive yourself for not knowing.

Conclusion

So now you are equipped to write a roadmap! We’ve written lots of them, and it’s mostly common sense. But let us know if you need any help.

- Tom Metzger

Share on Facebook

This morning, while riding in to work, it struck me that managers in large IT organizations exhibit something called “learned helplessness.” From Wikipedia:

Learned helplessness is a psychological condition in which a human being or an animal has learned to act or behave helpless in a particular situation, even when it has the power to change its unpleasant or even harmful circumstance.

Skinner Box: Today's IT Manager's office?

I’m no psyche major, but I know that they did research on this topic in the 50′s by subjecting dogs to electric shock in a lovely device called a Skinner Box. One group of dogs was given no control over the painful shocks – they appeared to start and end at random – and those dogs got pretty screwed up. After a while they didn’t even move when the shocks came, even if they had an easy way to escape them. They just whined.

Pretty horrible stuff, but what does this have to do with modernizing legacy software?

Well, IT managers have had a lot of shocks over the years, in the process of trying to rid themselves of their legacy software applications. Failure after failure after failure.

“Oh, the project is 200% over budget?” ZAAP!

“What do you mean SAP can’t be configured to handle our data model?” ZAAP!

“You must be joking – the vendor went out of business?” ZAAP!

“We can’t find the performance bottleneck in SQL Server, and the system has been down for two days.” ZAAP!

After a while, the poor bastards responsible for modernizing just throw up their hands and say “we’ve already tried that, and it doesn’t work.” They just lie down and whine, deeply suspicious of anyone who says they can help, and hoping they can make it to retirement age before the system breaks down entirely.

And this is one area where dogs actually do better than humans (again, from Wikipedia):

There are several aspects of human helplessness that have no counterpart among other animals. One of the most intriguing aspects is “vicarious learning (or modelling):” that people can learn to be helpless through observing another person encountering uncontrollable events (Bandura, 1986).

Isn’t that lovely? This is where a whole group of people decide that legacy applications can’t be modernized because they saw someone else fail at it! Yes, it’s the all-too-familiar “herd mentality” that we humans tend to exhibit. Probably helped our species survive back in the stone age, when things moved slowly. Ah, to be back in the stone age…

So even when things change, there’s a lot of inertia in people and in groups. What if a new approach or new technology made modernization of legacy application feasible all of a sudden?

Therein lies the opportunity for the bravest souls. The first ones to realize that there is a way will be the IT heroes of their age.

- Tom Metzger

Share on Facebook

In this article, we see the PR wing of a large mainframe vendor hard at work to turn back the clock and revive the mainframe age. The article has a defensive tone to it, as if to suggest that since everyone knows that mainframes are in decline for a half-dozen obvious and good reasons, they need to create some counter-spin to protect their revenues. My favorite part of the article was:

“Classic mainframe myths still exist today. Mainframes have traditionally been associated with high total cost of ownership; lack of advanced applications; inability to support real-time/low-latency processing; poor back-end data-integration support; a shortage of mainframe skills; and steep and inflexible development and maintenance curves,” he said. Some of these issues have been addressed, however.

Back to the Future?

Emphasis above is mine. How do you feel when someone tells you, “don’t worry about that – we’re dealing with it.” Are they trying a Jedi mind trick? Doesn’t it make you wonder which issues have been addressed? And what were the results of this addressing? Do mainframes now have low total cost of ownership, or is there no longer a shortage of mainframe skills?

Well clearly not the latter, since they go on to say:

Human resources, though, could prove a challenge. “There are simply not enough young, bright people wanting to learn mainframe skills over PHP, Java, Flash and other ‘hip’ Web 2.0 technologies,” said Sheina.

Yup, that’s what I keep hearing. But hey at least they do have “tie ups” with a whole bunch of educational institutions. That’s a bit ominous.  Does this mean they’re now doing to educational institutions what they do to clients?

If they’re having revenue increases on the mainframe side of the business, one can only assume they have become more adept at applying the thumbscrews to their existing customers. But this is good for the modernization industry, because the more pain their mainframe customers are feeling, the more they are going to want to get the hell off of it, and the biggest obstacle to that maneuver has always been those pesky COBOL and other legacy apps. The mythology around moving those applications is that it can’t be done.

Except now it can be done. The cost of moving those applications to Java or .NET in a useful way has plummeted in the last few years. And that’s not good for mainframe vendors who rely on “vendor lock in” to keep the racket money flowing.

- Tom Metzger

Share on Facebook