Got Legacy?

This is another entry in my ‘Legacy development paradigms are the greatest source of legacy problems’  series.

Architecture Astronauts are developers who create a complicated solution to a simple problem in order solve a bunch of other potential, and probably imaginary, future problems. Every day developers all over the world fight their inner Architecture Astronaut, and every day, thousands of them lose that fight.

Over the last couple of years I have seen a lot of legacy systems built in COBOL, NATURAL, RPG and PL/1.* Expecting overly designed hyper-abstract “frameworks”, I am always surprised at the incredible pragmatism and restraint shown by the developers of those systems. At worst you’ll find a dynamic call, where the name of the program being called is passed in as a variable, or a common error routine. 

Legacy mainframe code usually is incredibly verbose, not necessarily because the language requires you to write a lot, but because legacy developers did not reuse business logic or at the very least they try to avoid doing it. Or: 20 years ago code reuse was the stuff Architecture Astronauts came up with while mainstream mainframe developers avoided it.

Programmer A: So, I need to validate this postal code. Didn’t you already write code that does that?

Programmer B: Oh yeah, here: copy & paste the bugger!

Programmer A: Eh: can’t we move that in a common function?

Programmer B: ARCHITECTURE ASTRONAUT! ARCHITECTURE ASTRONAUT!

As usual I am exaggerating, but there is a core of truth there: reuse was not harder to implement than in modern languages, but developers just did not do it on a regular basis. I’d like to say that that was caused by unsophisticated specification and design phases and a less mature software engineering practice, but that is not true: most, not all, reuse in modernized development is done by developers, while implementing, after the designs were created.

Developers communicate business needs in code, and computer restrictions into conversations with users. But the whole practice of software development itself is also guided by a higher level of fads: assumptions about what can, should and may be done in code and how it should be implemented. The more I think about it, the more important these ‘paradigms’ seem to be compared to the individual choices of each developer or team. The quality and business alignment of your system is heavily influenced by the paradigm that was used to develop it.

Next in this series: ‘SOA: where Architecture Astronauts mate’.

- Mik Lernout

* Note that I always refer to legacy languages in uppercase but I do not line break after 80 characters. Inconsistent, I know.

Share on Facebook

What sort of challenges are expected during a large modernization project?

- We can easily think of the technical ones which are generally considered “fun” to tackle by technical teams.

- Difficulty in collecting and assembling the full picture of business processes and requirements from various sources/departments.

- Designing the modernized solution in a way that would serve the client’s business needs. This will be the topic of this blog:

The moment Business Analysts (in charge of analyzing business requirements) or System Analysts (in charge of analyzing technical requirements) get exposure to a legacy technology implementation of a business solution (user interface screens, batches, etc.), they can not help stop trying to think how to modernize it because that is the fun part of the game. In doing so, the BAs and SAs think of how to make the screens more efficient, how they might be able to replace an interface with a screen or a report and all kinds of exciting possibilities would come in mind.

It becomes as a shock or at least a disappointment (depending on how much effort was spent) when the client is presented with the exciting ideas of modernization and rejects them. This unfortunately happens more often than expected. So why does it happen?

- Either the new approach/workflow/design fails to address some unforeseen aspects of the business that the client with their experience can foresee. Once the client has expressed their concerns, a discussion can ensue to tweak the suggested solution to better serve the business requirements, or perhaps find another approach all together with the client’s help.

- Or the modernized approach employs modern technology paradigm that are still incomprehensible to the business user who is used to legacy screens and batches. This is where things get tricky for the following 2 reasons:

1. The client is unable to foresee any possible difficulties and warn against them. In this case, the client might accept the new design which could prove inadequates at a later time.

2. The client could reject the design because they don’t understand it fully and it could cause frustrations on both sides (BAs and SAs believing that it is the best solution, and the client being nervous about accepting a design that they do not understand).

The solution for the first problem is for the BAs and SAs to do their best possible efforts to ensure the safety of the design (mining related logic and understanding related business processes and data model). If this kind of effort is put in place, there would almost always be a way to tweak the design to bypass obstacles discovered at a later time with minimum to medium effort (risk management).

The solution to the second problem is for the BAs and SAs to try and put themselves in the client’s shoes and try to envision how it feels to be presented with a solution to your problem that you do not technically understand or unable to envision while expected to make a decision on. By thinking of it that way, the BAs and SAs would most of the time be able to find better ways to further explain their ideas more simply and clearly (screen mockups, presentations, use cases, meetings with white-board drawings).

Keep in mind that no matter how smart your solution is and how much you would love to fight to keep it, if the client simply rejects it, it would be of no value. When all fails to win the client’s buy-in, then we can still succeed by admitting that our brilliant idea will not work for the client and settle for a less brilliant solution that would be acceptable.

Sometimes we will have to draw our satisfaction from winning the client’s buy-in on less lustrous solutions that they accept than the more elegant ones that they won’t!

- Sinan Ali

Share on Facebook

See this funny set of customer reviews of Sun JCAPS on the Sun product site. I would really like to be in the room at the Sun Marketing HQ, especially when they chose ‘Yes, I would recommend this to a friend’ as a review metric for an SOA toolset.

- Mik Lernout

PS: I am just bitter because SOA toolsets are actually the only thing I can advise my friends on… I really need to get out more

Share on Facebook

It is always interesting to see an extreme opinion on legacy languages, such as The mainframe is not evil, but COBOL is… by James McGovern . Especially when they are dead wrong, or at the very least, misguided.

Most of the reasons listed on why COBOL really is evil (like ‘everything is a MOVE’, ‘Data layer is too tightly coupled’, etc.) can just as easily be applied to any modern programming platform. Ruby on Rails is all about coupling the data layer and has anyone ever figured out how many lines it takes in an EJB to write a Hello World? Without spoiling the surprise: a lot.

The problem with COBOL is not the code, nor the language and not even the mainframe. The problem is the people programming it. Nearly all COBOL programs out there follow programming paradigms which were very relevant about 20 years ago:

Computer! Take this crazy fixed width file filled with customers and produce another file with the same data and an added column to mark the customers that have overdue payments.

Now that we are up to our ears with computing power and network bandwidth, we much rather like to have realtime conversations:

Computer! Is ‘John Doe’ overdue? Hmm… And what about ‘Jane Doe’? Sweet.

The inherent batch processing embedded in legacy system does not exist by virtue of the programming platform, it is there because people coded it that way. And the reason they coded it that way is because it was the reigning paradigm. Which is why I should really blame an abstract entity such as the ‘batch processing paradigm’ for being evil, but that would be not as catchy as a title.

At MAKE, we modernize legacy system usually to the JEE platform. But we would love to modernize one to from batch-COBOL to realtime-COBOL one of these days. You know, just for the heck of it.

- Mik Lernout

PS: Some of my friends are COBOL programmers and they are really nice people, but this blogosphere-conversation-thing does not work without some controversy

Share on Facebook

According to the Merriam-Webster dictionary:

streamline – transitive verb

1: to design or construct with a streamline

2: to bring up to date: MODERNIZE

3a: to put in order: organize

3b: to make simpler or more efficient <a system that streamlines the process>

I’m not afraid to say that a very high percentage of people can quickly spot areas of improvement. Think about it for a minute and try to remember when was the last time you heard a comment like this: “I wonder why they decided not to put more cars to this train” or “Who was the ‘genius’ that came up with the idea of putting this stupid little voice in the elevator?”, or “Are you kidding me? A four year old could have done this job better”. I hear comments like these during my commute, at lineups, on the news, etc; it becomes clear that pretty much everybody has a natural ability to recognize potential for improvement or to fiercely criticize unnecessary made ‘improvements’.

In software development scenarios and, more specifically, in the legacy modernization arena, this becomes the daily bread and butter of teams that have to deal with modernizing old computer systems. I won’t speak for others (although I’ve heard them loud and clear… let me rephrase that: I hear them. Everyday.). I’ll speak just for myself: When I’m comfortably sitting and watching a client’s presentation of a system we’re about to embark on modernizing and how things are done in it, a frenzy of streamlining ideas come fast and furious to mind. It’s almost as if there was an entire new universe of opportunities for improvement. And this is only taking the notes at a preliminary session. Once the project begins and throughout the various phases of the engagement, more opportunities start to surface from all fronts: User interface, data model, batch and business processes, you name it. But we have to be extremely careful. Despite the natural human tendency of wanting to make these inherent to modernization decisions, we have to control ourselves and keep in mind that this particular screen, table, or process, if changed, even if it is for the better, may end up being for worse, much worse. Trust me on this. There could be way too many negative implications and, before shifting the gear to full streamlining mode, a thorough, constant analysis needs to be made by all team members and come to the very difficult conclusion of which ones can and will be implemented and which ones won’t. Not an easy task. Our clients usually complain and they have a valid argument: How come you can’t normalize this table/screen/process? Aren’t you supposed to be modernizing our entire system? We then have to explain to them the reasons that support the decision.

Sometimes these are logical decisions; very easy to back up. For example, one of our clients had at least a dozen search screens that did exactly the same thing for different departments. I guess during the twenty -something year stretch the system lived for, programmers simply used the good old copy and paste technique and tweaked here and there to accommodate their users’ needs at the time. When the new system was delivered, they ended up with only two new and improved search screens for the two largest departments with embedded logic and security that satisfied everybody’s needs. Result: lots of present and future happiness. The developers who will maintain two screens instead of two dozens will appreciate this.

Now, what happens when some of those items, in both the client’s and our own (as solution providers) wish list, don’t make the cut? I could answer this question in different diplomatic ways but in reality the answer is FRUSTRATION. Oh yes, big time. It may have been determined that moving forward would’ve had such an impact that it could be too costly in other areas. One of the trickiest areas is data migration. The rule of thumb in legacy modernization is that you shouldn’t go too far streamlining your data model; otherwise you’ll lose traceability to the present case scenario and that is too risky. So if we have a screen or a process that is screaming for streamlining but it would mean that drastic and complex data model changes are required… How do we deal with the old data that has to come across to its new home? It can’t be just tossed away, right?

But data migration is not the only difficult area. The standard architecture poses another challenge for streamlining. The requirement may be something that simply doesn’t exist in the menu. It can be added, but again, what’s the impact going to be like? Sometimes, we have to say no, unless there is room and willingness to accommodate the impact to the project’s budget and schedule. The good news is that this is a temporary no. By modernizing an entire system, it is a given that it will be strongly enabled for future changes, more streamlining, faster development time, and happier stakeholders.

- Alberto Rodriguez

Share on Facebook