Why systems and software development projects go bad

Inferior Developers

Lack of experience / lack of contact;
Programmers are not usually in a position that allows them to deal directly
with the clients / end users and have only programming, but not business experience.

First, a great amount of systems work is not done directly by the company that is hired to do the job in the first place, but rather by sub-contractors in twice or even thrice removed positions. This, of course, causes a severe communications breakdown that results in what the client really wants and needs and what they ultimately get being two entirely different things.

Second, the division of labor between system consulting and design on the one hand, and the actual programming and implementation on the other, does more harm than good.The result is that there are very few programmers who are coding a system while understanding the special necessities and details of the project as a whole.

System design is carried out by engineers who either can't or don't program

Quite often, the engineer in charge of system design is a person who 'used to' program, but is now an 'engineer' who leaves the actual coding up to another group of people. This type of engineer gets most of his current knowledge from the latest bleeding-edge articles that he's read, but he hasn't tried to put into practice what he's selling the client. In other words, the system design has been carried out by a person who lacks the skill and knowledge to foresee the very real problems and issues that arise during actual system development.

Developers who have years of experience, but have had no real opportunity for growth...

In this field, many engineers go from one project to the next without ever taking the time or having the opportunity to see the results of their work, and can't actually tell whether the result was a success as a whole or not. Their only standard of project completion is 'It's done', which doesn't involve a chance to see the system in use and how it works for the end user. This leads to a false sense of 'a job well done' and has many developers and designers believing that they've produced 'good work' just because a given project's over. In other words, there is an abundance of developers and designers who have put in many years who go around thinking that their skills and techniques are up to par just because there's more work to be done.

An Artisan or a Day Laborer?

Who should you trust to built the home that stores your company's priceless data?
A construction worker?
Or an Artisan who can both design and build a mansion according to your specific needs?

Developers who fail to grasp the importance of trying to understand the 'big picture'

A typical system is made up of various components that include sales management, financial management, production process control and other various general sales management issues. It goes without saying that it would be ideal for the system developer to be highly knowledgeable in all these facets of business. He must also be aware of and understand the client's business policy, the company's managerial policies, the skill level and competence of the system operators, and the system administrator's management policy. In other words, while it is imperative that the developer grasps the whole picture in order to produce a system that really works, very few even realize that they need to make that effort.

Client Issues: If the customer always knew best,
  they wouldn't need to hire pros...

The pitfalls in trying to give shape to something that has no form

Since system development is the process of creating something out of something that has no form, it is nearly impossible for most clients to visualize a system in the same terms that a system engineer does. Consequently, the practice is to use a mock-system called a prototype to help the client understand what the developers are conceptualizing. One of the major keys to a successful project is getting the client to understand how important this process is, and in getting their full cooperation.

What system software development and construction have in common.

Say a client orders a 10 story building built.

The client looks over the plans, but since architecture and construction aren't his line of work, he leaves it up to the designers and work begins on the building. The construction is coming along nicely and is up to the fifth floor when suddenly the client decides he'd like it to be a 15 story building instead.

The foundation and infrastructure laid out for a 10 story building won't hold a 15 story building, all the work to date has to be torn down, and construction started once more from the foundation.

This enormous loss of time, money and labor is exactly the same when it comes to designing systems. The ground work and foundation-laying make the building and the system, and the kinds of changes that can be made without starting again from zero once the foundation has been made and the infrastructure constructed are very limited.

A stubborn refusal to change practices and fixed ideas...

If the client takes either one or both of the following stances, it will be nearly impossible to create a better system for them:

1."But we've always done things this way"
2."I know my company and what it needs better than anyone and it does / doesn't / will / won't need..."

Just because a business owner does know their business, they don't usually know systems, and aren't always thinking of the many changes and kinds of growth that may or likely will occur when the initial system design is being made. This is what a system engineer does, and this is why it is imperative that the developer have the full cooperation of the client in order to make a good system.

Have you heard this story? (Or, thank you Bill for the perfect system development metaphor..)

The richest man on earth built a $50,000,000 house. As you may have heard, it has parking spaces for over 100 cars, an underground theater that is said to be the most technically advanced in the world, you can choose to have your favorite music follow you around the mansion, the floors and sidewalks are heated and there is said to be over 50 miles of hi-tech wiring laid. Over 300 technicians and construction experts as well as over 100 electricians were involved in its construction. When he first started building, he was a single man and had budgeted about 10 million dollars for his house, but along the way he found his princess who also started having a say in things and the construction ended up taking 7 years and going 40 million dollars over budget.

This project that cost over 50 million dollars and had over 400 people involved in the design had very few bedrooms and very little room for modification built in, which was a big problem when their third child was born with no room planned into the design for it.

Now, why didn't someone think to point this out sooner?

This is a prime example of what happens when a client takes over a project because they think they 'know best' and in fact end up with a far less perfect result than if they had listened to or taken the advice of the experts working on the project...