Rethinking NIH (“Not invented here”)

When was the last time you were annoyed by a piece of software? And, if you are a developer or working in that industry, did you think “I could do it better!”? This may seem a natural human reaction when things do not work the way they were planned, but there is also a lot of truth behind this. I will tell you why.

I recently learned that after switching to IMAP access, my web and mail provider (that I pay for) by default deletes all emails that are older than eight weeks. The “feature” is called “automatic cleanup” and deletes mails that are not moved into folders. I don’t use folders, I use tagging and search. So I was screwed. I wasn’t asked about that when I created or migrated the account(s). Now why would one, who pays for a usually free service would like his emails to be deleted? We are not talking about GB of volume but only a few MB or so. With some effort I could verify that all the important mails were still in my local Thunderbird cache. The provider probably saves a few ten-thousand dollars each year for backup costs with the result of costs and delays for the customers. Its interests are clearly not mine.

Developing components in-house and running your own services almost always makes at least one project stakeholder mentioning the NIH syndrome (see It will be easy to find examples where buying 3rd party systems, components and services would have done the job.

On the other hand, software running reliable and representing the very nature of an organization and its processes more often than not is responsible for company success. Being able to influence the direction of a component, its (engineering) quality, set of features and rate of change is directly related to costs for non-functional aspects of information technology. As an example, except for bug-fixes and improvements in quality and design, most small to mid-sized companies could very well live their current relational database management system in OLTP for years or even decades. Today and in future, databases are collections of passivated class instances in the best case or weird relational-procedural trash stores in the worst case. Nonetheless, having a stable component, where end-of-life dates are determined in-house reduces costs for integration, testing and upgrade fees, without even considering license and support fees.

But how could a small (or even a big) company develop their own DBMS, Java EE application server or ERP solution? There is good news: As reuse and marketing-relevant features are not required, one can concentrate on simplicity, quality and core functionality. Know what you’re doing and do it properly could be the catch phrase. Object-Oriented Software Construction, Domain-Driven Design and an Agile Development Process are the core ingredients that put the customer in charge.

As Eric Evans says, software development is not for unskilled labor (see this discussion), especially the domain model. An experienced, well-educated software-engineer is able to improve processes in a pragmatic, enduring way. Custom software that is constructed well and has its core components distilled, isolated and tested does never become legacy. Commerical software, even if it its price is only a few hundred dollars always involves the risk that after an initial benefit, indirect costs explode. We’ve seen it all with office software, application servers, database systems, persistence frameworks, operating systems and so on. As you might have experienced a large vendor does not necessarily mean stability and continuity. It usually means more costs as revenue goals rise. Everything needs to be integrated in the enterprise (and today even at home), so every approach lowering integration costs should be discussed, even if vendor BigAndFastRunOfTheMillComponentsCorporation has creative sales agents that will promise everything. The same conclusion applies for web services, SAAS and cloud computing.

A common experience of all software engineers is, that most projects are too tight on budget and/or time and lack a reliable and resilient set of requirements. It is interesting how the situation changes if all factors fit. Strange things happen – products get ready before the project end. Feature requests can be handled fast and easily. Going to production. Happy developers, happy business people, happy customers. “Doing the right thing right” still works.

My advice for every make or buy decision:

  • Do you exactly know what you need? Exactly means, do you have a specification or feature backlog that an in-house team could start development with? Only then you can estimate the integration efforts of a 3rd party component.
  • If you go for a buy, do you have an experienced engineer, who really understands what the product you are buying is doing and how it works? In the early days of J2EE nobody really understood what an application server really does and how. If you do not have access to the source code it is hard to verify how well-engineered and thus reliable and maintainable a product is. Maybe the current version is the last version that works.
  • Is the product you are buying light-weight enough to use it inside automated integration and acceptance tests? A DBMS or an application server should be embeddable and be able to run without complicated backend optimizations. Maybe that’s all you need in production, too (see Premature Optimization is the root of all evil). Is it easy to setup a fixture for testing the new version of your ERP system?
  • If you are buying, always have an exit plan.
  • Do not integrate with scripting languages. The third major enhancement of the once so innocent management script will be unmaintainable.
  • Never, ever place domain knowledge in systems in you do not control. In my experience going with a Java domain model is in the end the fastest, most flexible, most efficient and enduring way to success.
  • Have a student programmer write a clean adapter layer or generate it to integrate the domain layer with technical components. Adapters get thrown away all the time, domain models don’t. They are just reinvented if tangled with technical code.
  • Keep your developers well-educated. It’s a win-win situation.

Considering the constant stream of epic fails of integration projects, “not invented here” seems more than ever a sales phrase to make organizations buy certain products. Rolling your own, with a small set of engineering principles is often less expensive and more enduring than buying an over-sized, complicated component off the shelf.