Good point — I have certainly seen this pattern before. I guess that the key thing to keep in mind is the ratio between extra functionality provided by the custom system and extra effort required to maintain it.

What’s most difficult about it is that these things are quite difficult to measure. Take the example from my original essay. The extra functionality provided is things like the failover, the admin screens and the protection against failure to close cursors. How much value do these things add? The admin screens — little to none (any good standard solution should have such tools); the others are harder to estimate.

In some cases the maintenance effort is difficult to estimate also. For instance, Joe encountered a bug when upgrading oracle which he thinks (but as far as I know never actually demonstrated) may be caused by our connection pooling. So he thinks of it as requiring a significant amount of effort. I have 4+ years of experience using it with only the one reported issue that I know of (and that one just suspected), so I think of the effort as low. We could dig to the bottom of the issue to find out, but the effort it would take to do so is more than it’s worth! So either way we’re just guessing.

What I have learned in the many many clients I have had is it’s not just your company. Lots of companies have similar problems with pervasive homegrown code and it is pretty much 90% of their problems and their cost in maintenance and downtime. A perfect example is my current client. They have decided to log all application to a central log like Syslog. Its high performance UDP based logging so their monitoring group can be watching gobs and gobs of data in one central location and eventing and alarming on that log with something like Tivoli. Great idea right? Sure, except they decided to home grow the API and its a total POS. They spent about 3 months working on it and in the end it was a steaming pile that we poked about 100 holes in on code review. I spent 15 minutes and had the exact functionality to Syslog done in both Log4J and Logback. Two open source API’s with Syslog capabilities already built in. They could have spent 25$ for my time and a great standard solution, instead they spent about 45,000$ over the two months.

But they are going ahead with their homegrown solution because they can’t justify throwing it out. So now it will be pervasive in the company and when it rears its ugly head with deficiencies it will already be EVERYWHERE in the company. It just makes me shake my head.

Hope all is well with you guys!

Bud

1) specific to an upgrade paths, when attempting to change drivers to newer versions (which should be a simple drop in replacement) issues have arisen that can not seem to be worked around and are deeply ingrained in the way the pooling ‘was written’. Some have attempted to write this off as an oracle driver issue, you and I both know that it is more likely the home grown code causing issues.
2) There has been alot of talk on the blog-o-sphere about things being intuitive and how much one has to keep in their head when doing work. How the connection items function in relation to my code should not be an issue I should have to worry about. I should be able to just assume it works as I would expect (it gives me a connection) and move on. Currently this is not true as the connection pool does alot of work that is protectionary – and one could argue ‘good’ but it does it in a way that ‘newer’ programmers are not familiar with and as such have to ‘bootstrap’ into their head when they are working on the project. I would much rather they work on Business functionality and not have to worry about ‘how’ a bit of plumbing works when that plumbing isn’t something they have ‘done’ in the past.

Yes it works – and yes it has features but it just might be a good time to revisit and look for something that is ‘newer’ that meets a good number of the needs and extend that rather then continue to use something that we wrote FORCING the decision of weather to support it or not.