I am really glad that the first job I had out of university was at a place that did high-integrity and safety critical imbedded systems. These guys treated software development as a real (if unique) branch of engineering rather than some kind of black art or mystical craft. The result was software that was practically big free (and demonstrably so) and of stunningly high quality.

After four years I moved on and now work in web development, but a lot of the lessons I learnt there have stayed with me. The most important lesson of all being that with the right approach it is entirely possible to build really high quality software.

There are branches of the software industry that have been doing things “right” for decades, treating development as an engineering discipline. It’s a shame that the web-dev world has taken this long to pick up on the idea, but it’s a very welcome improvement.

Hopefully, as ideas like TDD spread we should see a definite improvement in what is being built.

Congrats on surviving the J megacorp. I was change/incident manager at one of those myself. We could have used more developers with your sort of insight.

A couple of comments on the matter of schema and data handling:

“For one, it is likely to be way, way the heck out of bounds for regulated industries.”

…and in Japan and other jurisdictions, responsible parties can do prison time for leaks of personal information. If you must have production-grade data volume, get the information risk management people to vet your data masking procedure.

“What do you do when production_deploy_82 includes a substantive database change (such as schema modifications)?”

As you suggest, manual review of well documented changes are, I believe, the only way to understand the state of your data. Here I mean someone who understands the data reviews his peers’ work.

In my opinion, developers shouldn’t attempt to treat their schema in any way similar to their code. Data is not code. It isn’t developed to suit today’s needed functionality in a bubble, knowing that at the end of the lifecycle comes the sunset. It has to persist even as uses come and go. And data isn’t “patched”.

I’m building a release process right now that will incorporate a review of the delta between before and after schema definitions. We’re starting late: the schema is now five years old and already some portions can’t be explained. Applying a long chain of small changes through an automated process via some abstract framework is a recipe for building a black box. It should be possible for an experienced developer to ‘read’ the schema and understand its uses. Or if not, to find documentation that explains the dense bits.

Thanks again,

-Michael

This is some mechanism for watching every commit that happens to your source control system and running the automated test suite(s) right away.

Set up buildbot or cruisecontrol or one of the many listed at http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix