Release It!

Michael T. Nygard’s book is one a rare one because it tells something about the time after a project was finished and is released for the first time into the world. The basic message is that you can not anticipate failures which certainly will occure if the system is used. Besides failures caused by programming failures, a lot of failures occure because the capicity of the system was never tested and the constraints of it indifferent situations were not correctly identified. He presents patterns and anti patterns for stability and capacity. Some are well known, but often not easy to handle (e.g. resource pooling), others are interesting as the circuit breaker which has the purpose to “break” the normal handling of operations if the failure rate becomes to high. After it has “opened”, all calls will fail immediately, but all ongoing transactions can be completed and the resources can recover, so no cascading failure happens. Another favorite is that we have always the wrong intuition about scaling, we always think linear, so as example I’ve clearly underestimated the change of database size from 100 GB to 1 TB and 2TB and so on.

So QA and load tests are not needless, far from it, but you should expect that the only accurate model of the world is the world itself. Building script able management interfaces, decouple systems as much as possible and expect that users will do unspeakable things will help. With the time and accurate data about the system’s health, it will become more predictable.

The book is a bit web site centric, but nearly all is true for all large system deployments. It covers a lot of points in very few pages but this can be a advantage because it is fast to read and will certainly help to smooth deployment and troubleshooting in operations.


No comments yet

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: