Guest blog post by Chief Technologist, Matt Bonetti of TransVault, our valued partner in all things "migration."

I have now had the pleasure of working as Chief Technologist at TransVault for four months, so it seems like a good time to write a blog post....

Since I started here, I've been on a steep old learning curve - who'd have thought that email archives could be as complicated as credit default swaps? And, as always with a new job, there's been lots of new terminology to learn. Most of it is just a case of updating my own internal dictionary, but if there's been one concept that's taken me a while to wrap my brain around, it's been 'adaptive development'. This phrase gets thrown around a lot, but what exactly does it mean??

I've noticed that I'm not the only one to also struggle with this, and I suspect our partners and customers might do too....

What is it?

In every software development team that I've worked with in the past, it's been fairly easy to distinguish between requests to develop new features of a system (I've heard them called them enhancements. change requests, stories.....) or requests to fix things that aren't working properly in the system (bugs, defects, "features"....). Intuitively we all know what the difference between these two is: If something's meant to work,and it doesn't, it's a defect and if it's something new in the system, it's an enhancement.

Here at TransVault we pride ourselves on being able to fix email migration problems as they crop up, on the fly, so to speak. When you buy our Migrator product, we will work with you to modify it as required by your project. This is what we call adaptive development, but what does it mean in practice?

As has become very apparent over my first few months here, no two archive systems (even the same vendor & version) are exactly the same! Two installations that appear on the surface to be the same can actually be very different 'under the hood' - more so for some archive systems than others - and we sometimes have to make small tweaks to the Migrator codebase to enable it to process a particular 'feature' of an archive installation. This is fine - Migrator was designed to be flexible, and we're geared up as a firm to deal with these changes - and we have a defined process to identify, resolve, test and release any changes as cumulative updates to the Migrator product.

Why is it beneficial to a migration project?

The most important thing for any archive migration is to keep on ‘making progress’. This was a design philosophy for our product set. For example, if during a migration we suddenly see an increased error rate of items failing to process, this is NOT the time to throw up hands and stop the migration pending an investigation into why they are failing. No, the migration is making progress and perhaps for every item that fails, there are 1,000 that do not. Individual migration failures can be investigated at a later opportunity and, if required, the adaptive development work process can go on in parallel to the continuing migration and deliver a solution to the failing items in a cumulative update at a later date. This way, the interruptions to the customer project are minimised.

So, here are some examples of where adaptive development would come to the rescue:

1. Migrator supports archive product A, but we find mid-project that the customer is using a new version of product A that we haven't come across before. We will make any required changes to Migrator for it to support this new version.
2. Migrator supports archive product B, but a customer finds that archive contains items of an unusual MAPI type that Migrator cannot process (or other data related issues). We will make changes to Migrator for it to process this new data type.

What this means for our partners is that they can keep moving their projects forward when the unexpected happens - and, of course, these changes then get rolled back into the main Migrator codebase, making it a more feature rich product for everyone to use in future.

What it isn't

There are, of course, things that can't or shouldn't be included under adaptive development. If we tried to shoe-horn every request we get in, the likely result would be everyone else losing out on getting their adaptive developments done - so we try to be pragmatic!

Some examples:

1. If there is an feature of Migrator that is advertised as working for a specific version of an archive product, but isn't, we will treat this as a defect. Depending on the severity of the defect - basically, is it holding up the whole migration project? - we will either release a fix asap as a patch, or we will release a fix as part of the next cumulative update. In general, we prefer to release it in the cumulative update because we know it will then go through a full set testing cycle - we will always introduce an element of risk by making a patch. More on this in just a second....
2. If a project discovers in its PoC stage (or later!) a technology that we have never dealt with before - let's say, a new form of document storage - we will most likely treat this as an enhancement. We will try to help our partners out as much as we can, but if extensive investigation or coding is required, we may have to move it out of our adaptive development process or it will swallow up all available resource.
3. If a project is struggling due to environmental problems - let's say that the network is particularly slow, or the target archive throttles the rate of data that can be ingested - we would treat this as a support issue. There are many, many reasons why a migration may not run as fast as originally expected - our support teams are best placed to assist with investigating why and mitigating the effects. Experience has shown that the answer may well not require any code changes, but a tweak to the environment here or a config parameter there, in which case it falls outside our adaptive development process.

What is the difference between a cumulative update (CU) and a patch?

All this does beg an obvious question - why don't we release everything as a patch? Won't our clients then get everything more quickly?!

Although this is undeniably true, it does miss an important point that is easy to overlook. Receiving a patch is quicker but riskier than receiving a CU. The CU process is clearly defined, and always allows time for changes to be rigorously tested. That's not to say that a patch is released without any testing, but the priority is on getting the patch to a partner asap rather than testing it against all possible scenarios.

There is always a risk trade off involved here, and we work with our partners to help them understand this trade off and select the best option for their client. At the risk of being a broken record, no two migrations are the same!