Do Not Underestimate a Document Migration

When new IT systems are implemented, the focus is on making sure they can do what we need in a good way. Only scarce interest is devoted to the migration that typically is waiting around the corner. This can backfire and cast a negative light on an otherwise well-implemented new system.

Focus is Elsewhere

When a new IT system goes live, the focus is naturally on developing the new system. When that is coming to an end, documents/data from old systems then need to be migrated in. This turns out to be time-consuming, data is not as good as expected and all sorts of annoying problems arise just as users are about to be let in.

To get the system up and running on time, one resorts to a Plan B, where one quickly fences in the old data, hides it in a corner and hopes no one misses it. Or worse, throw the documents in with a coal shovel so no one can find anything afterwards. It’s cartoonish, but only mildly cartoonish. There are many who have made a living out of document migration over the years.

In practical terms, migration requires that the new system is ready to receive data. Therefore, naturally, very significant parts of the migration work are late in the project, and that’s just not when you need surprises and challenges.

The remedy for surprises in document migration is preparation.

Why is Migration Dificult?

First you have to understand what you are up against:

  • Executing the migration takes time – perhaps longer than you can afford to be without a system, and then you have to embark on a more complicated incremental migration (Read more in Incremental Migration).
  • The data structure in the old system and the new system often do not match. Data must therefore be mapped so that it migrates meaningfully into the new system. While many other challenges in migrations are very IT heavy, this is something that requires deep business insight and a huge overview. There are also, for example, relationships between documents and between data on the document – possibly a case concept that needs to be preserved during a migration. (Read more in Why a Document Migration is Complex.)
  • Document history can be essential. It may be in a single log in one system, while the other system requires history to be embedded per object. (Read more in Migrate the Entire Document.)
  • And then there is the quality of the data one has in the old system. They almost always surprise in the annoying way.
  • The whole process of migration should be done under control – a special kind of control if you work in a tightly regulated industry like pharmaceuticals. (Read more about the background for why control is so important in Why a Document Migration is Critical).

The above is just a quick scratch in the surface, but please refer to other articles which go into more depth on the individual points. But hopefully what remains is a picture that migration is more than moving documents. Before starting to move, there are many considerations to be made.

The Right People

Another part of the preparation is to involve the right people.

A good migration involves a lot of business knowledge. Although there is a lot and heavy IT in a migration, it is not advisable to consider a migration as an IT exercise. It is more of a business exercise, where value is created by systematically addressing data, cleansing and enriching documents and data in the process.

It may reflect negatively on the experience of the new system if data is difficult to find or appears cluttered, but you can opt out of spending money and time to create value by improving data.

What you can’t opt out of is considering what metadata is significant to the document and making some fundamental considerations about the data. Because if you don’t make these considerations, it is a pure coincidence whether the documents, after migration, have retained their integrity or not.

The Right Tools

A third part of the preparations is to choose the tool with which to perform the migration. Here we have to say, from experience, yes, IT can “code anything” – how often we hear that phrase – but it is typically both cheaper and better to get your hands on a strong migration tool. But remember, it has to be made to migrate documents – both document files and metadata.

With good tools, for example, you will be able to phase your migration and maintain full control and overview. You will be able to monitor how the migration is going and find traceable documentation of the progress.

You will be able to set up mappings between the metadata structure in the old and new systems. Enrichment can be added to the process according to the analyses. Once this is set up, one can simulate a migration, to “practice” the migration and fine tune mappings and rules. This can be done while another part of the project is working on finalising the new system … not afterwards.

In general, you will get in control of the process at a completely different level than if “IT just codes something” and get started much, much earlier. And that’s what we want to achieve; control and the ability to get started quickly.

Tasks in a Migration

Finally, as part of the preparation, you need to be clear about what tasks you can, should and will do during the migration. We have compiled below an inspirational list of tasks that we consider appropriate to include in the migration process.

It appears that there is a lot of preparatory and “outside” work to be done in addition to the technical migration. This is very much where the value is created and the quality of the migration is ensured, so before they are crossed off the list and declared irrelevant, think again. It could be there that your business case is hidden.

Data Audit

  • What do we have? How much? Which quality? When was it created? When was it last used?
  • Assessment of possible mapping of metadata against the target system

Clean-Up

  • Identification of redundant, invalid and trivial data
  • Deduplication
  • Controlled deletion (or exclusion from migration)

Metadata Enrichment

  • Detailed mapping of metadata between source- and target system
  • Datatypes
  • Business context, key information

Data Security

  • Access control
  • Classification e.g. personal data
  • Retention schemes

Governance

  • Policies
  • Processes
  • Ownership

Technical Migration

  • Technical transfer of data
  • Decommissioning of legacy system

Quality Assurance

  • Risk analysis
  • QA programme
  • Test plan
  • Logging

6 Conclusion

Document migration is very rarely just an IT exercise that IT just fixes as the final part of implementing a new system.
The vendor of the new system may offer to import documents and call it migration, but as is hopefully clear by now, the work before we get to the point of being able to import anything is quite substantial. That work is unlikely to fall to the supplier of the new system. They are experts in the new system and usually the best at importing documents into it. But they are not necessarily experts in your old system and your business. There they will be very dependent on you and your knowledge and your preparation.

The remedy for surprises in document migration is preparation.

Our experience and best advice for a controlled migration, where the documents are well afterwards, is gathered in our Guide to Document Migration.