ISC Chronicle White Papers Contact Us

Successful Data Management Projects

Return to library


Data Management has been the subject of increased focus in recent years. Many firms have realised that without a coherent data management strategy, success in a wide range of projects cannot be guaranteed.

It is unusual these days to come across an organisation that has not spent time and energy implementing some form of data management solution. These range from the simple single data vendor storage and onward transmission type service to the more sophisticated and complex multi vendor, “golden copy” construction offering. Some of the more forward thinking organisations are even realising that reference data does not need to be constrained to security or instrument data, but that the same disciplines can be adopted when looking at other classes of reference data (for example client and Even in today’s turbulent economic times and despite downward pressure on budgets, a recent survey indicated that a majority of firms are increasing their investment in data management rather than cutting back.

Despite this increased focus and attention, data is still cited as one of the leading causes of project failure in our industry. This leads us to conclude that despite its trivialisation in certain circles, the implementation of a robust data management solution can be a complex undertaking. This article is aimed at data management professionals and seeks to illustrate a series of “do’s and don’ts” which may alleviate some of the problems typically found in such departments. These activities are practical in nature and are based on things which I have seen working in some data management functions or contrarily not implemented or only partially implemented and therefore the cause of problems in others.

Understand what your clients want

This would seem to be a remarkably simple premise, but equally remarkably, it is one that is commonly overlooked. The effort required in data coverage and data cleansing activities can be substantial. It therefore seems sensible to prioritise these efforts by addressing those areas or data attributes considered crucial by the recipients of the data. Talking to clients about their requirements provides the additional benefit of formulating an understanding of data and how it is used amongst the staff working in the data management area. This is a crucial step in building the client/service provider bond necessary for high quality service but is often missing in practice.

Implement SLAs with your clients

In common with many service delivery functions, the data management team is never mentioned when things are going well, but receives substantial publicity when something goes wrong. Implementing Service Level Agreements (SLAs) provides a simple mechanism to assess the performance of the service free of the emotion that always seems to be prevalent when things are not going well, thereby ensuring that the perception of the service provided is based on fact. In instances where SLA targets are being missed, this also provides data managers with an early opportunity to analyse causes and implement remedial strategies. Do not however underestimate the effort required to build the tools you will need to measure performance against SLA as automating this can involve substantial complexity.

Research your market

As mentioned in the point above, the provision of a high-quality service is partly about perception. It is important that people in the organisation regard you as the most knowledgeable people when it comes to data. Furthermore, in order to provide the best service to your clients, it is necessary to keep abreast of developments in the data space. Market data vendors and services change frequently. Just because vendor A provided you with the best price/ coverage combination when you implemented their solution doesn’t mean that this is still the case. A sensible review programme should be put in place to confirm that the solutions taken still offer the best in terms of quality and value.

Question data quality

Just because data is being provided by a reputable vendor with a good name in the market doesn’t mean that the data is accurate or complete. Even the most renowned data vendors have known quality issues in certain areas.

Be careful what you ask for

Some of the leading data vendor offerings have seen an explosion in content over the years. In some cases, this has been achieved by cobbling together multiple disparate sets of data. This has resulted in many of these becoming very complex. It is wrong to assume that data provided by a vendor is what you are actually looking for just because the attribute names match. It can be the case that you will need to look at a number of vendor attributes to derive the required value. For example, a source attribute may differ and depend on an instrument type classification.

Stick to your guns

If you are building “golden copy” records from multiple sources, treat client requests for alternatives with caution. The golden copy represents the preferred view of that data within your organisation. There may be valid reasons why people wish to deviate from this, but they should not be allowed to do so just on a whim.

Be religious about exception processing

Provision of high quality data in a timely fashion is at the core of any data management mission statement. It is essential to build mechanisms to enable the rapid review of data exceptions or potential problems if this principle is to be met on a consistent basis.

Purge data

Data archiving or purging is usually one of the first items to be de-scoped if a data management implementation project begins to experience pressure against targets. It may not be possible to implement a data purging process immediately for these reasons; however this is definitely something that should be addressed as soon as possible. Many data management installations struggle to perform all of the data collection, manipulation and publication activities required in a busy processing window. The capture and manipulation of unnecessary data is precisely that, unnecessary.

Talk to your peers

The well known American humorist and writer Sam Levensen once said “You must learn from the mistakes of others. You can't possibly live long enough to make them all yourself” I imagine most people would agree that learning by knowledge acquisition is often more efficient than trial and error. Yet despite the freedom with which people are prepared to discuss their experiences, the vast majority of us continue to re-invent the wheel.

Data management structure diagram

You can rest assured that any data management related problem you may be grappling with has either been encountered and resolved by one of your peers, or may be resolved simply by talking to like minded people with different experiences.

If you or your organisation has already addressed all of the above, stand up and take a bow. You are in a very small minority of organisations looked up to and envied by your data management peers. If, like the vast majority you have not considered some of these points or have tried to address them but not succeeded, I urge you to try again. I cannot guarantee that conformity will bring success, but it will certainly be an improvement on where you are today.

Return to library