0117 9717547

At the core of any sizable business IT system Is big data.  

The way systems interact with data is becoming more sophisticated than ever, with a need to manipulate, interpret and extract data at multiple levels, feeding into a wide range of business operations.  

For membership and not for profit organisations, data is an integral part of daily operations, with every member or donor interaction tracked and monitored at every stage within the system. Whether it’s renewals, donations, fundraising, or event attendance, every piece of information is only available, because it’s been collected and is stored within your system.  

So, what happens when you look to embark on your next digital transformation project? What are the new data requirements? Are there new features to consider? is the quality of data in your legacy system is poor?  

The truth is, no matter how fast your technology is accelerating, if you fail to get your data migration strategy right, even the most sophisticated technical solution will fail.

Understanding the problem 

Data requirements are constantly changing and as technology continues to evolve faster than ever before, there are still a huge number of organisations failing to innovate fast enough.  

This poses huge business risk, with outdated systems and legacy data significantly increasing the likelihood of duplication, system errors and unreliable business critical data. 

Although data complexities differ from organisation to organisation, there are a few key things to think about, not only if you are planning a major digital transformation project, but also if you are working in any organisation that relies on a system for business data. 

 1. Set best practices for collecting and looking after data.  

 

Building data management best practice into your organisational culture is the best way of ensuring data integrity. Simple regular checks, and a clear process within your organisation with check lists for how data should be collected and managed, will minimise risk of duplicate entries, missing data or any errors that will affect the integrity of your data in the long run. 

For most organisations, there are key times of year where bigger data shifts occur, such as renewals and during fundraising campaigns for example. Build your best practices to include regular routine data checks and a more significant review period during peak periods, for the best outcomes when it comes to data integrity within your system.  

Data best practices

 2. Cleanse your data  

 

If you are embarking on a large-scale digital transformation project and know your data is outdated and has been neglected, invest some time in cleansing data prior to starting your project with your selected development partner. 

Representatives within your organisations who interact with your system and data everyday are likely to have a good foundation of knowledge when it comes to how data is used. This is a key opportunity to try and solve any obvious data gaps and remove any glaring errors, saving time and money later in the project.  

Depending on the extent of the errors, it may be possible to manually fix. However, If the errors are seemingly major, there are an increasing number of specialist businesses that will cleanse your data, working closely with your chosen development partner.  Our typical recommendations include Microsoft Data Factory and Scribe insight software. 

Data cleansing

3.Project scoping  

 

Integrated/ end to end solutions have fast become the norm within the membership/ charity sector. CRM transformation projects now typically consist of a complex network of requirements, including website design/ development, third party integrations and customer portals, all linked together within a secure CRM environment. A complex development such as this requires significant consultancy and development time, with a best fit team being one with experience in working with similar projects and solving similar problems.  

Ensure that as part of your project there is a substantial scoping phase included within your project timelines. This will aim to identify any migration issues prior to starting the project and will allow specialists to work out how data will accurately migrate and map to elements within the new proposed system.  

This forward planning will help to define the most effective migration approach, mitigate risks and prevent any unnecessary down time, reducing the costs and overall business impact.  

Integrated/ end to end solutions have fast become the norm within the membership/ charity sector. CRM transformation projects now typically consist of a complex network of requirements, including website design/ development, third party integrations and customer portals, all linked together within a secure CRM environment. A complex development such as this requires significant consultancy and development time, with a best fit team being one with experience in working with similar projects and solving similar problems.  

Ensure that as part of your project there is a substantial scoping phase included within your project timelines. This will aim to identify any migration issues prior to starting the project and will allow specialists to work out how data will accurately migrate and map to elements within the new proposed system.  

This forward planning will help to define the most effective migration approach, mitigate risks and prevent any unnecessary down time, reducing the costs and overall business impact.  

 4. Selecting the right approach for your business  

 

There are a whole host of known methodologies for data migration projects, these are usually categorised within three main approaches.  

  • Big bang data migration – A migration within a predefined time period. There are no intermediary periods with this approach, with the entire migration process usually completed within a few days. Where this approach often fails, is where there is significant complexity within datasets or content is constantly changing. There is usually inevitable downtime when switching old for new, which can be a big risk for organisations that rely heavily on data.  
     
  • The parallel approach – For a limited time, the new system is installed and runs alongside the old one, with both operating in tandem while the transition takes place. Once the new system has been tested the old system is turned off. There isn’t typically downtime with this approach, with the old system continuing to run until the new system is ready, however if data continues to be updated on the old system while the new system is being tested, there are likely to be gaps which will need to be addressed post deployment.  
     
  • Two phased approach - This is the approach that would be perceived as best fit for membership and not for profit organisations as it  acknowledges the need for ongoing data updates. With this approach, a first migration will happen during the development phase on a pre-planned date. All integrations and development will be completed and tested, then prior to deployment, a second migration is conducted to gap fill data between first and second migration dates.  

5. Data security is paramount  

With strict new guidelines bought into place in 2018 relating to how data is stored, collected and processed, GDPR now forms a key part of any migration strategy.   This needs to be considered from the moment you decide to migrate your data, with questions asked such as; do you have logs of how your data was collected? Do you have permissions to use data? How are you planning to process data? Is your previous solution storing data securely?   Data security is paramount and selecting a development solution that is compliant from the start, with give you the peace of mind when it comes to whether you are compliant in your use of data.   Building a bespoke CRM/software solution may seem appealing at face value, but this can bring a significant risk when it comes to data management and security. Instead our recommendation is to work with an established infrastructure such as Microsoft, where you know the underlying system is widely used and compliance is at the forefront of solutions.  

data security

6. Cloud Hosting 

 

The days of storing data within physical server environments are numbered, with 60% of all enterprise workloads due to be in the cloud by 2020. Cloud hosting within an established environment such as Microsoft Azure is not only more cost effective and scalable but will also make the process of data migration easier, minimise security risks and reduce risk of data breaches.  

 7. Reporting and testing


Migration reporting should be an integral part of the testing process from your development team. Your development partner will use this phase to identify successes, failures and any partial migrations that occurred during the migrationAs a first step they will complete a record count and look at data sample subsets to make sure there is a match between old and new. Once complete, the new system will be deployed to a test environment for user testing.  

 

Make full use of this user testing period, as this will be the last opportunity to check the integrity of your data before the solution goes live. It is critical for key stakeholders within the organisation to review their data and check all fields look correct, because at this stage it is still possible to make changes without effecting the live version of the system.  

For any data migration project to run smoothly, buy-in from key stakeholders is essential. Although it requires a time investment, involvement from key representatives within each phase of the project timeline can mitigate migration errors and decrease project costs and delays associated with migration errors.  

When you start your digital transformation, make sure you select a development partner who has experience in delivering similar scale projects, with adequate planning during the early phases, to make sure you have a clear idea of strategy and data security prior to starting the project.  

Perhaps the biggest thing to take from this, is the need to always be ready for data migration. Build a culture within your organisation, where teams are invested in regular data cleansing exercises and routine checks. This will increase the lifespan of your system and significantly reduce risk of errors.  

;

Ready to get started?

Get in touch with us today