“Dynamics was an old system from an old company that was difficult, costly, and no longer met our needs.”
Catalogic Software, a new data protection company spun off from New Jersey-based enterprise data company, Syncsort, was implementing salesforce.com CRM for sales and marketing operations. The initial implementation involved the migration, deduplication, and enrichment of a test set of records, approximately 50,000, to Salesforce.
Within a few weeks, based on the success of the initial project, Catalogic decided to move forward with a full migration project by splitting their division-specific data from the former parent company, and fully moving to salesforce.com. Customer, accounting, and other business data, totaling approximately a million records for all divisions of the company, were housed in a shared Microsoft Dynamics implementation that needed to be split and migrated to other systems. Microsoft Dynamics had become a costly business expense both in terms of its monthly costs and IT staff expertise requirements. The costs just didn’t justify maintaining the system and outdated operations.
Meet Catalogic Software
Catalogic Software provides cataloging, backup, disaster recovery, and copy data management solutions in any combination of physical, virtual, or cloud environments. Begun in 1968, the company was originally a part of enterprise software and big data company, Syncsort. Spun off in 2013, Catalogic focuses on data protection management. Headquartered in New Jersey, Catalogic has subsidiaries in the United Kingdom and Germany. An international network of partners, system integrators, and resellers sell and deploy Catalogic data protection solutions.
The Challenge
Migrating data from an ERP system is challenging enough. But for Catalogic, the challenges were even greater. The data stored in a Microsoft Dynamics SQL server was shared by all divisions of the parent company. A long history, various levels of autonomy between divisions, multi-national operations, un-managed manual processes, and related key data stored in other systems meant there were no standards for data management. The ways in which the records owned by each division were identified, varied and were undefined. And each division had its own requirements for how its data was to be structured in the new system. Further complicating the project was a large amount of legacy data that needed to be preserved in the migration process. The project was more than simply moving basic contact data from one CRM to another. In addition to leads, contacts, accounts, and other attached objects, Catalogic needed to preserve custom objects, attached files and documents, and a specialized alerting system.
Choosing the right service
Catalogic turned to the data management and services arm of Dallas-based software company, Symphonic Source Inc. to design a project plan, execute the project, and to set up new systems as part of the migration. “Symphonic Source was the clear choice for us,” said Brian Sietsma, Sales Operations Analyst at Catalogic. “They’re known for their experience and expertise with salesforce.com data management, but that’s only the tip of the iceberg. We were impressed with their customer service and responsiveness to our CFP. And we were reassured by the fact that Symphonic Source does not outsource or sub-contract its services.”
Timing
Timing was critical as the new company needed its data set functional in order to begin sales and marketing efforts. Syncsort’s implementation of Microsoft Dynamics had been heavily and extensively customized, which meant the Microsoft Dynamics migration wizard was not a possibility for use.
Advanced Customizations
Typical data migrations focus on the primary contact data—leads, contacts, accounts. But the Catalogic project was far from typical. Various business divisions required various attached objects (emails, tasks, alerts, documents), custom objects, and specialized data fields and formulas be included in the migration. Because of the advanced customization of the Microsoft Dynamics implementation, these data points were stored in disparate ways and the connection back to the root record was far from uniform.
Multi-National Nature
The multi-national nature of Syncsort’s and Catalogic’s business divisions did not make things easy either. With two offices in the European Union, and customers worldwide, the project needed to take into account EU rules on data security as well as ensure that customer data reflected the correct currency at the correct exchange rate. Language variations with differences in diacritical marks and non-Latin alphabets further complicated the project.
“Essentially we were dealing with a data set that was highly complex. Historical data was static, but the root records remained dynamic with changing regulations, currency markets, and Syncsort/Catalogic operations,” said Lars Nielsen, Symphonic Source project lead for the migration. “Historical data impacted reporting and analysis, and current dynamic data impacted forecasting. We had to account for all of these needs and ensure the migrated data could function in the same way, and better, as it had in the original Microsoft Dynamics implementation.”
“Essentially we were dealing with a data set that was highly complex.”
The Solution
Symphonic Source put together a plan to divide the data according to business divisions, cleanse and dedupe the data, and migrate the Catalogic data to its new salesforce.com CRM.
For security reasons, direct access to the Microsoft Dynamics system was restricted. Further it was not possible to export the underlying globally unique identifier (GUID) for each record, which tied the records and their related data together. So, working from a 90 GB database backup, Symphonic Source’s data experts began to analyze the database schema and how the data had been structured.
Protect Attachments
Attachments to the various records posed one of the greatest challenges. Saving a file and re-uploading it to a new database is, in itself, an easy process. But with so many file types (spreadsheets, presentations, diagrams, documents, etc.) and so many records, extracting the attachments and tying them to the correct record in the new database was tricky. In the end, Symphonic Source used the mime type of each file to extract and save the file, and then created a CSV file with the name, description, file location, and the parent record to upload into the new salesforce.com database.
Assign and Manage User IDs
The users table in Microsoft Dynamics also posed certain challenges. Syncsort had more than 500 original user IDs in Microsoft Dynamics, but only about 90 would be created in the Catalogic Salesforce CRM. User IDs had different permission settings restricting what could and could not be accessed. Not only did Symphonic Source have to keep all of the relationships between users and their records, those records owned by non-Catalogic users had to be re-assigned according to Catalogic’s requirements. Symphonic Source created a custom field on the user object in the Salesforce CRM to store the Microsoft Dynamics GUID. Then the Salesforce CRM ID was tied back to the user table in Microsoft Dynamics. Records owned by users in Microsoft Dynamics who were not being migrated to salesforce.com had to be assigned new Salesforce GUIDs in order to properly create and populate the account table. In Salesforce, a batch apex job was run to replace the raw GUID data from Microsoft Dynamics with a friendly name.
Preserve Old Fields
Old field values from more than 100 custom fields in Microsoft Dynamics remained a part of the backup file Symphonic Source worked from. Because each division had different needs with respect to the data, it was impossible to determine which fields remained relevant to business operations moving forward. Symphonic Source’s solution was to create a section on the account object in salesforce.com to store all of the data where it could be used and referenced as needed, while also not cluttering the key data fields for each record. Fields were prefixed with a reference to the Microsoft Dynamics database and hidden from standard user and portal user views.
Maintain Relationships
Record relationships were also an issue. In Microsoft Dynamics, every object exists in two tables, a base table and an extension. When moving to Salesforce CRM, Symphonic Source had to create a holding place for the parent ID so that record data could be correctly re-parented once it had made it into the Salesforce database. To get a full picture of a record, the two tables had to be joined against all tables with lookup values relating to the record. The native merge function of Microsoft Dynamics only performs a soft delete of the record so related objects and attachments function in a lookup relationship to the soft deleted record. Symphonic Source had to maintain this relationship so that objects could be properly parented in salesforce.com and then dedupe with a hard delete.
Accounts were loaded into Salesforce CRM first, creating Salesforce ID numbers. Those ID numbers were then linked back to Microsoft Dynamics contacts and opportunity IDs, as well as custom objects, creating join requests in order to create the correct relationships in Salesforce.
Address Time Gaps
Because Symphonic Source was working from a backup file of Microsoft Dynamics data, by the time the migration process was completed and the new database constructed, the data backup was nearly two months old. In the interim period, all divisions of Syncsort and Catalogic continued to use the Microsoft Dynamics database. To compensate for any changes to the data set in that period, and to ensure the most current, complete, and accurate data in the new salesforce.com database, Symphonic Source ran a “Modified Report” from the Microsoft Dynamics front end in order to bring over to salesforce.com any changes to the data in the time since the backup file had been created.
Map Fields
In Microsoft Dynamics, fields don’t always correspond to the tables where the data is stored. The order of steps Symphonic Source followed was to first create Microsoft Dynamics users as Salesforce users, then create account tables in Salesforce, and finally to recreate the fields from Microsoft Dynamics in Salesforce. Because of the data’s importance to forecasting, reporting, and operations, it was critical to understand what a “field” is in Microsoft Dynamics versus Salesforce and how they relate.
“It is critical to ask the right questions in order to make the right choices for how to migrate and structure the data,” said Symphonic Source’s Nielsen. “The discovery phase of the project is arguably more important than the actual technical migration of the data. Had we not spent time talking to Syncsort/Catalogic, their data could have easily and irreparably been damaged.”
“Symphonic Source is known for their experience and expertise with salesforce.com data management. But that’s only the tip of the iceberg.”
The Outcome
The goal of Catalogic in migrating to a Salesforce CRM was efficiency and effectiveness. The Microsoft Dynamics database was unwieldy, expensive and difficult to manage, and was not used in an effective and meaningful way. The Salesforce platform offered greater ease-of-use and granularity to data analysis.
“After becoming an independent company, it made good business sense to rethink our operational procedures,” said Catalogic’s Sietsma. “Microsoft Dynamics was an old system from an old company that was difficult, costly, and no longer met our needs. Moving to Salesforce CRM was the clear choice so long as we could get our data moved and structured in the right way while minimizing the impact on day-to-day functions. ”
Within weeks of using its new system exclusively, Catalogic is realizing the benefits of a multi-tenancy, cloud-based, SaaS platform. “While this process was daunting, the beauty is that we are now able to access all of our relevant data structured in ways that are meaningful to our business from the old system, but in a new flexible and agile platform that adjusts to meet our needs,” said Sietsma. “Had we not been guaranteed that are data would survive the migration, we never would have made the switch. This goal would not have been achieved without Symphonic Source.”
“This goal would not have been achieved without Symphonic Source.”