Data imports and exports using Microsoft’s Data Management Framework (DMF) are not easy. Firstly, they can be time-consuming because you need to use the existing entities in the Data Management Framework which might not correspond with the data structure in other systems. Or it is possible that the data entities you need do not exist, and you need to start developing custom entities.
At To-Increase, we build solutions to solve customer challenges within Microsoft Dynamics ERPs. And we built our integration and data migration solution 17 years ago embedded inside Microsoft Dynamics to address the problem of mapping the data entities in DMF with other systems and applications.
Currently used by over 600+ customers, over time the solution has evolved from AX 2009, 2012 to the new versions of Microsoft Dynamics 365 Finance & Supply Chain Management (D365 F&SCM).
Read on to find out more about the 10 features of our application integration solution --Connectivity Studio for D365 that make it a versatile tool for data migration.
Why is Connectivity Studio for D365 F&SCM a great tool for data migration?
1. Configuration and not development
You can create messages with configuration. Select what data format is coming from the origin (ERP, Legacy) and what the structure is for those records in a source document. In the target document, select the tables, views and/or entities where the data needs to be imported into D365. In the message itself, you can do the complete mapping, set conversions, transformations, and default field values. Our integration engine gives you the ability to carry out these configurations. Also, in the mapping, you can describe the details of the field, such as financial dimension for example, or set up different possibilities.
2. Structured data imports and exports
Using Connectivity Studio, you can have different messages for migration which can be inbound and outbound. And using our platform you can ensure that you do it structurally. For example, if you want you can first import the base data, then the master data, and then the transaction data.
3. Import data using multiple options
The Data Management Framework offers roughly 1500 entities, whereas there are about 12,000 tables in D365. DMF is only able to perform integration with entities, which can be a huge challenge/limitation for data imports.
Connectivity Studio can make use of tables, views, and entities to simplify data imports. You can import information directly into tables, and use a combination of tables and have a reference between them. Optionally you can use views, entities, and tables, in a mix. So, if for your import, you create additional tables, have additional information, or the entity is incomplete, then we offer the option to select the entity and the corresponding reference table.
4. Block test data imports into production environments
The solution has been around for many years. In that time, features for migration have been added to the solution based on customer feedback and advances in technology. Anyone that has done migrations before has made the mistake or has knowledge of the issues that come up when test data was imported into the production environment. For that reason, a safeguard function was created that blocks users from accidentally importing data into the incorrect environment.
5. Web services as a connector
We offer several formats for data imports, including file-based imports, ODBC, and web services. If you would like to use web services, then what needs to change in Connectivity Studio is the document and connector setup and the rest of the functionality in the system will remain the same. You can send out the http request to an outside web service, and the outside web service with some parameters can just send the request to D365 and get the information back.
6. ODBC as a connector
We can configure direct connections using ODBC as a connector. It could be a D365 database or an AX2012, 2009 database, or a SQL server database – Connectivity Studio can connect to any database that has an ODBC driver. In the source document and the source connector, you add the input for the connector to the database, select the tables and fields and you can proceed. You can import and export from the external database.
7. Document formats
For data migration, we offer a variety of document types for data imports and exports, including text, XML, JSON, WMS journal, MS Word, MS Excel, Fixed text, Insert list, ODBC, Inventory journal, Ledger journal, Trade agreement journal table, staging, Report as a finished journal, Production picking list, Inventory compare, Project journal, ODBC Dynamics, and lastly from D365 FO.
8. Import into multiple legal entities
When configuring the data migration, in the target connector you can define the company you are importing that data, which could be one legal entity or multiple legal entities depending on how you want to set it up. There is a company-specific setup available to set defaults per company.
9. Structure your imports
In the project setup, you can structure your import. i.e., you can set up the structure of import. What is first then what’s next, etc. So, for example, you import the base data first, then customer, then item, and then the open orders, making sure that related data is imported before the next step/task starts. After this setup, the entire data migration project can be executed in the batch framework keeping in line with the setup import structure.
10. Running messages in parallel for faster processing
Connectivity Studio offers the functionality to run messages in parallel which can reduce the time for large data imports drastically. When we performed a test to run roughly 400,000 records that included data on products, customers, vendors, etc. it took about 6 hours. Then we split the files by about 10,000 each and utilized all threads on the running AOS of D365 and we managed to reduce the processing time to roughly 18 minutes from 6 hours for the migration which was also feasible for testing prior to data migration. This gave the customer the option of running many migration tests in a workday before the final import was carried out.
Are you interested in exploring our integration engine for D365 F&SCM?
While these features focus more on the data migration capabilities of Connectivity Studio, our solution’s been built for integration and can help you map your ERP, business systems, and applications all simply by configuration. Additionally, the integration tool has an application that allows you to track the health of your integrations on the go from mobile phones and tablets. So, if an integration fails, you can quickly track the failure URL and reduce the time you would otherwise have taken to fix the errors in the integration.
If you think your organization can benefit from an integration platform that also can be used for data migration, do read our honest review of the Connectivity Studio for D365 solution. And if you want to understand the differences between our solution’s integration capabilities compared to D365’s Data Management Framework, do click on the link below.
Dorian Sakowski
Michiel Toppers
James Smith
