Sitecore released the first version of Data Exchange Framework with Sitecore 8.2 initial release. Sitecore has already released three updates of Data exchange framework and Sitecore Data Exchange framework 1.3 is the latest release available.
Data exchange framework allows you to the transfer the data between two systems. You can think of the Data Exchange Framework as standalone
ETL (
Extract Transform Load) tool that extracts data from the source system, transform it as appropriate and loads them into the target system. Sitecore can be one of the two systems - either the source or target but it is not mandatory. This framework can be used for exchanging data between any two independent systems. A typical example can be product sync between your ERP and a Digital Asset Management system using Data Exchange framework. Developers can build their own custom provider that allow 3rd party system to serve as source or target system.
Documentation for data exchange is
available here and steps for implementing a new provider is documented very well. Follow instructions from
Developer Guide to build your own custom provider. After reading through the official documentation I thought it would be nice to create a working custom provider thus I have implemented XML System Provider for Data Exchange Framework. This provider reads XML data from file or web address and creates items into Sitecore. If you ever desired to extract daily hundreds of entries out of a XML stream and put into Sitecore then this XML provider for Data Exchange Framework can make your task easy.
I am using Data Exchange Framework 1.3 and Sitecore 8.2 Update 2 release. I’ve divided this topic into two blog posts:
- Building XML Provider for Data Exchange Framework 1.3
- Use XML Provider for Data Exchange Framework 1.3
Configuring Data Exchange Framework 1.3
- Download Data Exchange Framework 1.3 Sitecore package and install it in your Sitecore 8.2 instance.
- Download Sitecore Provider for Data Exchange Framework 1.3 Sitecore package and install it in your Sitecore 8.2 instance.
Building XML provider for Data Exchange Framework 1.3
In this blog post I am not going to re-write the provided documentation, but will touch up high-level steps and configuration and provide the code I have written while playing with Data Exchange framework. The full source code of XML provider is located on
GitHub. I strongly encourage you to follow official documentation step by step to build your own custom provider for Data Exchange framework.
- In Visual studio create a new “Class library” project using .NET Framework version 4.5.2
- Add references to below three assemblies:
a. Sitecore.DataExchange
b. Sitecore.DataExchange.DataAccess
c. Sitecore.Services.Core
- Add Template Folder for XML Provider. Follow the steps mentioned here in official documentation.
- Add Endpoint Template. An endpoint represents a data source. The data source may support the ability to read data, to write data, or to both read and write data. The endpoint does not provide the ability to interact with the data source. It simply represents the data source.
Examples of endpoints are:
a. URL for a web service used to read data from a CRM
b. Path to a directory used to read files
c. Connection string used to connect to a relational database
I’ve created XML system endpoint which has below two fields:
XMLPath: The path can be either a local XML file or a web address returning XML Data (http or https).
Web Address Example: https://www.w3schools.com/xml/cd_catalog.xml
XMLNodeName: Name of XML node which will be used to get a collection of matching nodes from XML Data. For example, XMLNodeName will be CD in below XML document:
<CATALOG>
<CD>
<TITLE>Empire Burlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST>
<COUNTRY>USA</COUNTRY>
<COMPANY>Columbia</COMPANY>
<PRICE>10.90</PRICE>
<YEAR>1985</YEAR>
</CD>
<CD>
<TITLE>Hide your heart</TITLE>
<ARTIST>Bonnie Tyler</ARTIST>
<COUNTRY>UK</COUNTRY>
<COMPANY>CBS Records</COMPANY>
<PRICE>9.90</PRICE>
<YEAR>1988</YEAR>
</CD>
</CATALOG>
- Implement Endpoint Settings Plugin. The Endpoint settings class will store the configuration information once information is extracted from the Endpoint Sitecore item.
- Implement Endpoint Converter. A converter is needed to convert a Sitecore item that represents a XML System endpoint into an endpoint component for Data Exchange Framework, and that adds the endpoint settings plugin to the endpoint component.
- Add Pipeline Step Template. A pipeline step is needed to represent a logic involved with reading XML data from an endpoint and handling the data that is read. A template is needed to represent the pipeline step. It allows the pipeline step to be configured.
- Implement Pipeline Step Converter. A converter is needed to convert a Sitecore pipeline step into a pipeline step component for Data Exchange Framework, and that adds the settings configured on the pipeline step item using the appropriate plugin.
- Implement Pipeline Step Processor. A processor is needed to implement the business logic for the pipeline step.
- Add Value Accessor Template. A value accessor is the component that is used to read a value (value reader) from a source object or write a value (value writer) to a target object.
| Data Exchange Framework | C# |
|---|
| Value accessor | Property |
| Value reader | Property getter |
| Value writer | Property setter |
A value reader is a component that is able to read a value from the source object. For example, if the source object represents a XML node (<CD>) from XML document, you might want to read the value of any specific XML element or field (e.g.: TITLE or ARTIST).
<CD>
<TITLE>Empire Burlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST>
<COUNTRY>USA</COUNTRY>
<COMPANY>Columbia</COMPANY>
<PRICE>10.90</PRICE>
<YEAR>1985</YEAR>
</CD>
A value writer is a component that is able to write a value to the target object. Being able to write values is an essential part of mapping values from the source object to the target object.
- Implement Value Accessor Converter. A converter is needed to convert a Sitecore value accessor item into a value accessor component for Data Exchange Framework.
In
next blog post, I’ll explain how to use XML provider for Data Exchange Framework 1.3. Comments and suggestions are most welcome. Happy coding!
I am working on CD Catalog Pipeline Batch item of Sitecore Data Exchange Framework 1.3 and whenever I click on Run Pipeline Batch button available in Data Exchange Ribbon, it creates multiple items under CD Catalog item from XML file. I am developing and testing custom XML provider for Data Exchange framework thus there is a frequent need to delete all subitems under CD Catalog item except CD Catalog item folder. If I right click on CD Catalog item then Sitecore Context Menu gives me option to delete CD Catalog item which will eventually delete all subitems under it but I don’t want to delete CD Catalog item folder.
Posts
