An introduction to Sitecore data providers

I recently responded to a SDN forum post looking for advice on setting up a custom data provider in Sitecore.

http://sdn.sitecore.net/SDN5/Forum/ShowPost.aspx?PostID=52346

I had previously written a read-only data provider to pull in product information from a RESTful api.  While I was able to find what I needed it took me a bit to piece together all of the information needed to accomplish this task.  This is my attempt to clarify things some or at least bring together the resources I found helpful.

First a little background:  In this post we’ll be setting up a data provider for the master database to get a list of products.  Things have been intentionally kept simple for the purposes of this demo.  I’ve created two methods GetProduct(int key) and GetProducts() which provide us with a single product or enumeration of all products.  I won’t bother listing them out since this will be specific to your solution so you’ll just have to use your imagination for now.

There are other options on how you could configure the data provider.  For example, if you had a news article feed you might want to use the same provider for both the content authoring and content delivery.   If you do choose to implement something like this there are some additional things to keep in mind that I’ll outline later.

There are a few things that need to happen in order to integrate an external data source with Sitecore and we’ll get there soon enough but before we begin its important to understand what the IDTable is.

The IDTable exists in each Sitecore database (core, master and web).  By default it is only used in the master database (as defined in the web.config).  The IDTable maps primary keys from external data to Sitecore IDs and the parent Sitecore item.  To avoid conflicts with the keys a prefix is stored along with the external data sources identifier.  Each external data source should have its own prefix.  In our news article example above we would want to make sure that both the CA and CD environments use the same IDTable, which means sharing a database somewhere.  In this case, we may also need to hook into the idtable:added and idtable:removed events to notify the history engine of a change.

The first step is to create a class that extends Sitecore.Data.DataProviders.DataProvider.  Create a public constructor with whatever parameters needed.  For this simple data provider I need the following pieces: a prefix for the IDTable, the template to map external data to and the parent item of our products.

[sourcecode language="csharp"]public class ProductProvider : DataProvider{

… public ProductProvider( string IDTablePrefix, string templateID, string parentTemplateID)[/sourcecode]

GetChildIDs – This method returns all the children of a given item.  Since the data provider will be checked for any Sitecore item we’ll need to first determine if we can handle the current item, which was set up in the config file and passed to the constructor.  If we can process the current item we’ll make a request to our API to get all the products.  For each of the products we’ll look up the Sitecore ID from the IDTable or create it if it doesn’t exist.  These IDs will then be returned.

[sourcecode language="csharp"]public override IDList GetChildIDs(ItemDefinition item, CallContext context)

{

if (CanProcessParent(item.ID, context))

{

IDList idList = new IDList();

var products = GetProducts();

foreach (var product in products)

{

idList.Add(CreateOrGetID(prefix, product.ProductId, item));

}

context.DataManager.Database.Caches.DataCache.Clear();

return idList;

}

return base.GetChildIDs(item, context);

}[/sourcecode]

GetParentID – Here we will return the ID of the parent item.  To make things simple this is stored in the IDTable.

[sourcecode language="csharp"]public override ID GetParentID(ItemDefinition item, CallContext context)

{

if (CanProcessItem(item.ID, context))

{

context.Abort();

IDTableEntry[] idEntries = IDTable.GetKeys(prefix, item.ID);

if (idEntries != null && idEntries.Length > 0)

{

return idEntries[0].ParentID;

}

return null;

}

return base.GetParentID(item, context);

}

[/sourcecode]

GetItemFields – This is where the magic happens.  We’re going to start by checking if we can process the item given. From here we’ll clear the cache and use the IDTable and Item ID to look up the external data key.  The external key is used to call the API and get the details for our product.  With this we’ll build up and populate a list of fields.

[sourcecode language="csharp"]

public override FieldList GetItemFields(ItemDefinition item, VersionUri version, CallContext context)

{

if (CanProcessItem(item.ID, context))

{

if (context.DataManager.DataSource.ItemExists(item.ID))

{

ReflectionUtil.CallMethod(

typeof(ItemCache), CacheManager.GetItemCache(context.DataManager.Database),

"RemoveItem", true, true, new object[] { item.ID });

}

CoreItem.Builder result = new CoreItem.Builder(item.ID, item.Name, item.TemplateID, context.DataManager);

int originalID = int.Parse(GetIDTableKey(item.ID));

var product = GetProduct(originalID);

foreach (var propertyInfo in product.GetType().GetProperties())

{

string name = propertyInfo.Name;

string value = propertyInfo.GetValue(product, null).ToString();

result.AddField(name, value);

}

return result.ItemData.Fields;

}

return base.GetItemFields(item, version, context);

}

[/sourcecode]

The only missing piece now is updating the configuration.  Create a new .config file in the App_Config/Include folder and create provider node under sitecore/dataProviders.  Set the type to your class and create param elements for each of the parameters in your constructor.

Once the provider is defined we’ll need to associate it with a database.  In the config file patch in our provider to the master database.

[sourcecode language="xml"]

<sitecore>

<dataProviders>

<ProductProvider type="DataProviders.Providers.ProductProvider, DataProviders">

<param desc="IDTablePrefix">Products</param>

<param desc="templateID">{6693437E-99B3-42CE-A632-4F7F23D38208}</param>

<param desc="parentTemplateID">{E5A7321A-1243-44FF-92C8-A6C6EB2E2567}</param>

</ProductProvider>

</dataProviders>

<databases>

<database id="master" singleInstance="true" type="Sitecore.Data.Database, Sitecore.Kernel">

<dataProviders>

<dataProvider patch:before="*[1]" ref="dataProviders/ProductProvider"/>

</dataProviders>

</database>

</databases>

</sitecore>

</configuration>

[/sourcecode]

I’d like to point out that most of the code seen came from the Northwind data provider, which, as all things Northwind seem to be, proved to be a useful example and learning tool.  Links to this and other resources can be found below.  Much of what I learned came from the following places and would be a great place to start looking if you needed more information.

References:

1. http://marketplace.sitecore.net/en/Modules/Northwind_Data_Provider.aspx
2. http://sdn.sitecore.net/Developer/Integrating%20External%20Data%20Sources/Implementation/Read-only%20Sources.aspx
3. https://github.com/hermanussen/MongoDataProvider
4. http://marketplace.sitecore.net/en/Modules/YouTube_Integration.aspx
5. http://www.sitecore.net/Community/Technical-Blogs/John-West-Sitecore-Blog/Posts/2012/05/When-to-Implement-Data-Providers-in-the-Sitecore-ASPNET-CMS.aspx
6. http://hermanussen.eu/sitecore/wordpress/2012/05/making-sitecore-faster-with-mongodb/