Thursday, August 25, 2016

Sitecore remove renderings or sublayouts with missing datasources

As a Sitecore implementation starts to get larger and older, items can start to have renderings or sublayouts which refer to datasources which no longer exist. The main cause for this is usually a content editor deleting the datasource item and leaving the links intact.

The following code can be setup as a scheduled task in Sitecore or added to a custom admin page to be run as needed. It will recursively move down the content tree and find items which have a layout, it then checks the renderings on those items with datasources and ensures the datasource exists. If a rendering with a broken datasource is found, it is removed from the item.
using Sitecore.Layouts;
using Sitecore.Data.Fields;
using Sitecore.Diagnostics;

public partial class DatasourceFix : System.Web.UI.Page
{
 private static Database database = Sitecore.Configuration.Factory.GetDatabase("master");
 private static string StartPath = "/sitecore/content/"; // Start path to enumerate
 private static string DefaultDevice = "{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}"; // Default device ID

 protected void btnRun_Click(object sender, EventArgs e)
 {
  var startItem = database.Items.GetItem(StartPath);
  CheckItem(startItem);
 }

 private Item CheckItem(Item mainItem)
 {
  RenderingRemover(mainItem);

  if (mainItem.HasChildren)
  {
   foreach (var myInnerItem in mainItem.Children.ToList())
   {
    CheckItem(myInnerItem);
   }
  }

  return null;
 }
 
 private void RenderingRemover(Item item)
 {
  if (item.Fields[Sitecore.FieldIDs.LayoutField] != null && !String.IsNullOrEmpty(item.Fields[Sitecore.FieldIDs.FinalLayoutField].Value))
  {
   // If item has a layout
   LayoutField layoutField = new LayoutField(item.Fields[Sitecore.FieldIDs.LayoutField]);
   LayoutDefinition layout = LayoutDefinition.Parse(layoutField.Value);
   DeviceDefinition device = device = layout.GetDevice(DefaultDevice); // Get the default device

   // Get renderings for the device
   var d = layout.GetDevice(device.ID.ToString());
   var ren = d.Renderings;

   // Query the renderings for those with a datasource
   var withDatasource = from RenderingDefinition renDef in ren
      where !string.IsNullOrEmpty(renDef.Datasource)
      select renDef;

   bool itemEdited = false; // item edited flag
   
   // Find missing datasources
   foreach (RenderingDefinition rendering in withDatasource.ToList() ?? Enumerable.Empty())
   {
    var datasource = database.GetItem(new Sitecore.Data.ID(rendering.Datasource)); // Get the datasource

    if (datasource == null)
    {
     // if datasource is broken, flag an item edit and remove the rendering
     itemEdited = true;
     d.Renderings.Remove(rendering);
     Log.Info(string.Format("Removing rendering {0} from item {1} due to datasource not being found.", rendering.ItemID.ToString(), item.Paths.Path), "");
    }
   }

   // edit the item if rendering(s) were removed
   if(itemEdited)
   {
    item.Editing.BeginEdit();
    layoutField.Value = layout.ToXml();
    item.Editing.EndEdit();
   }
  }
 }
}
It can be quite an expensive operation as it will go through an entire content tree, so be careful when/how it is run.

Sitecore removing a rendering or sublayout programatically

Sometimes a requirement comes up to remove renderings or sublayouts from Sitecore items programatically. This could be used in the case of a pipeline to detect and remove renderings/sublayouts with broken data sources or to bulk remove/change renderings/sublayouts which have been removed/replaced.

The code below will get a Sitecore item, then all renderings on that item which fall under the default device. It then queries to find a specific rendering, and edits the item with that rendering removed.
var database = Sitecore.Data.Database.GetDatabase("master");
var item = database.Items.GetItem("/sitecore/content/Home/Test/Test-Page/");

// If item has a layout
if (item.Fields[Sitecore.FieldIDs.LayoutField] != null && !String.IsNullOrEmpty(item.Fields[Sitecore.FieldIDs.LayoutField].Value))
{
 LayoutField layoutField = new LayoutField(item.Fields[Sitecore.FieldIDs.FinalLayoutField]);
 LayoutDefinition layout = LayoutDefinition.Parse(layoutField.Value);
 DeviceDefinition device = device = layout.GetDevice("{FE5D7FDF-89C0-4D99-9AA3-B5FBD009C9F3}"); // Get the default device

 // Get renderings for the device
 var d = layout.GetDevice(device.ID.ToString());
 var ren = d.Renderings;

 // Query the renderings for a specific one - use any logic here
 var rendering = from RenderingDefinition renDef in ren
  where renDef.ItemID == "{CD8C466E-22D3-4791-9871-C230AA57F71B}"
  select renDef;

 if (rendering.FirstOrDefault() != null)
 {
  item.Editing.BeginEdit();
  d.Renderings.Remove(rendering);
  layoutField.Value = layout.ToXml();
  item.Editing.EndEdit();
 }
}
The logic could be expanded to fit the needs of your given requirements.

Wednesday, August 24, 2016

Sitecore tip on the web database prefetch cache

Sitecore has a number of different caching areas which all work together to (hopefully) provide a seamless CMS experience with reasonable load times for content editors and end users alike. Sitecore explains the prefetch caches rather well as:
Prefetch caches are populated at application initialization, based on the information provided in the prefecth configuration files. This results in a smoother user experience after an application restart.However, excessive use of the prefetch cache can increase the time required for the application to restart, giving a negative user experience.
To understand how the prefetch caches are utilized once the application is up and running, a view into how Sitecore caching works is required:
When a database item cache does not contain an entry for an item, Sitecore retrieves the corresponding entry from the database data cache, converts it to a different type, and stores that converted data as an entry in the database item cache. When the database data cache does not contain an entry for an item, Sitecore retrieves the corresponding entry from the database prefetch cache, converts it to a different type, and stores that converted data as an entry in the database data cache. When an entry does not exist for an item in a database prefetch cache, Sitecore retrieves that item from the data provider for that database, converts it to a different type, and stores that converted data as an entry in the database prefetch cache.
What this means is that the prefetch caches are not only populated at initialization, but during the life of the application. This understanding is required when thinking about setting the size of the prefetch caches.
In the Website\App_Config\Prefetch folder of your Sitecore website there are three files which define the prefetch cache for the web, core and master databases. Here the maximum size for the caches is set as well as some other options:
  • <template> — This element tells Sitecore to cache all items based on a given template at application initialization.
  • <children> — This element tells Sitecore to cache all of the children of the specified item. 
By default, the web database prefetch will be configured to cache the default home item and all of it's children. However in many instances that home item is deleted or left unused. Therefore the tip is to investigate which items or templates you want cached for the web database of your Sitecore instance. In many cases, having the main home item and it's children will lead to performance benefits.

Remember that it also leads to a slower application startup which is not as much as a worry for production as development.

Friday, August 19, 2016

Bundling CSS and JavaScript with Sitecore

In terms of optimizing web pages for a better user experience and less load on the web server, bundling of css and javascript files can provide one of the quickest wins. The benefits of bundling the css and javascript are:
  1. Less HTTP requests on the web server - for example a single javascript file request instead of 10+, which is a real help when a server is under load.
  2. Less file size as the bundle is minified.
  3. "Bundling and minification in ASP.NET 4.5 is performed at runtime, so that the process can identify the user agent (for example IE, Mozilla, etc.), and thus, improve the compression by targeting the user browser (for instance, removing stuff that is Mozilla specific when the request comes from IE)."

Setting up bundling in your Sitecore solution (web forms)

  1. Install the Microsoft.AspNet.Web.Optimization nuget package. In this example I installed version 1.1.3
  2. Because the the nuget package above has some dependencies that Sitecore uses, set the WebGrease and Newtonsoft.json references to Copy Local false. As we don't want to overwrite the newer versions installed by Sitecore.
  3. The following dependent assembly changes need to be inside the web.config file
  4. <dependentAssembly>
      <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
      <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="6.0.0.0" />
    </dependentAssembly>
    <dependentAssembly>
      <assemblyIdentity name="WebGrease" publicKeyToken="31bf3856ad364e35" />
      <bindingRedirect oldVersion="0.0.0.0-1.6.5135.21930" newVersion="1.6.5135.21930" />
    </dependentAssembly>
    
    The Newtonsoft one will likely already exist, but the WebGrease one won't. Check the DLL versions installed by your version of Sitecore. I am on 8.1 so my Sitecore versions were newer than those used by System.Web.Optimization
  5. In my case, I will be bundling one bundle of CSS and one bundle of JS (referenced at the body closing tag to stop JavaScript page rendering blocking). My config/definitions of the bundle is as follows.
  6. public class BundleConfig
    {
     public void Process(PipelineArgs args)
     {
      BundleConfig.RegisterBundles(BundleTable.Bundles);
     }
    
     public static void RegisterBundles(BundleCollection bundles)
     {
      bundles.Add(new ScriptBundle("~/bundles/Global").Include(
       "~/assets/plugins/bootstrap/js/bootstrap.min.js",
       "~/assets/plugins/bootstrap-hover-dropdown.min.js",
       "~/assets/plugins/back-to-top.js",
       "~/assets/plugins/jquery-placeholder/jquery.placeholder.js",
       "~/assets/js/main.js"));
    
      bundles.Add(new StyleBundle("~/Content/Global").Include(
       "~/assets/plugins/bootstrap/css/bootstrap.min.css",
       "~/assets/plugins/font-awesome/css/font-awesome.css",
       "~/assets/css/styles.css"));
     }
    }
    
    Normally with bundles you define them inside the global.asax file, however as I don't want to edit the out-of-box Sitecore file, I am hooking into the initialize pipeline (which runs on the start off the application).
  7. The patch file to register the pipeline is:
  8. <?xml version="1.0" encoding="utf-8" ?>
    <configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
      <sitecore>
        <pipelines>
          <initialize>
            <processor type="MyProject.Helpers.BundleConfig, MyProject" />
          </initialize>
        </pipelines>
      </sitecore>
    </configuration>
    
  9. Now on the layout or master page you reference the assembly:
  10. <%@ import namespace="System.Web.Optimization" %>
    
  11. Now a runat server needs to be added to the head element
  12. <head runat="server">
    
  13. The CSS is registered in the head like so:
  14. <asp:PlaceHolder runat="server">
        <%: Styles.Render("~/Content/Global") %>
    </asp:PlaceHolder>
    
  15. The JS is registered in the page like so:
  16. <%: Scripts.Render("~/bundles/Global") %>
    
After a deploy your css and js will now be nicely bundled and minified and you should get a good scropt from YSlow!


Monday, August 1, 2016

Sitecore inconsistencies with the reminder feature

In the Sitecore content editor, there is a handy feature which allows content authors to set a reminder on a content item. This feature will then email the user with the reminder message they set at the specified date/time.

The first problem which I encountered was that the same reminder was being sent multiple times (from the master database and the web database). If an item had an older reminder set and you then published to the web database, the reminder would be re-added to the queue and sent again. The fix for these problems has been addressed by Christian Kay Linkhusen and involves overriding the task database to stop reminders from being sent off the web database.

The second problem I encountered was that some reminders were begin sent as expected where others would not. After looking at the tasks table in the core database, I noticed that when I set a reminder on the Content Management server, it would then be added into the table without an instance name. The task database agent is a timed job which is run on both content management and content delivery servers. When this is run (GetPendingTasks - to get pending tasks to process), the first method it calls is AssignTasks, which looks for any tasks without an instance name and assigns them to the current instance. This meant that one of the content delivery servers could then take ownership of a newly created reminder, and the content management server would never see it. Because this item is in the master database, the content delivery server could never access it so the reminder would never send.

The fix for this problem was to take a copy of the GetPendingTasks() method and remove the AssignTasks() method from running. The code to get the pending tasks actually gets all tasks due to run assigned to the current instance, or where the instance is null - so the tasks without the instance names would still send. The fix above actually overrides the method which calls GetPendingTasks() so these two fixes work well together.

using Sitecore;
using Sitecore.Configuration;
using Sitecore.Data;
using Sitecore.Data.Sql;
using Sitecore.Data.SqlServer;
using Sitecore.Diagnostics;
using Sitecore.Diagnostics.PerformanceCounters;
using Sitecore.Reflection;
using Sitecore.Tasks;
using System;
using System.Collections;
using System.Data.SqlTypes;
using Sitecore.Common;

namespace MyProject.Sitecore.Override
{
    public class TaskDatabaseAgent : Sitecore.Tasks.TaskDatabaseAgent
    {
        public new void Run()
        {
            Task[] pendingTasks = GetPendingTasks(); // Call our copy of the method here
            this.LogInfo(string.Concat("Processing tasks (count: ", (int)pendingTasks.Length, ")"));
            Task[] taskArray = pendingTasks;
            for (int i = 0; i < (int)taskArray.Length; i++)
            {
                Task logActivity = taskArray[i];
                try
                {
                    logActivity.LogActivity = this.LogActivity;

                    //Test if the Task is the EmailReminderTask and the current database is not master database
                    if (logActivity.GetType() == typeof(EmailReminderTask) && !logActivity.DatabaseName.ToLower().Equals("master"))
                    {
                        //If its not the master database
                        //Only mark the Task as done, don't execute the reminder
                        Globals.TaskDatabase.MarkDone(logActivity);
                    }
                    else
                    {
                        logActivity.Execute();
                    }

                    JobsCount.TasksTasksExecuted.Increment((long)1);
                }
                catch (Exception exception)
                {
                    Log.Error("Exception in task", exception, logActivity.GetType());
                }
            }
        }
        public Task[] GetPendingTasks()
        {
            //this.AssignTasks(); // Removed as CD servers were assigning master tasks to themselves
            ArrayList arrayLists = new ArrayList();
            int num = 8;
            string str = "SELECT [ID], [NextRun], [taskType], [Parameters], [Recurrence], [ItemID], [Database], [InstanceName] \r\n        FROM [Tasks] \r\n        WHERE [NextRun] <= @now AND [Pending] = 1 AND [Disabled] = 0 AND ([InstanceName] IS NULL OR [InstanceName] = @instanceName) \r\n        ORDER BY [NextRun]";
            object[] utcNow = new object[] { "now", DateTime.UtcNow, "instanceName", Settings.InstanceName };
            object[] array = SqlUtil.GetArray(str, utcNow, Settings.GetConnectionString("core"));
            for (int i = 0; i < (int)array.Length - (num - 1); i = i + num)
            {
                ID d = ID.Parse(array[i]);
                DateTime dateTime = ((DateTime)array[i + 1]).SpecifyKind(DateTimeKind.Utc);
                string str1 = array[i + 2] as string;
                string str2 = array[i + 3] as string;
                string str3 = array[i + 4] as string;
                ID d1 = ID.Parse(array[i + 5]);
                string str4 = array[i + 6] as string;
                string str5 = array[i + 7] as string;
                Task task = this.CreateTask(str1, dateTime);
                if (task != null)
                {
                    task.ID = d;
                    task.Parameters = str2;
                    task.RecurrencePattern = str3;
                    task.ItemID = d1;
                    task.DatabaseName = str4;
                    task.InstanceName = str5;
                    arrayLists.Add(task);
                }
            }
            return arrayLists.ToArray(typeof(Task)) as Task[];
        }

        protected Task CreateTask(string taskType, DateTime taskDate)
        {
            if (taskDate == SqlDateTime.MinValue.Value)
            {
                taskDate = DateTime.MinValue;
            }
            taskDate = DateUtil.ToUniversalTime(taskDate);
            object[] objArray = new object[] { taskDate };
            return ReflectionUtil.CreateObject(taskType, objArray) as Task;
        }

        /// <summary>
        /// Logs the info.
        /// </summary>
        /// <param name="message">The message.</param>
        private void LogInfo(string message)
        {
            if (this.LogActivity)
            {
                Log.Info(message, this);
            }
        }
    }
}
You'll also need to replace the task database agent to use the one above:
<?xml version="1.0" encoding="utf-8" ?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/" xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <sitecore>
    <scheduling>
      <agent type="Sitecore.Tasks.TaskDatabaseAgent" method="Run" interval="00:10:00">
        <patch:attribute name="type" xdt:Transform="Replace" xdt:Locator="Match(type)">Myproject.Override.TaskDatabaseAgent</patch:attribute>
      </agent>
    </scheduling>
  </sitecore>
</configuration>
As with any overrides or replacement of Sitecore code, you will need to check if the code is updated upon installations of CMS updates and update your code accordingly. This code was taken from version8 update 5.