Tuesday, April 26, 2016

Sitecore Advanced Sitemap Module Updated

Mohammed Syam has developed a great module for Sitecore called Advanced Sitemap Module. I have used this on a number of implementations, and it works well. However with Sitecore 8.1 the following error might occur when installing the package:
Empty strings are not allowed. Parameter name: itemName
This appears to be due to a corrupt package (or one not supported by Sitecore 8.1 +). So I recreated the items to create a fresh package that will install on Sitecore 8.1.

Code was updated from a fork of the original by Patrick Stysiak. Here are a couple of notes:
  1. Fixes have been made to work on Content Delivery servers (no master connection)
  2. Error with info logger in one case removed
  3. Icons may be different from the original
  4. GUIDs have changed from the original (in terms of template IDs and SiteMap configuration content item).
  5. Field changed from Show In SiteMap to Show In XML SiteMap
  6. Code will no longer error when root site item has descendants which don't include the sitemap base template. This was P => P.Fields["Show In XML SiteMap"].Value == "1" code. We now check for items which have the field and the field value is set to "1". P => P.Fields["Show In XML SiteMap"] != null && P.Fields["Show In XML SiteMap"].Value == "1"
  7. Fixed config bug where Sitemap would not generate unless MultilingualSiteMapXML was set to true 

The Github source for the code is available at: Sitecore Advanced SiteMap module. - This is not required as the package has the DLL included.

The updated package is available here.

Notes

  • ERROR Advanced SiteMap config item was not found: This is because the SiteMap config item (ID {38ACC950-E87F-4A5C-9271-C55C4336AAAB}) could not be found in the web database
  • The Sitemap Site template has a field called Site Name, this should refer to a site defined in the sites node of Sitecore.config
  • This module uses the web database, so ensure a publish will be needed for changes to appear in the Sitemap

34 comments:

  1. Hi Ryan,
    I just installed this module in Sitecore 8.0 Update 4, as the 1.0 version from the Marketplace kept failing upon installation.

    Your updated version installed correctly, but I do not see any configuration items under sitecore/system/modules ? Am I missing something? How do I configure this to work in a multisite environment?

    Thanks
    Evan

    ReplyDelete
    Replies
    1. Hi Evan, previously the configuration was being installed under the default home node. I have since moved this under sitecore/system/module and have updated the package.

      With a multi site implementation, are you expecting a different sitemap for each site, or one single sitemap?

      Delete
    2. Thanks Ryan, I've installed your updated package and now see the configuration items in sitecore/system/module.

      Yes, we would like a different Sitemap for each site.

      Based on this http://sitecoreinfo.blogspot.com/2015/10/sitecore-sitemap-part-2.html, it appears you removed the 'Search Engines' item that allows configuration to auto send your Sitemap to Google, etc. Can you please confirm this?

      Also, I have a few more questions:

      1. I configured a Sitemap site in the system/modules section, with the Site Name matching my site definition, and the SiteMap XML Filename a custom name for one site (ex. SiteMapSite1.xml). I have published out the configuration items, along with the Site items again, and do not see any new generated sitemap called 'SiteMapSite1.xml'. How do I complete the configuration?
      2. Per the 'SiteMap Config' item, you set your Target Database Name. If we have multiple web databases, can I duplicate the 'SiteMap Config' item and have multiple Target Database Names?
      3. If we have separate CD servers, do we need to enable the Sitecore interface on these servers and install the package again?

      Thanks for the info!
      Evan

      Delete
    3. Ryan,
      Are you able to provide any path forward on how to activate this module? (per my questions above) It seems our one sitemap is being delivered for all sites in our environment, which is terrible for SEO except for the single site we use it for.
      Thanks!

      Delete
    4. Hi Evan,

      I'll take a look and re-add the search engine submission template, it might have been missed from the package.

      1. The sitemap will not generate if there are no items (published to web database) which inherit from the base sitemap template and have "include in XML sitemap" checked.

      2. Unfortunately the code refers to the SiteMap config item (statically via it's ID), so any additional versions would be ignored. You would need to download the source and modify for this purpose.

      3. You will simply need the advanced sitemap DLL and Sitecore.AdvancedSiteMap.config file on the CD servers. The events in that config would pickup publishes and generate the XML sitemap.

      For XML sitemap the module is designed to create a file for each site. Each domain will resolve the XML files in the website root as well.

      The HTML sitemap builder appears to get the parent of the item which calls it and then puts all descendants of that item in the sitemap. I would think that if you had the sitemap page under the root node of each multi-site site, then the HTML sitemaps should generate correctly. Can you confirm?

      - Content
      --Site A
      ---Sitemap Page
      --Site B
      ---Sitemap Page

      Delete
    5. Hi Ryan,
      1. I found a page template for one of our sites and set it to inherit from the base sitemap template. I then 'included in XML sitemap' on the content item itself, and published the item to the web database, but still do not see any new sitemap. I also tried manually generating the Sitemap and that had no effect either. Is this compatible with Sitecore 8.0 Update 4?

      2. That makes sense. We will adjust accordingly.

      3. Good to know. I'll put the DLL and config on the CD servers in our production environment.

      I can test the HTML sitemap once the XML sitemap generates. We don't really have much use for the HTML sitemap, but I'm curious.

      Thanks for your help!
      Evan

      Delete
    6. Hi Evan,

      Can you try setting a minimum of 2 pages to 'include in XML sitemap' as I have noticed it does not work with just one.

      Delete
    7. Ryan,
      After setting 2 content pages to include in XML sitemap, the sitemap generates!

      On a separate note, there were 2 changes I had to make to get the file generation to work, and 2 bug fix requests I have for you.

      Changes I made:
      1. When you moved the configuration items to System/Modules from Content/Home, I believe you forgot to change the xPath queries on the templates for Priority and Frequencies. (they were still pointing to Content/Home)
      2. When the sitemap.xml generated, the frequency and priority did not seem to pull through. I fixed this by modifying the SiteMap Base template to make the Frequency and Priority droplink fields, instead of droplist type.

      Bugs I have encountered:
      1. the Sitemap reference in the robots.txt gets generated, however it does not have the full sitemap path. (i.e. I see "Sitemap: sitemap.xml" not "Sitemap: http://www.mywebsite.com/sitemap.xml" I believe the full reference is needed for a multisite solution.
      2. In the sitemap file itself, the URL I configured in the Sitemap Site config seems to append itself to the hostname of the Sitecore server (i.e. "http://www.mysite.com//http://myservername/en/sitecore/content/mypagename"

      I can send screenshots and/or create issues in your Github repository if you'd like.

      Hopefully this makes sense!
      Let me know!
      Thanks,
      Evan

      Delete
    8. Hi Evan,

      Thanks for sending through the bugs above. The source code/package has been updated:

      1. the xPath has been updated in the package.
      2. Droplink has now replaced droplist

      Bugs:
      1. robots.txt will now include the server URL - note that this simply appends, so you may need to clean the incorrect items out
      2. The URLs should now generate correctly when an over ride is setup - as well as with the robots.txt

      Thanks,

      Ryan

      Delete
    9. Thanks Ryan. Looks like those bugs are resolved now.
      However, I did notice a few related bugs. Please let me know if it's simply my configuration or user error.

      1. When manually generating the Sitemap, the sitemap URLs seem to duplicate themselves similar to the previous bug (http://www.mysite.com//http://www.mysite.com/en/sitecore/content/mypagename)

      2. It seems the auto-update sitemap function (upon publish) only updates the first sitemap of many. I created another site config, and set a couple content items in that site to be included in a 2nd sitemap, but that sitemap never gets created or updated.

      Please let me know if you witness the same issues.

      Thanks
      Evan

      Delete
    10. For further information, I just re-ordered my sitemap configs in the content editor and got the other Sitemap to update.

      Delete
    11. Hi Evan,

      Has the latest advanced sitemap DLL from the package linked in this post been applied to all servers?

      1. Is this with an override URL set? I've made a minor change to how URLs are generated (source updated along with package above). Please let me know if this works.

      2. The manual generation calls the same method as the automatic generation. This will query the web database and find the sitemap config item (item ID {38ACC950-E87F-4A5C-9271-C55C4336AAAB}) and then generate the XML sitemap for all descendants of sitemap site (template ID {9F454752-3C73-45F0-A03A-1B8EFD9F7F9E}). The site name field needs to be present and needs to match the site name defined against the site in sites element in Sitecore.config. Each should also have a unique XML file name.

      Thanks,

      Ryan

      Delete
    12. Yes, the latest package has been deployed to our development servers.

      1. I've deployed the updated package, but still witness the same issue upon sitemap generation. I even deleted the sitemap entirely and recreated it manually and by publish. The format is still 'http://www.mysite.comhttp://www.mitesite.com/en/pagename'. It does pull my Server URL, it just seems to duplicate it.

      2. This also still seems to be a bug. A publish only seems to auto update the sitemap of the first defined sitemap config item. I have confirmed the Site Name and server URL are correct, and match the site definition in the web.config.
      I can reorder the SiteMap site configs and get the first site listed to generate a sitemap upon publish. For example, say you have SiteMap Config > Site1, Site2. Publish generates sitemapSite1.xml. If I reorder to SiteMap Config > Site2, Site1, the publish generates sitemapSite2.xml only. It seems like the code isn't looping through each site config item.

      Thoughts?

      Thanks!
      Evan

      2.

      Delete
    13. 1. In your sitecore config file for the sites element for this site, is the hostName and scheme attributes set? If so what are they? Also has rootPath and startItem been set correctly?

      2. I'll debug through this case on my own multi-site setup and get back to you.

      Thanks,

      Ryan

      Delete
    14. 1. The hostname is set, but I don't see the scheme attribute. We don't set the scheme for any of our sites.. Our hostname for 1 site is 'lab.terathane.invista.com' (It is not publicly accessible) The rootPath and startItem have also been set correctly.

      2. Thanks, let me know!

      Delete
    15. 1. I'm wondering if you have the latest release linked here (I have done 3 or 4 in the course of these conversations), as that issue appears to be fixed. When a server override is not set the code uses LinkManager.GetItemUrl(item, options); to get the item URL. options.AlwaysIncludeServerUrl will be set to true in this case which will allow LinkManager to get the correct hostname based on the path of the item.

      I would recommend setting the scheme element on the site definition to "http" as this might be what is causing the issue. I have just re-released the module (linked above) as I noticed another bug where sites with a scheme of "https" would get "http://" added to the beginning of the URL.

      An example would be:

      2. I am unable to reproduce this error. Please see a PDF here: http://file.ryanbailey.co.nz/module/sitemap-multiple-site.pdf which details the settings I had in my environment.

      Delete
    16. Just to be sure, I downloaded and installed the latest package again.

      1. I'm still witnessing duplicate host names in the sitemap (http://http://mysite.com://mysite.com/en)

      2. For some reason, in our environment a publish still only generates a sitemap for the first site config listed.

      I should mention we use a custom LinkManager for every one of our sites, as our setting requirements such as the AlwaysIncludeServerUrl are different from site to site, so it's certainly possible our custom LinkManager settings are conflicting with the code.
      I will review with my team internally, but thank you very much for your help and response!

      Delete
    17. Hi Evan,

      In case you are still seeing duplicated URLs, I came across this problem when implementing another sitemap module: http://blog.ryanbailey.co.nz/2016/06/sitecore-linkmanagergetitemurl.html

      It occurs when targetHostName is set on the site definition. If you can confirm you have this attribute set in your environment, I'll make an updated release that handles this.

      Thanks,

      Ryan

      Delete
  2. Hey Ryan,
    Based on the comments at https://marketplace.sitecore.net/en/Modules/S/Sitecore_Advanced_SiteMap_Module.aspx it looks like me and EVAN HOFFMAN are having the same issue.

    "Mohammad/Ryan,
    I have installed the 1.1 version from Ryan for use in our Sitecore 8.0 environment. The install worked correctly, but I do not see any related items under sitecore/system/modules...how do I configure this module to work in a multisite environment?
    THanks"

    Any Idea what could be going on here?

    Luke

    ReplyDelete
    Replies
    1. It was being installed under the home node and has been since changed to sitecore/system/modules

      Delete
  3. Hi Ryan,
    I have installed sitemap advanced module. I am able to install it successfully. how to use the module in content tree and able to generate the sitemap xml. Please help me on this. thanks

    ReplyDelete
    Replies
    1. Did you add the base Sitemap template to the standard values for your custom page template? I have also noticed that you need to select at least 2 pages to add to the sitemap (on the custom page items) for it to generate successfully.

      You can also check the logs to see if there were permission errors in creating the sitemap XML file.

      Delete
    2. Hi Ryan,

      Thanks for your quick reply. Is there any demo/link for handling sitemap template to the standard values of custom page template. could you please help me on this.

      Delete
    3. since i have searched for the documentation to use sitemap advanced sitemap moduel. I don't find enough documentation.
      can you please send me any documenation to use.

      Delete
  4. I have added base sitemap template to the custom page template and i tried publishing individual items and republishing all items. Also i have checked i don't find any permission issue in logs. But still sitemap xml is not generated. Also I manually clicked GenerateSiteMap button under Advance Sitemap tab. I don't find that button is working or not bcoz nothing is happening when i click the GenerateSitemap button. could you please help on this

    ReplyDelete
    Replies
    1. Do at least two of the pages have the checkbox selected to be included in XML Sitemap?

      Delete
  5. I am able to generate the sitemap xml file. please find one of the output below

    -

    http://domain.com/en/about
    2016-09-08T14:53:27+05:30
    yearly
    0.0



    In the above output url: Language 'en' gets embedded:
    we don't want language to be embedded in the URL(http://domain.com/en/about) . It should be (http://domain.com/about)

    To fix this issue - we have done below steps in sitecore.config file
    but it is still not working








    But still we are facing the issue. we need your help on this.

    ReplyDelete
    Replies
    1. Have you changed this setting: http://blog.ryanbailey.co.nz/2015/05/sitecore-remove-en-from-urls.html

      Delete
  6. Hi Ryan,

    As I am facing "http://" is getting appended to the below URL : http://https://auth1.domain.com/about. . Could you help me on this fix the issue asap. As we need to move to the production.

    thanks
    arun

    ReplyDelete
    Replies
    1. Hi Arun,

      In your sitecore config file for the sites element for this site, is the hostName and scheme attributes set?

      The scheme attribute would be "http" or "https", if not set I have seen your issue happen before.

      Delete
  7. how to prevent our website data from crawlers or search engines. please suggest us. as we added below text in user-agent: * Disallow: / in robots.txt file. But no hopes

    ReplyDelete
    Replies
    1. Search engines such as Google will follow the robots.txt file, however other sources may choose to ignore it. You may have to write custom code to detect them.

      Delete
  8. Is that sitemap advanced module automatically submit to the search engines?.

    If so how to prevent the same. Because our requirement is we need the module to be automatically submit to the search engines on CD server and not CM Server.

    ReplyDelete
    Replies
    1. This would require some customization. Is your CM server using the different host name than the CD server(s)?

      Delete