Tuesday, July 10, 2018

Sitecore Experience Commerce - Management.GetCommerceTerms and related methods in logs

Looking at the logs for a long running Sitecore Experience Commerce minion, I noticed large chunks of the following logs:
INFO Management.GetCommerceTerms./sitecore/Commerce/Commerce Control Panel/Commerce Engine Settings/Commerce Terms/BusinessTools/ViewPropertyNames|en
INFO Management.block.getitemsbypath./sitecore/Commerce/Commerce Control Panel/Commerce Engine Settings/Commerce Terms/BusinessTools/ViewPropertyNames|en
INFO Management.block.getitembyid.7f475f9d-341e-4a21-bec2-0f5f74987a3b: Language=en
It turns out that my minions environment (and the Habitat example) did not enable caching for queries related to management. For example the OOTB Habitat minions experience commerce environment (defined in PlugIn.Habitat.CommerceMinions-1.0.0.json) references a PolicySetId of Entity-PolicySet-MinionsMemoryCachePolicySet which in turn is defined in PlugIn.MinionsMemoryCache.PolicySet-1.0.0.json.

To allow caching of these management related queries, the following can be added to the PlugIn.MinionsMemoryCache.PolicySet-1.0.0.json file (or your own custom version).
{
  "@odata.type": "Sitecore.Commerce.Plugin.Management.ManagementCachePolicy, Sitecore.Commerce.Plugin.Management",
  "AllowCaching": true,
  "Expiration": 3600000
},
It may appear more cut-down than other caching examples, but worked in my environment.

Extract of a caching policy for a Sitecore Experience Commerce engine environment

Monday, July 9, 2018

Sitecore SOLR - unknown field MyFieldName

After adding a new computed index field to my SOLR web index, I was getting the following error on rebuild:
Job started: Index_Update_IndexName=sitecore_web_index|#Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> SolrNet.Exceptions.SolrConnectionException: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">400</int><int name="QTime">1</int></lst><lst name="error"><lst name="metadata"><str name="error-class">org.apache.solr.common.SolrException</str><str name="root-error-class">org.apache.solr.common.SolrException</str></lst><str name="msg">ERROR: [doc=sitecore://web/{57606db8-1237-463b-a934-01f3854a2909}?lang=en&amp;ver=0&amp;ndx=sitecore_web_index] unknown field 'MyFieldName'</str><int name="code">400</int></lst>
</response>
 ---> System.Net.WebException: The remote server returned an error: (400) Bad Request.
This is due to the fact that when I added the field to the index, the returnType was set as:
returnType="System.Double"
Where it should have been set as:
returnType="double"
This data return type is mapped on the SOLR field map:
Sitecore SOLR Field Map

Wednesday, July 4, 2018

Sitecore Experience Commerce - ERROR Site Disabled. Failed to logging

In a Sitecore Experience Commerce environment running the minions role, the following set of errors would occur when attempting to access an item in Sitecore.
17 19:13:17 INFO Management.block.getitembyid.0F65742E-317F-44B0-A4DE-EBF06209E8EE: Language=en
17 19:13:17 ERROR SitecoreConnectionManager: RETRY 1 out of 3 with ERROR Site Disabled. Failed to logging. /sitecore/api/ssc/auth/login for GET /sitecore/api/ssc/item/0F65742E-317F-44B0-A4DE-EBF06209E8EE?database=master&language=en
17 19:13:17 ERROR SitecoreConnectionManager: RETRY 2 out of 3 with ERROR Site Disabled. Failed to logging. /sitecore/api/ssc/auth/login for GET /sitecore/api/ssc/item/0F65742E-317F-44B0-A4DE-EBF06209E8EE?database=master&language=en
17 19:13:17 ERROR SitecoreConnectionManager: RETRY 3 out of 3 with ERROR Site Disabled. Failed to logging. /sitecore/api/ssc/auth/login for GET /sitecore/api/ssc/item/0F65742E-317F-44B0-A4DE-EBF06209E8EE?database=master&language=en
17 19:13:17 ERROR SitecoreConnectionManager.Error: Message=RETRY FAILED for GET /sitecore/api/ssc/item/0F65742E-317F-44B0-A4DE-EBF06209E8EE?database=master&language=en|Trace=   at Sitecore.Commerce.Plugin.Management.SitecoreConnectionManager.ProcessRequest(CommerceContext commerceContext, String action, String method, ItemModel itemModel)
System.Exception: RETRY FAILED for GET /sitecore/api/ssc/item/0F65742E-317F-44B0-A4DE-EBF06209E8EE?database=master&language=en
   at Sitecore.Commerce.Plugin.Management.SitecoreConnectionManager.ProcessRequest(CommerceContext commerceContext, String action, String method, ItemModel itemModel)
17 19:13:17 ERROR Management.block.getitembyid: Sitecore Item Service Get item failed, Item 0F65742E-317F-44B0-A4DE-EBF06209E8EE not found.
I had checked the configuration file along with the content set policy (PlugIn.Content.PolicySet-1.0.0.json) and all connection details and user accounts were correct for the Sitecore instance.\\

It turned out that this error was occurring because the minion experience commerce engine role was blocked from accessing the Sitecore content management instance due to an IP white listing rule.

Tuesday, July 3, 2018

Sitecore Experience Commerce - Facets are limited to 100 results

I was debugging a Sitecore Experience Commerce product list locally (using SOLR as a search provider) and noticed that a category with a large number of brands to facet on, would always limit out at 100 results. This was not ideal as that 100 would include facets with 0 aggregate count (matches for a given search) which meant actual brands that were in the results would not appear as a valid facet.

This turned out to be a default setting with SOLR whereby the maximum number of facets to return for a given query was 100. This can be increased to a valid number as required or to unlimited with a setting of -1.

To increase the number of facets to return with a SOLR index in Sitecore, you will need to do the following.

  1. In the SOLR directory locate the index you wish to target.
    1. In my case this was the web index - which is located in C:\solr\server\solr\sc_web_index
  2. Inside the conf child director is a solrconfig.xml configuration file.
  3. The facet.limit setting needs to be edited - in my case this setting was not present
    1. A valid number to set a hard limit of facets to return or -1 for unlimited.
  4. Add or edit the setting in the following location.
  5. Restart the SOLR service.
<requestHandler name="/select" class="solr.SearchHandler">
<!-- default values for query parameters can be specified, these
  will be overridden by parameters in the request
  -->
<lst name="defaults">
  <str name="echoParams">explicit</str>
  <int name="rows">10</int>
  <str name="facet.limit">-1</str>

Note: the snippet has been cut-down, however there is enough here to see where to add the setting.

After making this change, I was now seeing all of the brand facet data that I was expecting.

Friday, June 22, 2018

Sitecore Experience Commerce - Remove Commerce Experience Accelerator JavaScript

Out of the box, if you have a site running Sitecore Experience Commerce (even with a completely custom theme) some JavaScript will get added to the page. This happens if you are using AssetLinksGenerator.GenerateLinks to give you the required CSS and JS to output on the page.

Theses scripts (as on update 1) appear as:
<script src="/Scripts/Commerce/Services/cxa.services.dom.observer.js"></script>
<script src="/Scripts/Commerce/Services/cxa.services.messaging.js"></script>
<script src="/Scripts/Commerce/Services/cxa.services.ajax.js"></script>
<script src="/Scripts/Commerce/Services/cxa.common.productselection.js"></script>
<script src="/Scripts/Commerce/Services/cxa.common.productprice.js"></script>
<script src="/Scripts/Commerce/Services/cxa.services.cart.context.js"></script>
<script src="/Scripts/Commerce/Core/cxa.core.application.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.productlistsorting.js"></script>
<script src="/Scripts/Commerce/Core/cxa.core.component.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.orders.model.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.models.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.productlistitemsperpage.js"></script>
<script src="/Scripts/Commerce/Core/cxa.core.startup.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.recent-orders.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.order-history.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.addtocart.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.address.model.js"></script>
<script src="/Scripts/Commerce/Core/cxa.core.form.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.addtocart.productselection.handler.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.delivery.model.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.lines.model.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productprice.model.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.billing.model.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.lines.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productprice.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.review.model.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.total.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.delivery.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.total.model.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productinventory.model.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.billing.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.promotion.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productinventory.js"></script>
<script src="/Scripts/Commerce/Feature/Checkout/cxa.feature.checkout.review.js"></script>
<script src="/Scripts/Commerce/Feature/Cart/cxa.feature.cart.promotion.model.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productvariant.model.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.catalog.productvariant.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.productfacets.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.productlist.js"></script>
<script src="/Scripts/Commerce/Feature/Minicart/cxa.feature.minicart.model.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.productlist.model.js"></script>
<script src="/Scripts/Commerce/Feature/Minicart/cxa.feature.minicart.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.promotedproducts.js"></script>
<script src="/Scripts/Commerce/Feature/Catalog/cxa.feature.promotedproducts.model.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.registration.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.login.js"></script>
<script src="/Scripts/Commerce/Feature/LanguageSelection/cxa.feature.language.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.forgotpassword.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.changepassword.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.profileeditor.js"></script>
<script src="/Scripts/Commerce/Feature/MessageSummary/cxa.feature.messageSummary.model.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.address.list.js"></script>
<script src="/Scripts/Commerce/Feature/MessageSummary/cxa.feature.messageSummary.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.address.list.model.js"></script>
<script src="/Scripts/Commerce/Feature/Navigation/cxa.feature.navigationbar.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.address.editor.js"></script>
<script src="/Scripts/Commerce/Feature/Account/cxa.feature.address.editor.model.js"></script>
If you don't require any of this JavaScript, they can be stopped from being output by patching out the logic with the following configuration patch file.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <pipelines>
            <assetService>
                <processor type="Sitecore.Commerce.XA.Foundation.Common.Pipelines.AssetService.AddJavaScript, Sitecore.Commerce.XA.Foundation.Common">
                    <patch:delete />
                </processor>
            </assetService>
        </pipelines>
    </sitecore>
</configuration>
Appears to save quite a few requests, which should speed up page load times as well.

Tuesday, June 19, 2018

Sitecore Experience Commerce - Catalog Item of the Storefront catalog Configuration cannot be found

When viewing a Sitecore Experience Commerce shop on the Storefront (content delivery node). The following error was appearing:
Catalog Item of the Storefront catalog Configuration cannot be found.
The code appeared to be breaking when selecting a catalog node which appeared under the Catalogs item in the tree (/sitecore/Commerce/Catalog Management/Catalogs). I confirmed this item present in the web/master databases as well as in the relevant search index.

This issue was resolved by enabling the Sitecore.Commerce.Engine.DataProvider.config configuration file in the App_Config/Include/Y.Commerce.Engine folder. In our case this did not appear to be enabled in Azure PAAS with the ARM templates.

Sitecore Experience Commerce - Queries to debug whats in a search index

When debugging an issue with the front-end (storefront) site in a Sitecore Experience Commerce instance, it often becomes necessary to see what is in the search index. As an example I have had to check if a given category is appearing along with various sellable items which should appear inside it.

There is a difference on the query if you are using SOLR or Azure Search and I have provided both examples.

Index structure

It's worth noting that in an experience commerce instance there are several indexes which may need to be investigated and they each server as different sources of truth.
  • CatalogItemsScope - this index is used by the commerce engine role(s) along with the business tools for commerce.
  • mysite_web_index - used by the management (CM) servers.
  • fsni_master_index - used by the delivery servers (CD) to drive the front-end (storefront).
Note that index names may slightly vary based on your implementation.


To query SOLR

To query a SOLR index, you can open up the web administration, select the relevant index (purple box below), submit your query (red box below) and see if there are results (blue box below).

Querying commerce data in SOLR

To query Azure Search

To query an Azure Search index, you simply open the index in the Azure portal and use the search explorer to submit a query (red box below) and view any results (blue box below).

Querying commerce data in Azure Search

Categories

The following queries can be used to query a given category and the category ID can be obtained from the business tools.

Category ID visible in the business tools
In CatalogItemsScope - commerce engine index
Azure Search
"Entity-Category-MyCatalog-Bread Rolls  Buns"&searchFields=entityid
SOLR
entityid:"Entity-Category-MyCatalog-Bread Rolls  Buns"
In web/master indexes- Sitecore website
Azure Search
"Entity-Category-MyCatalog-Bread Rolls  Buns"&searchFields=catalogentityid
SOLR
catalogentityid_t:"Entity-Category-MyCatalog-Bread Rolls  Buns"

By commerce entity type

When debugging what commerce data is actually in the index, it's often handy to search based on the commerce entity type. Examples of types which I query on are:
  • SellableItem
  • Category
  • Catalog
In web/master indexes- Sitecore website
Azure Search
"SellableItem"&searchFields=commercesearchitemtype
SOLR
commercesearchitemtype_t:"SellableItem"

Conclusion

The above post gives some details on which search index you would want to query to debug a given issue along with some query samples for both Azure Search and SOLR. These queries can then be modified to query other fields (such as the Sitecore ID visible in the content editor interface. 

Wednesday, June 13, 2018

Sitecore Experience Commerce - Commerce service Error code Forbidden

On the content management server of a Sitecore enthronement containing Experience Commerce 9 (update 1), the following was appearing in the logs when attempting to refresh the commerce cache:
An error occured while trying to contact the Commerce Service. Error code Forbidden
I tried two things and one of which (or the combination) was able to resolve this error:
  1. In the show config page (site/admin/showconfig.aspx) the configuration where the URLs for the shops/ops service had line spacing. I moved these onto one line.
  2. On the commerce engine shops environment I opened the config.json file and added the URL for the content management server to the AllowedOrigins array.
Please leave a comment below, if you have found other causes for this error.

Sitecore - the controller for path was not found or does not implement IController

When attempting to load a Sitecore page via a content delivery server, the following error occurred:
Server Error in '/' Application.
The controller for path '[PageUrl]' was not found or does not implement IController.
Description: An unhandled exception occurred.
Exception Details: Sitecore.Mvc.Diagnostics.ExceptionWrapper: The controller for path '[PageUrl]' was not found or does not implement 
The cause of the error was a rendering on the page having an incorrect controller set. Fixing this was able to allow the page to load as expected. 

Saturday, June 2, 2018

Sitecore - errors occurred while attempting to load the app

When attempting to load a Sitecore 9 instance the following error would show:
The following errors occurred while attempting to load the app.
- No assembly found containing an OwinStartupAttribute.
- The discovered startup type 'Sitecore.Owin.Startup, Sitecore.Owin, Version=1.1.0.0, Culture=neutral, PublicKeyToken=null' conflicts with the type 'SolrNet.Startup, SolrNet, Version=0.4.0.2002, Culture=neutral, PublicKeyToken=null'. Remove or rename one of the types, or reference the desired type directly.
To disable OWIN startup discovery, add the appSetting owin:AutomaticAppStartup with a value of "false" in your web.config.
To specify the OWIN startup Assembly, Class, or Method, add the appSetting owin:AppStartup with the fully qualified startup class or configuration method name in your web.config.
Sitecore startup error with OWN and SOLR
This was caused by a rogue web project deploying an empty web.config and overwriting the Sitecore default. By replacing the web.config file with the correct version, the error stopped occurring.

Tuesday, May 29, 2018

Sitecore and Azure Search - Conflict between type of incomming field and field in schema

When attempting to rebuild the web/master indexes for a Sitecore 9 instance using Experience Commerce and Azure Search as the provider, the logs were filled with the following fatal errors:
FATAL Could not add field {B775BCA9-2605-4D60-B367-A0C354BE2504} : default value for indexable sitecore://master/{2F5990B9-E6F2-4D2E-9179-7851789EB66C}?lang=da&ver=1Exception: System.ApplicationExceptionMessage: Conflict between type of incomming field 'Edm.String' and field in schema 'Edm.Boolean'Source: Sitecore.ContentSearch.Azure at Sitecore.ContentSearch.Azure.Schema.CloudSearchIndexSchemaBuilder.AddFields(IndexedField[] fileds) at Sitecore.ContentSearch.Azure.Schema.CloudSearchIndexSchemaBuilder.AddField(String fieldName, Object fieldValue, CloudSearchFieldConfiguration configuration, CultureInfo culture)   at Sitecore.ContentSearch.Azure.CloudSearchDocumentBuilder.AddField(String cloudName, Object fieldValue, CloudSearchFieldConfiguration cloudConfiguration, Boolean append, Boolean fieldIsEmpty)   at Sitecore.ContentSearch.Azure.CloudSearchDocumentBuilder.AddField(String fieldName, Object fieldValue, CloudSearchFieldConfiguration cloudConfiguration, Boolean append) at Sitecore.ContentSearch.Azure.CloudSearchDocumentBuilder.AddField(IIndexableDataField field)   at Sitecore.ContentSearch.AbstractDocumentBuilder`1.CheckAndAddField(IIndexable indexable, IIndexableDataField field)
Strangely enough, this bug was supposedly fixed in version 8.2 of Sitecore Experience Platform. However it's likely the addition of new fields in later version that were not fixed up have caused it to come back.

The issue boils down to the fact that Azure search will store a type against a given field name. So in the case above, the field {B775BCA9-2605-4D60-B367-A0C354BE2504} is called default value and is of type single-line text (string). A quick search for other fields called default value, leads to 3 others, including one of type checkbox (boolean) and another of type integer.

That's where the issue comes in, the first occurrence of the field name will set the type in stone. So the only fix (if you wish to stop these errors) is to exclude the "duplicate fields" from the index.

Below is a patch file which has cleaned up many of these errors for me, I have not found any adverse affects (as these are often irrelevant system fields as opposed to rich text content).
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
<sitecore role:require="Standalone or ContentManagement" search:require="Azure">
    <contentSearch>
      <configuration>
        <indexes>
          <index id="sitecore_master_index">
            <configuration>
              <documentOptions ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/documentOptions">
                <exclude hint="list:AddExcludedField">
                  <NameConflictA tag="{EA2A0983-36EA-4957-B613-6F23EA5CDAD4}">{EA2A0983-36EA-4957-B613-6F23EA5CDAD4}</NameConflictA>
      <NameConflictG tag="{3F67BE6A-F3F0-41C5-B379-C6AF29685666}">{3F67BE6A-F3F0-41C5-B379-C6AF29685666}</NameConflictG>      
      <NameConflict1 tag="{FF1383BB-F095-4958-9B6D-E555DB653C44}">{FF1383BB-F095-4958-9B6D-E555DB653C44}</NameConflict1>
      <NameConflict2 tag="{D660A7A4-3F42-4953-A13F-CFCA850BDFEA}">{D660A7A4-3F42-4953-A13F-CFCA850BDFEA}</NameConflict2>
      <NameConflict3 tag="{7DC7CD49-9B23-4D44-B21E-FA5443224A50}">{7DC7CD49-9B23-4D44-B21E-FA5443224A50}</NameConflict3>
      <NameConflict4 tag="{B775BCA9-2605-4D60-B367-A0C354BE2504}">{B775BCA9-2605-4D60-B367-A0C354BE2504}</NameConflict4>
      <NameConflict5 tag="{E43E4E5F-6B45-4E3B-8D85-85AB87D2CC12}">{E43E4E5F-6B45-4E3B-8D85-85AB87D2CC12}</NameConflict5>
      <NameConflict6 tag="{CEDA3E7F-7406-40D0-A154-7CF7E3E7E85B}">{CEDA3E7F-7406-40D0-A154-7CF7E3E7E85B}</NameConflict6>
      <NameConflict7 tag="{4F17697F-BD00-4D76-B4F5-1B53C0C8D884}">{4F17697F-BD00-4D76-B4F5-1B53C0C8D884}</NameConflict7>
      <NameConflict8 tag="{BEFC61BF-8CFD-4478-9F5C-8422E6E2F005}">{BEFC61BF-8CFD-4478-9F5C-8422E6E2F005}</NameConflict8>
      <NameConflict9 tag="{4F892959-FDE6-4020-8569-2CF934BA412C}">{4F892959-FDE6-4020-8569-2CF934BA412C}</NameConflict9>
      <NameConflict10 tag="{85E2D73B-F11A-4BD7-9826-86BB191925C5}">{85E2D73B-F11A-4BD7-9826-86BB191925C5}</NameConflict10>
      <NameConflict11 tag="{EA45EDF7-CD8B-433F-99A6-EBFE3C7CE3A5}">{EA45EDF7-CD8B-433F-99A6-EBFE3C7CE3A5}</NameConflict11>
      <NameConflict14 tag="{1B9D1028-C20C-407B-9DCD-AFFE86A6F793}">{1B9D1028-C20C-407B-9DCD-AFFE86A6F793}</NameConflict14>
      <NameConflict15 tag="{09147FB2-EBFB-4949-8C8E-26A424409D5E}">{09147FB2-EBFB-4949-8C8E-26A424409D5E}</NameConflict15>
      <NameConflict16 tag="{3DD612C4-3AA7-48A9-9F46-FD8DE9AE3ACA}">{3DD612C4-3AA7-48A9-9F46-FD8DE9AE3ACA}</NameConflict16>
      <NameConflict17 tag="{F917C951-1F75-4D62-B14A-BC6888D7EECA}">{F917C951-1F75-4D62-B14A-BC6888D7EECA}</NameConflict17>
      <NameConflict18 tag="{290DEFD0-7B9A-4541-A326-87C44BEE5309}">{290DEFD0-7B9A-4541-A326-87C44BEE5309}</NameConflict18>
      <NameConflict19 tag="{59B79BAF-4F30-4ECA-961D-4AFAB14D157C}">{59B79BAF-4F30-4ECA-961D-4AFAB14D157C}</NameConflict19>
      <NameConflict20 tag="{9EDD2FCC-04AE-4230-84B4-C46C963F2284}">{9EDD2FCC-04AE-4230-84B4-C46C963F2284}</NameConflict20>
                </exclude>
              </documentOptions>
            </configuration>
          </index>
        </indexes>
      </configuration>
    </contentSearch>
  </sitecore>
</configuration>
Ignore the mismatched field names, this was after some cleanup.

Sitecore and Azure Search - An index can have at most 1000 fields

When attempting to run a rebuild of the master and web indexes in a PAAS environment of Sitecore 9 using Azure search, as the search was nearing the end, the following error would stop the index in it's tracks.
Exception: Sitecore.ContentSearch.Azure.Http.Exceptions.AzureSearchServiceRESTCallExceptionMessage: {"error":{"code":"","message":"The request is invalid. Details: definition : Invalid index: The index contains 1087 field(s). An index can have at most 1000 fields.\r\n"}}
As it turns out, Azure Search can only index a maximum of 1000 unique field names. Annoyingly enough, the OOTB Sitecore 9 with Experience Commerce appears to actually go over this limit - hence the error.

There are a couple of configuration patch files that come OOTB that attempt to cull down the number of fields by excluding a whole lot of them:

  1. App_Config/Include/Sitecore.XA.Foundation.Search.Azure.Index.Master.ExcludeFields.config
  2. App_Config/Sitecore/ContentSearch.Azure/Sitecore.ContentSearch.Azure.Index.Master.ExcludeFields.config
Which is a start, but it means the next step is for you to create your own version of one of these field exclude configs and start adding your custom fields (and any others you can find for that matter). In my case it took a number of deploys, because as the index was able to progress further the error kept rearing it's ugly head. Ultimately I settled on 200 odd excludes, which were made up with a combination of my custom fields and OOTB Sitecore fields.

A handy tip is to keep the Sitecore show configuration page open, and before you exclude a field, do a quick search on that page to ensure it has not already been excluded.

This isn't an elegant solution, but I will update this page if a patch is provided.

Sitecore and Azure Search - contains a term that is too large to process

Similar to the PostFailedForSomeDocumentsException error, when I was attempting to rebuild the master index on a Sitecore 9 instance in Azure PAAS, the following error would appear in the logs:
{"key":"a1c7786e795d5d2f16525bda3d7087b9","status":false,"errorMessage":"Field 'content_1' contains a term that is too large to process. The max length for UTF-8 encoded terms is 32766 bytes. The most likely cause of this error is that filtering, sorting, and/or faceting are enabled on this field, which causes the entire field value to be indexed as a single term. Please avoid the use of these options for large fields.","statusCode":400},
I was able to trace the content_1 field back to the Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config configuration file to the field definition for _content (where the main item content is stored).
<field fieldName="_content" cloudFieldName="content_1" searchable="YES" retrievable="NO" facetable="NO" filterable="NO" sortable="NO" boost="1f" type="System.String" settingType="Sitecore.ContentSearch.Azure.CloudSearchFieldConfiguration, Sitecore.ContentSearch.Azure" />
As suggested by Azure search I set filterable, sortable and facetable to NO on this field. It would be unlikely that I would need any of these features on a field containing the bulk of the indexes text.

Furthermore I also found the same error with a mysterious field name 'script':
"errorMessage":"Field 'script' contains a term that is too large to process. The max length for UTF-8 encoded terms is 32766 bytes. The most likely cause of this error is that filtering, sorting, and/or faceting are enabled on this field, which causes the entire field value to be indexed as a single term. Please avoid the use of these options for large fields."
This field wasn't as easily traced, so I simply added a number of fields called script the to the list of fields to not be indexed. A patch file is shared below for the master database with Azure Search.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/" xmlns:search="http://www.sitecore.net/xmlconfig/search/">
<sitecore role:require="Standalone or ContentManagement" search:require="Azure">
    <contentSearch>
      <configuration>
        <indexes>
          <index id="sitecore_master_index">
            <configuration>
              <documentOptions ref="contentSearch/indexConfigurations/defaultCloudIndexConfiguration/documentOptions">
                <exclude hint="list:AddExcludedField">
      <scriptField tag="{CD8DA5E2-3B65-4A14-B7A6-9F41181CE172}">{CD8DA5E2-3B65-4A14-B7A6-9F41181CE172}</scriptField>
      <scriptField2 tag="{3AC4B854-6CF9-4B30-9D52-1E5518AFF0E8}">{3AC4B854-6CF9-4B30-9D52-1E5518AFF0E8}</scriptField2>
      <scriptField3 tag="{DF23B990-55C1-401F-8F77-4698EDBD6FA9}">{DF23B990-55C1-401F-8F77-4698EDBD6FA9}</scriptField3>
      <scriptField4 tag="{FF1383BB-F095-4958-9B6D-E555DB653C44}">{FF1383BB-F095-4958-9B6D-E555DB653C44}</scriptField4>
      <scriptField5 tag="{CEDA3E7F-7406-40D0-A154-7CF7E3E7E85B}">{CEDA3E7F-7406-40D0-A154-7CF7E3E7E85B}</scriptField5>
      <scriptField6 tag="{C0A5BD91-E658-46CE-B631-77CE337D8E6E}">{C0A5BD91-E658-46CE-B631-77CE337D8E6E}</scriptField6>
      <scriptField7 tag="{B1A94FF0-6897-47C0-9C51-AA6ACB80B1F0}">{B1A94FF0-6897-47C0-9C51-AA6ACB80B1F0}</scriptField7>
      <scriptField8 tag="{CD8DA5E2-3B65-4A14-B7A6-9F41181CE172}">{CD8DA5E2-3B65-4A14-B7A6-9F41181CE172}</scriptField8>
                </exclude>
              </documentOptions>
            </configuration>
          </index>
        </indexes>
      </configuration>
    </contentSearch>
  </sitecore>
</configuration>

Monday, May 28, 2018

Sitecore and Azure Search- PostFailedForSomeDocumentsException error

When attempting to rebuild the master index of a Sitecore 9 instance in PAAS, the indexing would freeze and the following error was in the logs:
Exception: Sitecore.ContentSearch.Azure.Http.Exceptions.PostFailedForSomeDocumentsException
Message: Partial success for insert or update. Some documents succeeded, but at least one failed.
This traces back to the number of items being sent to Azure Search in a given patch. Inside the Sitecore.ContentSearch.Azure.DefaultIndexConfiguration.config configuration file is a node called cloudBatchBuilder with a child of maxDocuments. Decreasing this value from the default (1000 at the time of writing), should resolve this error.

Tuesday, May 22, 2018

Sitecore Experience Commerce - Service Proxy execute compilation error

When attempting to call my custom API in the commerce engine, I was getting a compilation error on the Execute method of a service proxy call.
Cannot convert from 'Microsoft.OData.Client.DataServiceActionQuerySingle<MyModel>' to 'Microsoft.OData.Client.DataServiceActionQuery<MyModel>' 
At first I thought it was a duplicate method across both the shops and ops proxy. But as it turns out, I was expecting a return type of collection and the proxy was expecting a single return object (hence the single at the end of DataServiceActionQuerySingle).

This was resolved by going back into the commerce engine where I configured the API and changing the return type from:
configuration.ReturnsFromEntitySet<ExtendedPricingDetail>("Api");
To:
 configuration.ReturnsCollectionFromEntitySet<ExtendedPricingDetail>("Api");
Then a re-generation of the service proxy and all was working as expected.

Monday, May 21, 2018

Sitecore Experience Commerce - Creating a custom API in the Commerce Engine

Due to a customization in how my Sitecore Experience Commerce implementation structured pricing, there was a requirement for a custom API which returned this additional data. In my post Drilling into storefront rendering code and commerce engine pipeline code, I dug down to the proxy executing the API GetBulkPrices. In this post I am going to create my own custom implementation GetExtendedBulkPrices.

The return model

First I needed to create a model, which I wanted this method to return.
using System;
using System.ComponentModel.DataAnnotations;

namespace Plugin.MyProject.PriceDetail.Model
{
    public class ExtendedPricingDetail
    {
        [Key]
        public string ProductId { get; set; }
        public Decimal OriginalPrice { get; set; }
        public int Flybuys { get; set; }
        public string PromotionCode { get; set; }
        public string PromotionText { get; set; }

        public ExtendedPricingDetail()
        {
            ProductId = string.Empty;
            OriginalPrice = 0.00M;
            Flybuys = 0;
            PromotionCode = string.Empty;
            PromotionText = string.Empty;
        }
    }
}
The key attribute was required as it would error otherwise.

The command

I needed a command for my custom API to call. A command will generally not contain any business logic and will simply be used to call a pipeline. This post won't go as deep as pipelines, however my empty commerce minion code sample post has a pipeline example.
using Plugin.MyProject.PriceDetail.Model;
using Plugin.MyProject.PriceDetail.Pipelines;
using Sitecore.Commerce.Core;
using Sitecore.Commerce.Core.Commands;
using Sitecore.Commerce.Plugin.Catalog;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace Plugin.MyProject.PriceDetail.Commands
{
    public class GetExtendedBulkPricesCommand : CommerceCommand
    {
        private readonly IExtendedPricingDetailPipeline _pipeline;

        public GetExtendedBulkPricesCommand(IExtendedPricingDetailPipeline getSellableItemPipeline, IServiceProvider serviceProvider) : base(serviceProvider)
        {
            _pipeline = getSellableItemPipeline;
        }

        public async Task<IEnumerable<ExtendedPricingDetail>> Process(CommerceContext commerceContext, IEnumerable<string> itemIds)
        {
            using (CommandActivity.Start(commerceContext, this))
            {
                CommercePipelineExecutionContextOptions contextOptions = commerceContext.GetPipelineContextOptions();
                var items = new List<ExtendedPricingDetail>();

                foreach (var current in itemIds)
                {
                    ProductArgument productArgument = ProductArgument.FromItemId(current);
                    if (!productArgument.IsValid())
                    {
                        string str = await contextOptions.CommerceContext.AddMessage(commerceContext.GetPolicy<KnownResultCodes>().Error, "ItemIdIncorrectFormat", new object[1]
                        {
                            (object) current
                        }, string.Format("Expecting a CatalogId and a ProductId in the ItemId: {0}.", (object)current));
                    }
                    else
                    {
                        var priceDetail = await _pipeline.Run(productArgument, contextOptions.CommerceContext.GetPipelineContextOptions());
                       
                        if (priceDetail != null)
                        {
                            items.Add(priceDetail);
                        }
                    }
                }

                return items.AsEnumerable();
            }
        }
    }
}
As you can see, this example calls my custom pipeline IExtendedPricingDetailPipeline.

The API Controller

Now we need a controller with an action which can act as an API method, this code is a standard MVC example.
using Sitecore.Commerce.Core;
using Microsoft.AspNetCore.Mvc;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using System.Web.Http.OData;
using Plugin.MyProject.PriceDetail.Commands;

namespace Plugin.MyProject.PriceDetail.Controller
{
    public class ApiController : CommerceController
    {
        public ApiController(IServiceProvider serviceProvider, CommerceEnvironment globalEnvironment): base(serviceProvider, globalEnvironment)
        {
        }

        [HttpPut]
        [Route("GetExtendedBulkPrices()")]
        public async Task<IActionResult> GetExtendedBulkPrices([FromBody] ODataActionParameters value)
        {
            if (!ModelState.IsValid)
                return (IActionResult)new BadRequestObjectResult(ModelState);
            if (!value.ContainsKey("itemIds") || !(value["itemIds"] is JArray))
                return (IActionResult)new BadRequestObjectResult((object)value);
            JArray jarray = (JArray)value["itemIds"];

            var bulkCommand = Command<GetExtendedBulkPricesCommand>();
            var res = await bulkCommand.Process(this.CurrentContext, jarray != null ? jarray.ToObject<IEnumerable<string>>() : (IEnumerable<string>)null);

            return (IActionResult)new ObjectResult(res);
        }
    }
}

In this example we are calling this ApiController, which means it follows an example of a Shops API method. Much like the GetBulkPrices method, that means the Postman call URL for this API would be {{ServiceHost}}/{{ShopsApi}}/GetExtendedBulkPrices(). It takes in an input of a list of strings and will output my custom model defined above.

This simply calls my command (as defined above).

Registering Exerything

Inside the Configure Sitecore class for the given plugin which contains these newly added classes, they need to be configured. This is done as follows (and does not show the Pipeline configuration):
public void ConfigureServices(IServiceCollection services)
{
    var assembly = Assembly.GetExecutingAssembly();
    services.RegisterAllCommands(assembly);

    services.Sitecore().Pipelines(config =>
        config
            .ConfigurePipeline<IConfigureServiceApiPipeline>(c => 
            {
                c.Add<RegisterPricingApiBlock>();
            })
    );
}
There is a single method which will register all commands, and added is a reference to RegisterPricingApiBlock. This is where the API and it's added entities are registered/built up for the oData service layer (and therefore commerce proxy used from the front-end). The code for this class is:
using System.Threading.Tasks;
using Microsoft.AspNetCore.OData.Builder;
using Plugin.MyProject.PriceDetail.Model;
using Sitecore.Commerce.Core;
using Sitecore.Framework.Conditions;
using Sitecore.Framework.Pipelines;

namespace Plugin.MyProject.PriceDetail.Pipelines.Blocks
{
    class RegisterPricingApiBlock : PipelineBlock<ODataConventionModelBuilder, ODataConventionModelBuilder, CommercePipelineExecutionContext>
    {
        public RegisterPricingApiBlock() { }

        public override Task<ODataConventionModelBuilder> Run(ODataConventionModelBuilder modelBuilder, CommercePipelineExecutionContext context)
        {
            Condition.Requires(modelBuilder).IsNotNull($"{base.Name}: The argument can not be null");

            modelBuilder.AddEntityType(typeof(ExtendedPricingDetail));
            modelBuilder.EntitySet<ExtendedPricingDetail>("ExtendedPricingDetails");

            var configuration = modelBuilder.Action("GetExtendedBulkPrices");
            configuration.CollectionParameter<string>("itemIds");
            configuration.ReturnsCollectionFromEntitySet<ExtendedPricingDetail>("Api");

            return Task.FromResult(modelBuilder);
        }
    }
}
Here the entity (custom model) and API method have been registered, which now allows them to be called via Postman and accessed via the Service Proxy project.

Calling the API via Postman

The API call can be a copy of one of the existing examples that come with the SDK. In my case, as I was using the same input (string array) as the GetBulkPrices API call, I simply copied this and updated the action name.

Calling the API via Postman


Updating the Service Proxy

Rob Earlam, has a great explanation up about what the Service Proxy is. Basically it's used to be able to share models between the .NET Core commerce Engine and standard .NET Sitecore website. However as I have added a custom model, I will need this to re-generate at the Service Proxy level.

The prerequisite for this is to install the OData v4 Client Code Generator plugin into Visual Studio 2017.

Now you simply add the Sitecore.Commerce.ServiceProxy project to your solution and under connected services are the ops and shops services which have a right click option to update (as long as a commerce engine instance is listening on port 5000). There is a JSON configuration if required.

Regenerating the Service Proxy

Conclusion

This was a simple example of how a custom API endpoint can be added to the commerce engine to call custom code and return custom models. This code could be slightly modified/expanded to actually extend a given model in the commerce engine layer.

Sitecore Experience Commerce - The entity does not have a key defined

After creating a custom command, API and return entity, I was getting the following error when starting the commerce engine in debug mode.
The entity 'MyModel' does not have a key defined
at Microsoft.AspNetCore.OData.Builder.ODataModelBuilder.ValidateModel(IEdmModel model)
The fix was to add the [Key] attribute to one of the attributes of my model. This can be found under the System.ComponentModel.DataAnnotations model.
public class ExtendedPricingDetail
{
 [Key]
 public string ProductId { get; set; }
 public Decimal CurrentPrice { get; set; }
 public Decimal OriginalPrice { get; set; }
 public string PromotionCode { get; set; }
 public string PromotionText { get; set; }
 public bool OnSpecial { get; set; }

 public ExtendedPricingDetail()
 {
  ProductId = string.Empty;
  CurrentPrice = 0.00M;
  OriginalPrice = 0.00M;
  PromotionCode = string.Empty;
  PromotionText = string.Empty;
  OnSpecial = false;
 }
}

Sitecore Experience Commerce - Command won't resolve in commerce engine

I have a custom API in the commerce engine which was able to be called via Postman. However when debugging the API, it was unable to get a copy of my command. The line of code which was failing was:
var myCommand = Command<MyCustomCommand>();
This was returning null. What was missing that the command was not properly being registered, this is done in the  ConfigureSitecore class of the plugin which contains the command and is achieved with the following line of code:
services.RegisterAllCommands(assembly);
After this the command was able to be accessed and run correctly.

Thursday, May 10, 2018

Sitecore Experience Commerce - Brand facet is splitting text by spaces

During an implementation of a product list in Experience Commerce, I found that the brand facet was splitting the brand's text at the occurrence of spaces. So a brand of "My Brand" would appear as two separate facet items: "my" and "brand".

The field was set to "UN_TOKENIZED" which should have not allowed this to happen. Nevertheless, the fix is to change the fields returnType from "test" to "string". Then rebuild the web and master indexes for your Sitecore instance. Below is a patch file to make this change (for SOLR search provider).

    
        
            
                
                    
                        
                            
                        
                    
                
            
        
   

Wednesday, May 9, 2018

Sitecore Experience Commerce - A minion to import/sync catalogs and products

At the time of release, Sitecore Experience Commerce came with an API method which could be used to import and export commerce data. It used a zip containing many JSON files, each of which was a catalog, sellable item, category or relationship of some type. This is handy for once off exports of data which could then be imported into another environment (perhaps developer machines). But for a real world environment, something more robust is required and that's where a custom import minion comes in.

Introduction to minions

In Sitecore Commerce a minion can be thought of as a scheduled task, which runs in the minions commerce engine environment and has access to the full commerce context. 

The habitat minions environment which comes out of the box with commerce (defined in the PlugIn.Habitat.CommerceMinions-1.0.0.json file) has minions defined such as the full index minion and the incremental index minion. These two minions both handle indexing of the commerce data into SQL. The incremental index minion is scheduled to run every 5 minutes and looks for new or updated items to index and the full index minion is not scheduled and can be called via the run minion API call.

Creating a commerce minion and defining it's schedule has been covered in an earlier post. Much like a standard Sitecore pipeline it has a run method which is where things kick off (including initialization). From here you are able to call commerce pipelines (either your own custom or the out of the box ones). 

A pipeline can be consider a wrapper for a number of blocks which will contain the actual code and logic. As an example, there might be a need to get a Sellable Item which already exists inside Sitecore commerce. This is done using the pipeline IGetSellableItemPipeline. If you were to look inside the log files of one of your commerce engine environments (NodeConfiguration_Deployment01 log files in particular), you would see this pipeline is configured as follows:
IGetSellableItemPipeline (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.GetSellableItemInitializeBlock (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.ProductArgument)     ------------------------------------------------------------     Plugin.Catalog.GetSellableItemBlock (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Inventory.ResolveSellableItemInventoryInformationBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.EnsureSellableItemPoliciesBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Availability.EnsureSellableItemAvailabilityPoliciesBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.ICalculateSellableItemPricesPipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Availability.IPopulateItemAvailabilityPipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.AddSellableItemToContextBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
It takes in an argument of ProductArgument and will output a SellableItem. You can see that the pipeline itself calls a number of various blocks, each of which performs it's own piece of logic/code. In fact this particular pipeline actually calls additional pipelines (such as ICalculateSellableItemPricesPipeline) which again call their configured blocks (and perhaps further pipelines).

The import minion structure

If you are looking to build an import/sync minion then you obviously have the data available in some format. In my case, I was building a sync minion and used SQL server to provide delta information of the data for each run of the minion. This was structured into several key areas (each of which could be accessed and actioned in a separate block of my import minions custom pipeline. These are the required actions I identified:
  1. Added category
  2. Added sellable item
  3. Added ranging (a given sellable item appearing in a given catalog)
  4. Removed ranging (a given sellable item no longer appearing in a given catalog)
  5. Deleted sellable item
  6. Deleted category
  7. Updated sellable item
Therefore I structured my minion as follows:
  • Import Minion
    • Import Pipelines
      • ProcessAddedCategories
      • ProcessAddedSellableItems
      • ProcessRanging
      • ProcessDeranging
      • ProcessDeletedSellableItems
      • ProcessDeletedCategories
      • ProcessUpdatedSellableItems
Note that this is a cut down example, from what I actually built as there was a large amount of business requirements which dictated the needs to separate catalogs for distinct stores each of which required their own price books and price cards to handle separate pricing at each.

The import minion code

The example commerce minion code solution I have placed on GitHub is a great start for this project, as it comes with the minion itself along with a pipeline and block. That means that it only would need the extra blocks created as defined above.

I'm not going to provide a full working example of an import or sync minion, but instead will show the commerce (out of the box) pipelines you would call for some of the key interactions (creating a sellable item for example). These pipelines would be called as part of the relevant block section inside a loop (for example the loop of categories to add inside ProcessAddedCategories).

The reason we call these commerce pipeline (as opposed to the service calls to the API) is the fact that those APIs will actually themselves call the pipeline code. This saves going over the wire and dramatically will cut down on processing time.

To see which pipelines are available, head over to the NodeConfiguration_Deployment01 log files in one of your commerce instance engines logs folder (C:\inetpub\wwwroot\CommerceMinions_Sc9\wwwroot\logs in my case). A search through this for keywords such as SellableItem as an example, can show you whats available. From there it's a bit of trial and error to correctly call the pipeline. 

The following code snippets and commerce minions will require access to the Sitecore Commerce nuget repository. You should add this to Visual Studio: https://sitecore.myget.org/F/sc-commerce-packages/api/v3/index.json.

Adding a SellableItem

This example shows how to create a basic sellable item and once completed will return a SellableItem object. 
using Sitecore.Commerce.Core;
using Sitecore.Framework.Pipelines;
using System;
using System.Linq;
using System.Threading.Tasks;
using Sitecore.Commerce.Plugin.Catalog;

namespace Plugin.MyProject.Import
{
    class ImportMinionBlock : PipelineBlock<MinionRunResultsModel, MinionRunResultsModel, CommercePipelineExecutionContext>
    {
        private readonly ICreateSellableItemPipeline _createSellableItemPipeline;

        public ImportMinionBlock(ICreateSellableItemPipeline createSellableItemPipeline)
        {
            _createSellableItemPipeline = createSellableItemPipeline;
        }

        public override async Task<MinionRunResultsModel> Run(MinionRunResultsModel arg, CommercePipelineExecutionContext context)
        {
            try
            {
                var result = await _createSellableItemPipeline.Run(new CreateSellableItemArgument("MyProdID", "TestProduct", "Test Product", "This is a description of my test product"), context);

                if (result != null) // Null here means it failed to add
                {
                    SellableItem addedItem = result.SellableItems.FirstOrDefault(); // This is the item which has just been added
                }
            }
            catch (Exception)
            {
                // Some sort of failure - for example trying to add a duplicate product
                throw;
            }

            return arg;
        }
    }
}
The key pipeline in this case is the ICreateSellableItemPipeline.


Editing/updating a SellableItem

Now lets continue on from the previous example and add some additional data to our SellableItem.
using Sitecore.Commerce.Core;
using Sitecore.Framework.Pipelines;
using System;
using System.Linq;
using System.Threading.Tasks;
using Sitecore.Commerce.Plugin.Catalog;

namespace Plugin.MyProject.Import
{
    class ImportMinionBlock : PipelineBlock<MinionRunResultsModel, MinionRunResultsModel, CommercePipelineExecutionContext>
    {
        private readonly ICreateSellableItemPipeline _createSellableItemPipeline;
        private readonly IPersistEntityPipeline _persistEntityPipeline;

        public ImportMinionBlock(ICreateSellableItemPipeline createSellableItemPipeline, IPersistEntityPipeline persistEntityPipeline) 
        {
            _createSellableItemPipeline = createSellableItemPipeline;
            _persistEntityPipeline = persistEntityPipeline;
        }

        public override async Task<MinionRunResultsModel> Run(MinionRunResultsModel arg, CommercePipelineExecutionContext context)
        {
            try
            {
                var result = await _createSellableItemPipeline.Run(new CreateSellableItemArgument("MyProdID", "TestProduct", "Test Product", "This is a description of my test product"), context);

                if (result != null) // Null here means it failed to add
                {
                    SellableItem addedItem = result.SellableItems.FirstOrDefault(); // This is the item which has just been added

                    addedItem.Brand = "My Brand";
                    addedItem.Manufacturer = "My Manufacturer";
                    addedItem.ListPrice = new Money("USD", 9.99m);

                    var saveResult = _persistEntityPipeline.Run(new PersistEntityArgument(addedItem), context);
                }
            }
            catch (Exception)
            {
                // Some sort of failure - for example trying to add a duplicate product
                throw;
            }

            return arg;
        }
    }
}
The key pipeline used here is the IPersistEntityPipeline, and in fact can be used to persist (or save) any commerce entity of which you make changes to.

Final thoughts

Hopefully this blog post has provided a good base for anyone who is looking to create a Sitecore minion to handle importing and syncing products. As someone who has built a complete sync minion here are a few further tips:
  • Work with subsets of product data instead of testing you minion with larger data sets.
  • Test out how the minion will handle any corrupted data - that commerce server might not like
  • Logging is your best friend, as in many cases an exception on a given pipeline will stop the whole minion run
  • The example Habitat data that comes with commerce is your best friend, and is an example of what your end product should look like.
Any further questions, feel free to comment below or contact me on twitter @RyanBaileyNZ.

Friday, May 4, 2018

Sitecore Experience Commerce - Drilling into storefront rendering code and commerce engine pipeline code

After importing my own catalog containing various categories and sellable items into Sitecore Experience Commerce. I found that pricing was displaying as $0 for all items instead of resolving via price cards.

The problem rendering

Sitecore Experience Commerce - Product list product
The presentation details of the search results page, showed that the rendering which was incorrectly displaying pricing was the 'Product List' rendering.

Search results page renderings
This rendering then traced back to the following controller: Sitecore.Commerce.XA.Feature.Catalog.Controllers.CatalogController found inside the Sitecore.Commerce.XA.Feature.Catalog DLL.

On the load of the page, the product list also populates after page load via a JavaScript call. The network tab of the console in chrome shows this call as GetProductList.

Experience Commerce - API call

Tracing the website code

Using dotPeek we are able to de-compile the DLL and view the code contained inside the controller.

Controller source 
The main controller action of ProductList (and therefore it's model source of ProductListRepository.GetProductListRenderingModel()) is not the real source of this data, as it actually comes via an API call. However that same ProductListRepository has a method called GetProductListJsonResult and this is where the API call gets it's data from. Looking into this method leads down a rabbit hole as follows:
  1. AdjustProductPriceAndStockStatus
  2. CatalogManager.GetProductBulkPrices
  3. PricingManager.GetProductBulkPrices
  4. PricingServiceProvider.GetProductBulkPrices
  5. RunPipeline<GetProductBulkPricesRequest, GetProductBulkPricesResult>("commerce.prices.getProductBulkPrices", request);
    1. Resolves to Sitecore.Commerce.Engine.Connect.Pipelines.Prices.GetProductBulkPrices
    2. Then calls Proxy.Execute<SellableItemPricing> ... GetBulkPrices
  6. Sitecore.Commerce.Engine.Container.GetBulkPrices
    1. References the URI: "/GetBulkPrices"
  7. Sitecore.Commerce.Plugin.Catalog.ApiController.GetBulkPrices
  8. IGetSellableItemPipeline
After all of that digging it finally leads to a commerce engine pipeline being run. This is where the actual logic will query the commerce data and attempt to resolve the price.

Tracing Commerce Engine code

How the commerce engine works is that a given pipeline such as IGetSellableItemPipeline will call a number of blocks (which is where code the actual logic sits) or even another pipeline. To see what code a given pipeline is actually calling, head over to the logs folder of one of your commerce engine instances (C:\inetpub\wwwroot\CommerceAuthoring_Sc9\wwwroot\logs for example) and open up the latest NodeConfiguration log file.

Performing a search in the file for IGetSellableItemPipeline returns the following:
Sitecore.Commerce.Plugin.Catalog
IGetSellableItemPipeline (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.GetSellableItemInitializeBlock (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.ProductArgument)
     ------------------------------------------------------------
     Plugin.Catalog.GetSellableItemBlock (Sitecore.Commerce.Plugin.Catalog.ProductArgument => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Inventory.ResolveSellableItemInventoryInformationBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.EnsureSellableItemPoliciesBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Availability.EnsureSellableItemAvailabilityPoliciesBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.ICalculateSellableItemPricesPipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Availability.IPopulateItemAvailabilityPipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.AddSellableItemToContextBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
What this means is that for each call to IGetSellableItemPipeline, it will run each of these children blocks and pipelines in turn. Because I am interested in how pricing is resolved for a sellable item, the ICalculateSellableItemPricesPipeline is the where this happens. This pipeline is configured as:
Sitecore.Commerce.Plugin.Catalog
ICalculateSellableItemPricesPipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.GetSellableItemCatalogBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.ICalculateSellableItemSellPricePipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.ICalculateSellableItemListPricePipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
     ------------------------------------------------------------
     Plugin.Catalog.ReconcileSellableItemPricesBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
In my case, as I am not looking to debug list price and instead pricing via price cards the ICalculateSellableItemSellPricePipeline is what is relevant to investigate. This pipelines is configured as:
Sitecore.Commerce.Plugin.CatalogICalculateSellableItemSellPricePipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.CalculateSellableItemSellPriceBlock (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Plugin.Catalog.ICalculateVariationsSellPricePipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)
Which finally leads to a code block CalculateSellableItemSellPriceBlock and this is where the true logic to calculate the price for a given sellable item is.

Replacing and debugging Sitecore Commerce Engine code

The great thing about the Sitecore Commerce Engine, is that it is fully customizable. Any of the pipelines above can be removed, replaced and even reconfigured (in terms of what blocks/pipelines the pipeline run). In my case I would like to replace the CalculateSellableItemSellPriceBlock with my own version (the same code taken via DotPeek) so that it can be debugged and changed if required.

This is easily achieved by using the Commerce Engine source solution (which is what is deployed to the 4 commerce engine roles in IIS). This is the Sitecore.Commerce.Engine project and comes with the SDK when you install Experience Commerce (in my case Sitecore.Commerce.Engine.SDK.2.1.10). The Sitecore.Commerce.Engine project out of the box will reference a number of plugins, such as the braintree payment provider - Plugin.Sample.Payments.Braintree. In my case I have a number of custom plugins to extend commerce entities such as Price Cards and Sellable items. 

The first step is to create the replacement block inside a plugin project. I have called this example: CalculateSellableItemSellPriceBlockCustom
namespace MyPlugin.Pipelines.Blocks
{
    [PipelineDisplayName("Catalog.block.calculatesellableitemsellprice")]
    public class CalculateSellableItemSellPriceBlockCustom : PipelineBlock<SellableItem, SellableItem, CommercePipelineExecutionContext>
    {
        private readonly IResolveActivePriceSnapshotByCardPipeline _resolveSnapshotByCardPipeline;
        private readonly IResolveActivePriceSnapshotByTagsPipeline _resolveSnapshotByTagsPipeline;
        private IGetCatalogPipeline _getCatalogPipeline;

        public CalculateSellableItemSellPriceBlockCustom(IResolveActivePriceSnapshotByCardPipeline resolveSnapshotByCardPipeline, IGetCatalogPipeline getCatalogPipeline, IResolveActivePriceSnapshotByTagsPipeline resolveActivePriceSnapshotByTagsPipeline)
          : base((string)null)
        {
            this._resolveSnapshotByCardPipeline = resolveSnapshotByCardPipeline;
            this._resolveSnapshotByTagsPipeline = resolveActivePriceSnapshotByTagsPipeline;
            _getCatalogPipeline = getCatalogPipeline;
        }

        public override async Task<SellableItem> Run(SellableItem arg, CommercePipelineExecutionContext context)
        {
            // Code removed
            return arg;
        }
    }
}
Inside each plugin is a ConfigureSitecore class, it's in here that we can configure the replacement of the out of the box CalculateSellableItemSellPriceBlock block to my custom CalculateSellableItemSellPriceBlockCustom block.
public void ConfigureServices(IServiceCollection services)
{
 var assembly = Assembly.GetExecutingAssembly();
 services.RegisterAllPipelineBlocks(assembly);

 services.Sitecore().Pipelines(config =>
  config
   .ConfigurePipeline<ICalculateSellableItemSellPricePipeline>(c =>
   {
    c.Replace<CalculateSellableItemSellPriceBlock, CalculateSellableItemSellPriceBlockCustom>();
   })
 );
}
Now the code can be debugged and customized as required by a given commerce instances business rules. If you open up the node configuration after deploying the commerce engine, you can see that the out of the box pipeline is not referenced and the custom one is instead:
Sitecore.Commerce.Plugin.CatalogICalculateSellableItemSellPricePipeline (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)     ------------------------------------------------------------     Myproject.Pipelines.Blocks.CalculateSellableItemSellPriceBlockCustom (Sitecore.Commerce.Plugin.Catalog.SellableItem => Sitecore.Commerce.Plugin.Catalog.SellableItem)

Conclusion

Ultimately I traced my particular error back to a timezone issue (which stopped price card snapshots resolving correctly). However this post serves as an example of the processes required to trace back issues from the Commerce Experience Accelerator components back to the Sitecore Commerce Engine.

Tuesday, April 24, 2018

Sitecore Experience Commerce - No commerce data in Sitecore indexes

After creating a catalog with product/category data and hooking it up to your Storefront site, you might expect the data to start appearing on the front-end. Even after full index rebuilds of the web and master indexes, this was not working as expected for me. The query I was using on the web index was:
commercesearchitemtype_t:"Category"
Which would be expected to return me all categories as defined in the catalog. Likewise the following, should return all products (sellable items):
commercesearchitemtype_t:"SellableItem"
 In my case I had changed the environment name from "HabitatAuthoring", and had updated the defaultEnvironment setting. However there are a number of index related settings which require the updated environment as well. The following patch file if applied should correctly connect the Sitecore indexer up with the correct commerce environment.
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
    <sitecore role:require="Standalone or ContentDelivery or ContentManagement">
        <contentSearch>
            <indexConfigurations>
                <indexUpdateStrategies>
                    <sellableItemsIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.SellableItemsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </sellableItemsIntervalAsynchronousStrategyMaster>
                    <sellableItemsIntervalAsynchronousStrategyWeb type="Sitecore.Commerce.Engine.Connect.Search.Strategies.SellableItemsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </sellableItemsIntervalAsynchronousStrategyWeb>
                    <categoriesIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CategoriesIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </categoriesIntervalAsynchronousStrategyMaster>
                    <categoriesIntervalAsynchronousStrategyWeb type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CategoriesIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </categoriesIntervalAsynchronousStrategyWeb>
                    <catalogsIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CatalogsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </catalogsIntervalAsynchronousStrategyMaster>
                    <catalogsIntervalAsynchronousStrategyWeb type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CatalogsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
                        <Environments hint="list">
                            <environment patch:instead="environment">MyAuthoring</environment>
                        </Environments>
                    </catalogsIntervalAsynchronousStrategyWeb>
                </indexUpdateStrategies>
            </indexConfigurations>
            <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
                <indexes hint="list:AddIndex">
                    <index id="sitecore_master_index">
                        <locations hint="list:AddCrawler">
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.SellableItemsCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.CategoriesCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.CatalogsCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                        </locations>
                    </index>
                    <index id="sitecore_web_index">
                        <locations hint="list:AddCrawler">
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.SellableItemsCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.CategoriesCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                            <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.CatalogsCrawler, Sitecore.Commerce.Engine.Connect">
                                <Environments hint="list">
                                    <environment>MyAuthoring</environment>
                                </Environments>
                            </crawler>
                        </locations>
                    </index>
                </indexes>
            </configuration>
        </contentSearch>
    </sitecore>
</configuration>

Sitecore - Failed to create counter

Looking into the logs of my local Sitecore 9 instance, there were a number of warning related to performance counters:
ManagedPoolThread #0 17:13:08 WARN  Failed to create counter 'Sitecore.System\Events | Events Raised / sec'. Sitecore has no necessary permissions for reading/creating counters. Message: Access to the registry key 'Global' is denied.
ManagedPoolThread #1 17:13:08 WARN  Failed to create counter 'Sitecore.System\IO | File Watcher Events / sec'. Sitecore has no necessary permissions for reading/creating counters.Message: Access to the registry key 'Global' is denied.
To resolve this error, we need to add the application pool identity into two groups on your machine. This can be achieved by:
  1. From the start menu, open run and enter 'lusrmgr.msc'
  2. The two groups we are interested in are:
    1. Performance Log Users
    2. Performance Monitor Users
  3. Right click each group and select 'Add to group'
  4. Add the app pool identity of the Sitecore website and xConnect website. In my case these were: 'IIS APPPOOL\prefix.sc' and 'IIS APPPOOL\prefix.xconnect' where prefix is what was set in the Sitecore install PowerShell script.
  5. Restarting IIS via the start menu command (IISRESET) should now allow Sitecore to work correctly with performance counters.

Thursday, April 12, 2018

Sitecore Experience Commerce - Catalog Item of the Storefront catalog Configuration cannot be found

After performing a deployment to a local Sitecore Experience Commerce instance, the Storefront site was not loading and instead showed the following error:
Catalog Item of the Storefront catalog Configuration cannot be found
Catalog Item of the Storefront catalog Configuration cannot be found
The Sitecore back-end was still loading as expected. What caused this error was that at startup of the Sitecore site, the commerce engine services were not started. This caused Sitecore to not be able to load/cache the commerce data. To resolve this error, ensure the commerce engine services are started and restart the IIS web site for the Sitecore instance

Tuesday, April 10, 2018

Sitecore Experience Commerce - Shop CommerceEngineDefaultStorefront does not exist

On a local instance of Sitecore Experience Commerce, the following error was appearing inside the commerce business tools and the front-end Storefront website.
"@odata.context":"https://localhost:5000/Api/$metadata#Sitecore.Commerce.Core.CommandMessage","MessageDate":"2018-04 10T10:03:59.2415572Z","Code":"Error","Text":"Shop 'CommerceEngineDefaultStorefront' does not exist.","CommerceTermKey":"InvalidShop"
Sitecore Experience Commerce - CommerceEngineDefaultStorefront does not exist
This error traces back to the Sitecore.Commerce.Engine.Connect.config configuration file (available in App_Config\Include\Y.Commerce.Engine). The defaultShopName setting should match that of a valid Storefront setting node (available under /sitecore/Commerce/Commerce Control Panel/Storefront Settings/Storefronts).

Sitecore - XConnect XdbCollectionUnavailableException error

Inside the logs of a Sitecore 9 instance the following error was appearing inside the website logs:
ERROR Exception when executing agent aggregation/pathAnalyzerLiveAgentException: Sitecore.XConnect.XdbCollectionUnavailableExceptionMessage: The HTTP response was not successful: ServiceUnavailableSource: Sitecore.Xdb.Common.Web  at Sitecore.Xdb.Common.Web.Synchronous.SynchronousExtensions.SuspendContextLock[TResult](Func`1 taskFactory) at Sitecore.XConnect.Client.XConnectSynchronousExtensions.SuspendContextLock(Func`1 taskFactory) at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.Initialize(XmlNode configNode)   at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert, IFactoryHelper helper)   at Sitecore.Configuration.DefaultFactory.CreateObject(XmlNode configNode, String[] parameters, Boolean assert)   at Sitecore.Configuration.DefaultFactory.CreateObject(String configPath, String[] parameters, Boolean assert)   at Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient(String clientConfigPath)   at Sitecore.PathAnalyzer.Processing.Agents.TreeAggregatorAgent.Execute()   at Sitecore.Analytics.Core.BackgroundService.Run()
Inside the ConnectionStrings.config there were 4 settings which contained a thumbprint (FindByThumbprint) for the xConnect site:
  1. xconnect.collection.certificate
  2. xdb.referencedata.client.certificate
  3. xdb.marketingautomation.reporting.client.certificate
  4. xdb.marketingautomation.operations.client.certificate
For some reason this was incorrectly set, this should match the thumbprint of your xconnect client certificate. This is likely to be name.xconnect_client where name is that of your Sitecore site set in the install PowerShell script. This same thumbprint is also set in the configuration of the xconnect site.