Uploading scheduled auto-backups of editors content for your Helix / SXA website into Azure Blob storage

read full post

INTRODUCTION

I assume you are following good practices and develop Helix-powered websites (or SXA, that follows Helix by definition). In that case, you do separate actual user-generated content items from definition items, normally shipped along with each release and created by developers (if not, then please refer to an article I've previously written on that), so you end up having a Unicorn configuration that stores all of your author-edited content:

Will are going to split the process into 3 major steps

  1. Re-serialize Unicorn configuration for content items
  2. Archive serialized content
  3. Upload an archive into Azure Blob Storage

IMPLEMENTATION

1. Re-serialize Unicorn configuration for content items. Luckily, Unicorn provides us with MicroCHAP.dll library helping to automate the sync process as a part of your deployment process (note that the DLL and corresponding PowerShell module Unicorn.psm1  should be referenced from your code). The good news is that any verb can be passed to it, not just sync, so one can use 'Reserialize'. That call will look like:

$ErrorActionPreference = 'Stop'
$ScriptPath = Split-Path $MyInvocation.MyCommand.Path

Import-Module $ScriptPath\Unicorn.psm1
Sync-Unicorn -Verb 'Reserialize' -Configurations @('Platform.Website.Content') -ControlPanelUrl 'https://platform.dev.local/unicorn.aspx' -SharedSecret '$ecReT!'

2. Archiving serialized content would be the next step. If you click Show Config button for Platform.Website.Content configuration in Unicorn configuration page, you may find all the relevant information about it, including the physical folder where items are serialized. We need this folder to be archived. The step comes as:

$resource = "Content_$(get-date -f yyyy.MM.dd)"
$archiveFile = "d:\$resource.7z"
$contentFolder = "C:\inetpub\wwwroot\Platform.dev.local\App_Data\serialization\Project\serialization\Content"

7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on $archiveFile $contentFolder
I am using 7zip as an archiver, as my content is slightly more than 4GBs and traditional zip cannot handle that. As a hidden bonus, I am getting the best compression ratio coming with 7zip. Also, It would be worth checking if a file with such name exists at target and deleting it before archiving, especially if you run the script more often than daily.

3. Uploading to Azure Blob Storage concludes given routine. To make this happen you should have a subscription, and ideally a connection string to your Storage account. Then you may use the following code to achieve the result:
$containerName = "qa-serialization"
$ctx = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=lab1234;AccountKey=y9CFAKE6PQYYf/vVDSFAKEzxOgl/RFv03PwAgcj8K80mSfQFDojdnKfakeaLMva0S9DbrQTzNjDMdGCp7rseRw==;EndpointSuffix=core.windows.net"

Set-AzureStorageBlobContent -File $archiveFile -Container $containerName -Blob $resource -Force -Context $ctx

Also, it is assumed you already have blob container created, if not you need to create it upfront:
New-AzureStorageContainer -Name $containerName -Context $ctx -Permission blob
Optionally, you may want to delete the temporal archive, once it's uploaded to blob storage.


RUNNING

Here's an entire code that works for me:
# Step 1: re-serialize user-generated content
$ErrorActionPreference = 'Stop'
$ScriptPath = Split-Path $MyInvocation.MyCommand.Path

Import-Module $ScriptPath\Unicorn.psm1
Sync-Unicorn -ControlPanelUrl 'https://platform.dev.local/unicorn.aspx' -Configurations @('Platform.Website.Content') -Verb 'Reserialize' -SharedSecret '$ecReT!'
# Step 2: archive serialized user-generated content with 7zip using best compression $resource = "Content_$(get-date -f yyyy.MM.dd)" $archiveFile = "d:\$resource.7z" $contentFolder = "C:\inetpub\wwwroot\Platform.dev.local\App_Data\serialization\Project\serialization\Content" 7z a -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on $archiveFile $contentFolder # Step 3: upload generated content into Azure Blob Storage $containerName = "qa-serialization" $ctx = New-AzureStorageContext -ConnectionString "DefaultEndpointsProtocol=https;AccountName=lab1234;AccountKey=y9CFAKE6PQYYf/vVDSFAKEzxOgl/RFv03PwAgcj8K80mSfQFDojdnKfakeaLMva0S9DbrQTzNjDMdGCp7rseRw==;EndpointSuffix=core.windows.net"
Set-AzureStorageBlobContent -File $archiveFile -Container $containerName -Blob $resource -Force -Context $ctx #Step 4: clean-up after yourself Remove-Item $archiveFile -Force
I run it on a daily basis by Windows Task Scheduler in order to get a daily snapshot of editors' activity. The script produces the following output:


As a result of running script, I get an archive appearing in the Azure Blob Storage:



RESTORING

There's no sense in making backups unless you confirm restoring the data out of it works well. For content items, download an archive, restore and substitute content serialization folder with what you've extracted, then sync content configuration. As simple as that!

Note that your content should be aligned with the definition items, or it may not work well!

Hope this post helps!

Field level deny permissions in Helix based on Habitat and how that affects your workflows?

read full post

If you decide to use Habitat as a bootstrap platform for your Helix solution, while setting up workflows for your solution, you may come across a situation described below. By this blog post I will try to explain what happens, why is it so, and how to make things work.

Symptoms: you are about to set up workflows for the solution and have created a role for the content editors. Then you give read / write permissions for that role to the site content (likely to be /Home and /Global nodes under your site definition item, recursively). When logging as a user having Content Editor role mention above, you are able to Lock and Edit and later to Check-In an item, but the fields for that item are disabled. Weird. But doing the same on other items outside your website works well (for instance - Home item coming with Sitecore initial installation). Why is it so?


There are few of StackOverflow questions trying to sort this out: one and two. I have left few comments there helping other to solve the situation.


Explanation: Habitat uses an "intersection" of feature-or-foundation-level permissions (also knows as Functional roles) with project-level permissions (also knows as organisational rights). Most of the Habitat modules have such a functional role coming as a part of the module, is in the following format: modules\Feature XXX Admin or modules\Foundation XXX Admin.

What habitat does - it denies write access for the inheritance for all the fields by default and then explicitly allows writing permission for that particular Functional Role within a module. That is briefly explained in the official Helix documentation but two images below would be more descriptive:



Solution: two potential ways of sorting this out. The first option is when you decide to keep Functional roles as a part of your solution. In that case, you need to make sure your Content Editor roles also inherits from these Functional roles (or from an umbrella role inheriting a combination of Functional roles).

Another way will be if you decide to drop these Functional roles. In that case, you'll need to remove them from serialization config and source control, and also perform the following for each field affected:

1. Navigate to that field in Sitecore, for example: /sitecore/templates/Feature/Navigation/_Navigable/Navigation/ShowInNavigation

2. Click Security tab, then Assign. You'll see at least two roles available - Everyone and a Functional role for that module.

3. Selecting Everyone, remove Inheritance denial for both Item and Descendants by clicking both red crosses, then save (OK).

4. Repeat that for each field of each template for each of the Feature / Foundation layers.

Then users from Content Editors role will be able to edit all the fields.

Hope this helps!

Creating XML Sitemap for the Helix solution

read full post

I am working on a solution that already has HTML sitemap as a part of Navigation feature. Now I got a request to add also a basic XML sitemap with common set requirements. Habitat ships with an interface template _Navigable, so let's extend this template by adding a checkbox field called

ShowInSitemap, stating whether a particular page will be shown in that sitemap:


In order to start, we need to create a handler. Having handlers in web.config is not the desired way of doing things, it will require also doing configuration transform for the deployments, so let's do things in a Sitecore way (Feature.Navigation.config file):

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
    <sitecore>
        <pipelines>
            <httpRequestBegin>
                <processor type="Platform.Feature.Navigation.Pipelines.SitemapHandler, Platform.Feature.Navigation"
                           patch:before="processor[@type='Sitecore.Pipelines.HttpRequest.CustomHandlers, Sitecore.Kernel']">
                </processor>
            </httpRequestBegin>
            <preprocessRequest>
                <processor type="Sitecore.Pipelines.PreprocessRequest.FilterUrlExtensions, Sitecore.Kernel">
                    <param desc="Allowed extensions">aspx, ashx, asmx, xml</param>
                </processor>
            </preprocessRequest>
        </pipelines>
    </sitecore>
</configuration>

We rely on httpRequestBegin pipeline and incline our new SitemapHandler from Navigation feature right before CustomHandlers processor.

SitemapHandler is an ordinary pipeline processor for httpRequestBegin pipeline, so is inherited from HttpRequestProcessor:

    public class SitemapHandler : HttpRequestProcessor
    {
        const string sitemapHandler = "sitemap.xml";

        private readonly INavigationRepository _navigationRepository;

        public SitemapHandler()
        {
            _navigationRepository = new NavigationRepository(RootItem);
        }

        public override void Process(HttpRequestArgs args)
        {
            if (Context.Site == null 
                || args == null
                || string.IsNullOrEmpty(Context.Site.RootPath.Trim()) 
                || Context.Page.FilePath.Length > 0 
                || !args.Url.FilePath.Contains(sitemapHandler))
            {
                return;
            }

            Response.ClearHeaders();
            Response.ClearContent();
            Response.ContentType = "text/xml";

            try
            {
                var navigationItems = _navigationRepository.GetSitemapItems(RootItem);
                string xml = new XmlSitemapService().BuildSitemapXML(flatItems);

                Response.Write(xml);
            }
            finally
            {
                Response.Flush();
                Response.End();
            }
        }

        private Item RootItem => Context.Site.GetRootItem();

        private HttpResponse Response => HttpContext.Current.Response;
    }

And XmlSitemapService code below:

    public class XmlSitemapService
    {
        public string CreateSitemapXml(IEnumerable<NavigationItem> items)
        {
            var doc = new XmlDocument();

            var declarationNode = doc.CreateXmlDeclaration("1.0", "UTF-8", null);
            doc.AppendChild(declarationNode);

            var urlsetNode = doc.CreateElement("urlset");

            var xmlnsAttr = doc.CreateAttribute("xmlns");
            xmlnsAttr.Value = "http://www.sitemaps.org/schemas/sitemap/0.9";
            urlsetNode.Attributes.Append(xmlnsAttr);
            doc.AppendChild(urlsetNode);

            foreach (NavigationItem itm in items)
            {
                doc = CreateSitemapRecord(doc, itm);
            }
            return doc.OuterXml;
        }

        private XmlDocument CreateSitemapRecord(XmlDocument doc, NavigationItem item)
        {
            string link = item.Url;

            string lastModified = HttpUtility
             .HtmlEncode(item.Item.Statistics.Updated.ToString("yyyy-MM-ddTHH:mm:sszzz"));

            XmlNode urlsetNode = doc.LastChild;

            XmlNode url = doc.CreateElement("url");
            urlsetNode.AppendChild(url);

            XmlNode loc = doc.CreateElement("loc");
            url.AppendChild(loc);
            loc.AppendChild(doc.CreateTextNode(link));

            XmlNode lastmod = doc.CreateElement("lastmod");
            url.AppendChild(lastmod);
            lastmod.AppendChild(doc.CreateTextNode(lastModified));

            return doc;
        }
    }
Also, NavigationItem is a custom POCO:
 
public class NavigationItem
{
    public Item Item { get; set; }
    public string Title { get; set; }
    public string Url { get; set; }
    public bool IsActive { get; set; }
    public int Level { get; set; }
    public NavigationItems Children { get; set; }
    public string Target { get; set; }
    public bool ShowChildren { get; set; }
}


Few things to mention.
1. Since you are using LinkManager in order to generate the links, you need to make sure you have full URL path as required by protocol, not the site-root-relative path. So you'll need to pass custom options in that case:

2. Once deployed to production, you may face an unpleasant behavior of HTTPS links generated along with 443 port number (such as . That is thanks to LinkManager not being wise enough to predict such a case. However there is a setting that make LinkManager works as expected. Not obvious
var options = LinkManager.GetDefaultUrlOptions();
options.AlwaysIncludeServerUrl = true;
options.SiteResolving = true;
LinkManager.GetItemUrl(item, options);

or better option in Heliix to rely on Sitecore.Foundation.SitecoreExtensions:

item.Url(options) from


//TODO: Update the code with the recent



That's it!

Handling 404 in Helix

read full post

When creating websites with Sitecore it is always nice to a have a nice and shiny 404-page that complies with the website styles and is content editable. Which will not work in all the cases, unfortunately. If the request falls out of site context - user won't get that nice and shiny page. So let's see what we can do in Helix, as it assumes multisite by design. The code below is written on top of Habitat, but that should not make a big difference.

Since this functionality is website agnostic and relates to Project layer, we will be using Project.Common.Website. Here is the  configuration within Common.Website.config:

<httpRequestBegin>
   <processor patch:before="processor[@type='Sitecore.Pipelines.HttpRequest.LayoutResolver, Sitecore.Kernel']"
    type="Sitecore.Common.Website.Pipelines.HandleRequestError, Sitecore.Common.Website"
    patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']" />
</httpRequestBegin>

And the processor HandleRequestError.cs:

public class HandleRequestError : HttpRequestProcessor
{
    public override void Process(HttpRequestArgs args)
    {
        if (Context.Item != null || Context.Site == null)
        {
            return;
        }

        string filePath = HttpContext.Current.Server.MapPath(args.Url.FilePath);

        if (IsValidItem() 
             || ReservedUrls.Includes(args.Url.FilePath) 
             || File.Exists(filePath)) 
        {
            return;
        }

        if (Context.Database != null)
        {
            // if we are in a context of a specific site - serve content-editable 404
            Context.Item = Context.Database.GetItem(Context.Site.StartPath + "/404");
        }

        // return 404 HTTP status code in either cases, to be picked up further ahead
        HttpContext.Current.Response.TrySkipIisCustomErrors = true;
        HttpContext.Current.Response.StatusCode = (int)HttpStatusCode.NotFound;
    }

    private bool IsValidItem()
    {
        if (Context.Item == null || Context.Item.Versions.Count == 0) return false;

        if (Context.Item.Visualization.Layout == null) return false;

        return true;
    }
}
So briefly, if we have website resolved - we serve its customized content-editable /404 page item (each of the sites has this page under the same root-relative path).

Creating a simple workflow in Helix

read full post

This may be a not as comprehensive guidance, as it should be, however, I am using this blog post mostly for leaving notes in a cheatsheet manner for later. So, there are several steps to make things happen.

1. Let's create a security domain for our website - that should typically be in a site config on a project layer (Website1.Website.config for my example):
<?xml version="1.0"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
    <sitecore>
        <domainManager defaultProvider="file">
            <patch:attribute name="defaultProvider">config</patch:attribute>
            <domains>
                <domain id="website1" type="Sitecore.Security.Domains.Domain, Sitecore.Kernel">
                    <param desc="name">$(id)</param>
                    <ensureAnonymousUser>false</ensureAnonymousUser>
                </domain>
            </domains>
        </domainManager>
    </sitecore>
</configuration>


2. Create 3 roles: website1\Editor and website1\Approver on the same domain, as well as website1\everyone that is a member of sitecore\Sitecore Client Authoring and is shared between first two. Make website1\Editor and website1\Approver members of website1\everyone and website1\everyone in turn member of sitecore\Sitecore Client Authoring.Also make website1\Approver member of sitecore\Sitecore Client Publishing


3. Create user accounts. I am creating them as part of website1 domain but they may be part of sitecore domain if need them instance-wide.


4. Assign users to their appropriate roles - editors or approvers. Once assigned - the user is able to log in and load content editor, however not able to insert new item for our website or change presentation details. User also is able to open a page in experience editor but again cannot edit this item because do not have write access to it.


Image above shows how LaunchPad looks when users log into Sitecore. However due to not having permissions they will see the following message in a Content Editor:

5. Create your workflow. I won't be original calling new workflow as Website1 Workflow. The easiest for a quick start would be to clone Sample Workflow and adjust states and other refs to point within corresponding items within that newly created workflow.


6. Then assign page relevant templates into Website1 Workflow. (Standard values -> Workflow section. Set default workflow into Website1 workflow). This ensures all the new items of this template will have Workflow field set into Website1 Workflow and State field will have state preselected as per workflow's Initial state field (workflow definition item)



7. Now give permissions to the role:
Open Access Viewer, click Account from the left top corner and select website1\Editor. Then having it selected give permissions to for everything under
/sitecore/content/Website1/Home,/sitecore/content/Website1/Global (but explicitly deny editing and deleting the top node itself), and do not forget media library for that project

Once complete - users will be able to Lock and Edit items and later submit for approval. So far so good.

8. Next step is to ensure that editors will not be able to approve items. It can be done by denying permission on Awaiting Approval state for editor.


That will result in the following permissions set for Editors in Access Viewer:


While Approvers' Acess Viewer shows Awaiting Approval state available:



9. One more thing to mention - if you got your Helix solution created from Habitat - you may come into a situation when certain fields are not editable. That happens due to write permission of fields for Feature-level templates are set to deny. I have written an explanation and the solution in a separate blog post.


10. Add language permissions to a shared role website1\everyone:



11. Last, but not the least - serialization. What you will serialize? Standard values for all templates that now became part of the workflow. Workflow itself (as a part of your Website1 project serialization configuration), Roles, possibly Users (however remember that there is no way to serialize a user with password - the only option is deserializing a user by Unicorn with setting a default password, also apart having the same default password these users will have Created field updated, which in turn will trigger source control changes). Also need to serialize Languages (foundation layer) with updated permissions - .\src\Foundation\Serialization\serialization\Foundation.Serialization.Languages\Languages.yml should be serialized.


12. Testing workflows in its basic falls into three steps routine:

- Firstly, you need to log in as an Editor and create a page and provide the rest of required content. Once done - submit that for approval. An important check is opening workbox to ensure that editors can only see Draft mode but not Awaiting approval


- Secondly, re-login as Approver and open Workbox. Now you should see Awaiting Approval section and will be able to Approve using it.


- Finally, login as an admin, switch to web database and make sure all the content has been published. That includes related and child items.

Updating Sitecore 9 with Helix to 9.0 update 1 (rev. 171219)

read full post

I am currently developing a greenfield Helix-based solution on Sitecore 9. That is a challenging but thrilling path, resulting in nicely setup working platform with "one-click install + one click deploy" process, perfectly suitable for continuous integration. 

However, as soon as 9.0 update 1 has been released, I started anticipating upgrading my solution to that revision, but for a week or so we've been missing NuGet packages for the latest version. Since January 18-th they have been released so finally it became possible to update the solution. Below there are few things I have done to make it work.

Phase 1. Choosing an update approach to take. 

There are few options:

  • *.update file with actual version
  • upgrade zip archive with only changes + pdf guidance on how to update
  • SIF archive with CMS, xConnect and corresponding configuration *.json files

For sanity purposes, I prefer to fully uninstall the previous version and reinstall the updated afterwards, rather than overwrite things. Thanks to a flexibility of Helix, now developer should not worry about losing an existing state, as soon as everything is checked into a version control system - gulp script will pick the latest changes and do that job for us. That's why approach number 3 becomes a choice. 

As I am keeping web folder under source control (just clean install of each version in order to easily restore that state) it makes sense to move .git folder outside from web folder before we go with uninstall, as it obviously will remove entire web folder. After as we remove the previous version, and newer version settles there, one can return .git folder back and immediately benefit from seeing the difference between two versions. I will cover that below during stage 2. So, to uninstall, open PowerShell and do:

.\uninstall-xp0.ps1

Once complete, got to the next phase.

Phase 2. Preparing and installing new version

First of all, download Sitecore 9.0 Update 1 XPSingle from SDN download page. Since we're doing that for our development environment, make sure you get OnPrem edition, rather than Cloud.

Obviously, to install newer version one need to update Sitecore Install Framework that supports particular version. Luckily, Sitecore Installer knows everything it should regarding how to install the version. PowerShell (with admin rights):

Update-Module SitecoreInstallFramework

If for some reason it breaks with execution policy exception, modify current user appropriate permissions:

Set-ExecutionPolicy Unrestricted -Scope CurrentUser -Force

Copy Sitecore packages for both CMS and Xconnet into build/assets folder, along with previous version packages. In my case, these files:

Sitecore 9.0.1 rev. 171219 (OnPrem)_single.scwdp.zip and Sitecore 9.0.1 rev. 171219 (OnPrem)_xp0xconnect.scwdp.zip

and also replace *.json scripts from XP0 Configuration files 9.0.1 rev. 171219.zip

Then, quite important, update version details at settings.ps1 script:

$SitecoreVersion = "9.0.1 rev. 171219"
$InstallerVersion = "1.1.0"
Finally, run the installer:
install-xp0.ps1
and wait till it finishes. Warning: do not open Sitecore after it installs.

Phase 3. Saving clean state of web folder with new version
Previously, I have described an approach of restoring a newly installed and previously never run Sitecore from a dedicated git branch, called SitecoreFiles_CM. During this exercise given approach saved me plenty of efforts, as I had to restore initial state of web folder at least 20 -25 times, if not more. That's why, even before I first run Sitecore and it generates plenty of artefacts (logs, caches, etc.) - I do one another commit with a new version of webs folder on top of existing clean install of 9.0 (initial). As a bonus, I get a wonderful tool for identifying changes between clean install of both versions, including config changes, file structure, built-in apps and similar.

Phase 4. Update the solution
Open Visual Studio, but do not deploy as you need to update your solution with all the latest package version. A goal of that phase its to make sure that any of DLLs being published to webroot by gulp script will match existing version of their counterparty within /bin folder of webroot. Sitecore comes with 337 DLLs from the box, and one would probably need to write a decent automation tool to do matching. Again, thanks to SitecoreFiles_CM approach for simplifying my life, as git commit immediately shows all the DLLs that differ from those checked in as a part of 9.0.1 clean install commit. 
As the first step, I went and changed all Sitecore.*.NoReference libraries for each project so they all correspond to 171219 revision. Keep in mind, that some of Sitecore NuGet libraries are versioned differently and do not have revision number in their name.
Changing references in such a large solution taking much efforts, attention and time. That's why I recommend you to do quite often commits while updating your solution, as you'll likely to restore one or few times during that process. Finishing with Sitecore.* references is a good time to check the code.
Then, I went updating Glass Mapper, Unicorn, Unit tests, code generation libraries and the rest of third-party packages. Below there is a list the version I used that are compatible and function:
Autofixture 3.51.0
BuildWebCompiler 1.11.375
EnterpriseLibrary.* 6.0.1304
FluentAssertions 4.19.4
GlassMapper 4.5.0.4
HtmlAgilityPack 1.4.9.5
Kamsar.WebConsole 2.0.0
Lucene.Net 3.0.3
MicroCHAP 1.2.2.2
Moq 4.8.1
Mvp.Xml 2.3.0
netDumbster 2.0.0.1
Newtonsoft.Json 9.0.1
NSubstitute 3.1.0
Rainbow.* 2.0.0
SharpZipLib 0.86.0
Sitecore.FakeDb.* 1.7.2
Unicorn.* 4.0.3
xunit 2.3.1
Lastly, it is a good time to update System.* and Microsoft.* assemblies. 

One thing to mention is that you still need to keep several assemblies at non-final versions and do not update it to the latest, as they will break dependencies. These are listed below along with correct versions:
Microsoft.Extensions.DependencyInjection 1.0.0 
System.Reflection 4.1.0
Newtonsoft.Json 9.0.1
HtmlAgilityPack 1.4.9.5
System.Net.Http 4.0.0
Castle.Core 3.3.3
It took inadequate time to troubleshoot some of those mentioned above, thanks to git in both solution and web folder it became easier to try-test-restore the state.

Finally, it all looks great. Everything works perfect, including generation works, unit tests, Glass Mapper, Unicorn etc.  And few bonuses, of course.

Bonus 1: as soon as I merge this feature branch with updated solution into develop branch of our GitFlow repository, other developers will get the update semi-automatically - with next re-install after refreshing /build/assets folder.

Bonus 2: just nearly forgot to mention: as the reward for an update efforts, I have got an EXM module as part of the platform, yes - now since 9.0 update 1 it comes out of the box!

Routine for renaming a Helix module

read full post

That would be a quick one. Once you decide to rename existing module, you should follow this consequence order to avoid potential errors. When modifying a module, the higher layer of the changed module is - the more changes require to commit. There is a general consequence of steps below, some of them may not be necessary for your module, so you may skip them. Anyway, here we go:

  1. In serialization file, rename module name for each of predicate records. However, do not rename serialization config file itself at this stage. Publish that particular config to web root (Alt + ,; ALT + P hotkey combination) in order to overwrite existing config. That will enable change tracking for serialization for new paths already.
  2. Once published, rename corresponding items in Sitecore, as per serialization predicates changed on the previous step. Keep in mind that not all predicates are listed in serialization as they are configured at other places by the wildcard (ie. templates), but you need also to change them as well. As the result, there will be a new folder created under src\{layer}\{module_name}\serialization\ and it will have all root items serialized, per each of predicate.
  3. No go to Unicorn console (/unicorn.aspx) and re-serialize modified project. That will bring the rest of the items serialized.
  4. Now it's time to rename the module in Visual Studio:
    1. Change solution folder
    2. Change project name, right underneath solution folder
    3. Remove project out of solution and move it into parent folder having a new name, then add this project back to the solution as existing.
    4. Change Assembly Name and Default Namespace, also perform changes at AssemblyInfo.cs file.
    5. Iterate through all C# classes and adjust the namespaces to match the one changed at 4.3, (In case you're using ReSharper, it will highlight you invalid namespace, so hit ALT + ENTER in and select "Move to {your_new} namespace"). For T4 code generated models however it would be nice to delete those models and re-create just to ensure code generates correctly.
    6. Now go to App_Config/include folder and rename configs: {module_name}.{layer}.config and {module_name}.{layer}.Serialization.config
    7. Do the same for other configs, ie. App_Config/Environment/{layer} and so on.
    8. Also, in configs, you'll need to adjust namespaces, where {type, assembly} are referenced. That may be for instance a configuration of dependency injection, route definition, or any other references to custom classes.
    9. Go and check your /Views folder in order to adjust references
  5. Next step is to delete old module already existing at web root (if you're deploying on clean - you may omit this step). To get rid of existing items with old name, delete the following:
    1. App_Config\Include\{layer}\ - all configs having old module name as the part of their filename
    2. Within /bin folder, delete compiled module, for example: {Solution_name}.{old_name}.Website.dll
    3. Clean up /Views folder for old references as well.
  6. Now it is a good time to remove old serialization - it is kept under an old project name, actually almost duplicating the same you've produced at step 3 (except location and serialized paths, of course). 
  7. Before publishing changes to web root folder, go to Sitecore and rename strongly type references within the Sitecore content. That's in case you've got any custom functionality stored within items that having references in format "{srong_type, assembly}", so please go to Sitecore and rename them all to new namespaces, as was set in 4.3
  8. Finally, re-build the solution and use gulp script, select: publish-all-{layer} task.
  9. If you're renaming a project layer - there're few more things to be done. You would likely to change hostName attribute of the site definition node. In that situation, adjust IIS binding and create a hosts file record for that hostname (if not yet there).
  10. Post-deploy step: rename other content stored in Sitecore items. As for example, you could have a layout called oldsiteLayout.cshtml. Despite it still functions, it would be great to change its name to newsietLayout.cshtml for consistency. But in that case, you also need to change that name in Layout definition item for that project in Sitecore. Do that for all layouts or other affected items.
  11. Last, but not the least. Perform manual and/or integration tests in order to make sure your solution won't break. Breaking tests are always good as they make you critically review certain pieces of implementation, refactor them, or at least keep tests actual to your solution.

Hope that guide helps.

10 Useful tricks for Helix development

read full post

1. First, is the most important hotkey that published a single file - whether a static file oк Razor view, to the web folder, preventing you from running gulp script. Selecting the desired file, hold ALT and then click ; (semicolon) and p. Your file will be immediately available at web destination. Then most time-taking is Application pool recycle process, running each time you add/remove/modify a config file or DLL, so current approach prevents you from wasting time on that. The result:



2. Next, in order to run Sitecore in a debugger, one usually need to go to "Debug" menu of Visual Studio, then pick up "Attach to process". Then find w3wp.exe from a long list of available processes after selecting "Show processes from all users" checkbox. When you need to do that multiple times in a day (and you do) it becomes an unwanted waste of time. There is a Visual Studio extension called AttachToAllTheThings that adds 3 handy menu items into Tools menu, one of which is "Attach to IIS". So far so good, but it would be even better to assign a hotkey to make attachments quicker, I use a combination of SHIFT + ALT + I:



3. Working with Helix you often need to run gulp tasks, that is executed from Task Runner Explorer. Normally you need, to go to View menu item then Other windows and there find out Task Runner Explorer. In a clean vanilla Visual Studio you may use CTRL + ALT + BackSpace binding. However those using ReSharper may find that combination re-mapped, so I opt-out in favour of neutral SHIFT + ALT + T combination:



4. If you are using ReSharper, you can benefit from its solution-wide code analysis tool. This is a very handy tool that is often overlooked, but it may give you plenty of insights of what's going wrong with your solution and how to fix that. But please be careful as if it is enabled permanently - if affects your performance. Double click a grey circle at the right bottom corner of Visual Studio in order to enable this feature.


After a while, it presents you with a list of issues you need to review (and likely to fix):



5. Also, one more useful trick of ReSharper will be useful for everyone working with abstractions. We got used to navigating to a class by doing CTRL + Left Mouse Click. But when doing the same on an interface, you'll navigate to the interface definition - not something useful when you expected to get some concrete implementation. Do CTRL + ALT + Left Mouse Click instead and you'll navigate to exact implementation if the is one, or will be presented with a list of all concrete classes implementing that abstraction.


6. Helix: An approach(es) for restoring WebRoot to an initial state of vanilla Sitecore installation.


7. Know your tools: The easiest way to add a new project to your Helix solution


8. Use Helix + Glass Mapper + T4 Templates = Code Generation


9. In addition to T4 Templates code generation, you may use young brother - Visual Studio code snippets. Recently Raul Jimenez prompted at https://github.com/rauljmz/Sitecore.VisualStudio.Snippets.


10. ZeroDeploy from Hedgehog, who also made TDS, Razl and Feydra. This is something unbelievable, you can't imagine that! The idea of the project is to get rid of Application Pool recycles upon deployments of code and config files. How could that be at all possible? ZeroDeploy relied on Sitecore Dependency Injection (that's why it only can run on Sitecore 8.2 or later) and instead of /bin folder it stores versioned libraries into Sitecore database, being even visible at the Sitecore tree! To make this magic possible, it is advised to separate most of your high-level abstractions into standalone libraries, that are very rarely change and that referenced from your web project. At the same time most of the concrete implementation, including Controllers, Services, Repositories and libraries down-referenced by them can be dynamically deployed and loaded back without being put to /bin, retaining the functionality. ZeroDeploy also uses Visual Studio extension and NuGet packages for Sitecore. Seriously, the magic! I am now testing a beta version of ZeroDeploy and am going to write a detailed guide on how to enable this on your development environment (your UAT / staging / prod) still remain untouched.

Hope these tips and tricks will improve your productivity!

Helix working on Sitecore 9

read full post

I have managed to make Sitecore 9 and Helix working together. Yaaay!

There is a feature/v9 branch of Sitecore Habitat, where habitat guys unhurriedly adjust Habitat to Sitecore 9. I would advise you to take a look at that at least. One thing to mention (as for 01-Dec-2017) - you'd need to reference all the Sitecore 9 dependencies from NuGet manually, please use *.NoReferences version of each library)

Update: I've recorded entire process of installation Helix with Sitecore 9 on a clean developer machine, that already has Visual Studio 2017, SQL Server 2016 and IIS installed. For installation process also required: PowerShell and Chocolatey. Important: all the manipulations are done as Administrator.

Hope one may find useful this 24 minutes video. Thanks for watching!

Updating existing presentation details of base template's standard values after it has been set on derived templates

read full post

Problem: let's imagine the situation when you may have some complex template inheritance. You would like to set presentation details for each of these templates' standard values, somehow like below:

A - define layout, and header/footer common holders that will be shared for all sites (we've got a multisite setup for now)

B - define footer and header on a site level so that they remain the same within each of implemented sites 

C - define base page template that will add the rest of presentation shared between the pages (but not homepage)

Once you set presentation for standard values of A, you can go to standard values of B and see these changes, so that you add only B-specific components; the same exact story will be when you will go next to std. values of C an so on. So far, so good.

The problem occurs when you want to modify presentation coming with a base template after a derived presentation is already set. In that case, you will not see any difference, that may be seen a weird for a while. In our example, imagine you've set presentation for std. values of all three templates - A, B and then C, and then decided to add one more component to a presentation on A template (or change datasource item of existing etc.). You do changes for std. values of A, save it and as you see - these changes come into a play for template A, however, once you open B or C  they won't be there...

Explanation: let's think what in fact Standard Values are - just the default values for each of field defined in that (or parent) template. In the second case if a field has standard values for both parent and inherited template - it simply overrides parent value with inherited child's value. But, wait for a second - presentation is also stored in fields of Standard Template that all pages inherit from, how that makes possible, does it simply override?

No, for such particular cases when presentation fields are involved - override behaviour would not work at all. Let's look at our example - template A defines layout and header/footer sublayout and all that goes within __Renderings field - it's where (shared, not versioned) presentation is stored in XML-serialised format. But then, it would be overridden by setting concrete footer with no layout. Since it is the same field - it will lose layout at template B level and entire behaviour does not make any sense. To address this issue Sitecore implements a feature called Layout Deltas - so that presentation fields are not stupidly overwritten. Instead, after we defined default presentation for template A, it goes as is, as A does not have any base template with presentation set. But when setting presentation for B - it will only save the difference between itself and presentation of base template (if exists). When page is being rendered, Sitecore is wise enough to construct resulting page presentation from all the base templates only adding deltas with each derived template. That is how Layout Deltas work.

One may create multilevel presentation inheritance of standard values, appending more and more presentation on each of derived levels. However, when we want to adjust the presentation of base template (A) of current template (B) - changes will be affected only for A, but not B or C if they already have layout deltas defined. That behaviour raises questions without a doubt.

Solution: what we need to do to in order to ensure changing presentation details for A will be affected for all derived templates' items is to recourse inheritance tree of A and re-calculate layout delta for each of them with recent updates from A. In order to get this done I have re-worked a solution suggested by ... The difference is that since that time we now got a feature called Versioned Layouts, so that we need to operate both fields - __Renderings and __Final Renderings correspondingly. Apart from that I have tested it for a while and fixed few of stability issues.

Implementation: when we change presentation - we change the field, so eventually the holding item is being updated. In order to intercept this we add a pipeline processor for item:saving event:

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:set="http://www.sitecore.net/xmlconfig/set/">
  <sitecore>
    <events>
      <event name="item:saving">
        <handler type="Sitecore.Foundation.Presentation.Services.LayoutInheritance, Sitecore.Foundation.Presentation" method="OnItemSaving"/>
      </event>
    </events>
  </sitecore>
</configuration>

And the code. I am using XmlDeltas.GetDelta() and XmlDeltas.ApplyDelta() static classes of Sitecore.Data.Fields namespace to make this work.

using System;
using Sitecore;
using Sitecore.Data;
using Sitecore.Data.Events;
using Sitecore.Data.Fields;
using Sitecore.Data.Items;
using Sitecore.Data.Managers;
using Sitecore.Data.Templates;
using Sitecore.Diagnostics;
using Sitecore.Events;
using Sitecore.Globalization;
using Sitecore.SecurityModel;

namespace Sitecore.Foundation.Presentation.Services
{
    public class LayoutInheritance
    {
        public void OnItemSaving(object sender, EventArgs args)
        {
            Item item = Event.ExtractParameter(args, 0) as Item;
            PropagateLayoutChanges(item);
        }

        private void PropagateLayoutChanges(Item item)
        {
            if (StandardValuesManager.IsStandardValuesHolder(item))
            {
                Item oldItem = item.Database.GetItem(item.ID, item.Language, item.Version);

                PropagateLayoutChangesForField(item, oldItem, FieldIDs.LayoutField);
                PropagateLayoutChangesForField(item, oldItem, FieldIDs.FinalLayoutField);
            }
        }

        private void PropagateLayoutChangesForField(Item item, Item oldItem, ID layoutField)
        {
            string layout = item[layoutField];
            string oldLayout = oldItem[layoutField];

            if (layout != oldLayout)
            {
                string delta = XmlDeltas.GetDelta(layout, oldLayout);
                foreach (Template templ in TemplateManager.GetTemplate(item).GetDescendants())
                {
                    ApplyDeltaToStandardValues(templ, layoutField, delta, item.Language, item.Version, item.Database);
                }
            }
        }

        private void ApplyDeltaToStandardValues(Template template, ID layoutField, string delta, Language language, Sitecore.Data.Version version, Database database)
        {
            if (template?.StandardValueHolderId != (ID)null)
            {
                try
                {
                    Item item = ItemManager.GetItem(template.StandardValueHolderId, language, version, database, SecurityCheck.Disable);

                    if (item == null)
                    {
                        Log.Warn($"Foundation.Presentation: Item is null {template.StandardValueHolderId} in database {database.Name}", template);
                        return;
                    }

                    Field field = item.Fields[layoutField];

                    if (field == null)
                    {
                        Log.Warn($"Foundation.Presentation: Field is null in item {item.ID} in database database.Name", item);
                        return;
                    }

                    if (!field.ContainsStandardValue)
                    {
                        string newFieldValue = XmlDeltas.ApplyDelta(field.Value, delta);
                        if (newFieldValue != field.Value)
                        {
                            using (new EditContext(item))
                            {
                                LayoutField.SetFieldValue(field, newFieldValue);
                            }
                        }
                    }
                }
                catch (Exception e)
                {
                    Log.Info($"Foundation.Presentation: Exception {e.Message}", e);
                    throw;
                }
            }
        }
    }
}
As I am using Helix in my development, I created a foundation module Presentation and placed the above code and config into it correspondingly. 

Hope this helps someone!

Helix Glass Mapper T4 Templates = Code Generation

read full post

The objective of given exercise is to achieve automatic code generation of Glass Mapper model (an interface) from a corresponding serialized interface template.

First of all, you need to install Visual Studio Extension for T4 templates: for Visual Studio 2017 or 2015. Also very recommend installing Devart T4 Editor - a markup highlighter and intellisense for T4 templates, that is also a plugin for Visual Studio 2017 or 2015.

Another prerequisite is Glass Mapper itself, obviously. I would advise going with BoC.Glass.Mapper.Sc by Chris Van Steeg from Sitecore NuGet repository. The fork contains several fixes for code-first and content search usage (project fork GitHub page).

The main thing you need is Sitecore CodeGenerator, located in https://github.com/hermanussen/sitecore.codegenerator

You'll also need to have "base" templates - GlassGenerator.tt and GlassMappedClassTemplate.tt as below:


SitecoreTemplates.tt - is the template you are going to copy to each module where you may need to generate models, you may want to rename each instance of it to reflect module name.

Once copied, just paste a list of your interfaces templates into SitecoreTemplates.tt (or WhateverYouCallIt.tt, in the example below I called that same as a module name) and save the file, or optionally right-clicking "SitecoreTemplates.tt" and choosing "Run Custom Tool". You'll see you interfaces magically generated as the child items of *.tt file, with all the references set up (in my case ILink.gen.cs and ILinkMenuItem.gen.cs)

NavigationTemplates.tt (from Feature/Navigation module):

<#@ template language="C#" debug="True" #>
<#@ output extension="gen.txt" #>
<#@ include file="T4Toolbox.tt" #>
<#@ include file="$(SolutionDir)tools\CodeGeneration\T4.Templates\base\GlassGenerator.tt" #>
<#
	GlassGenerator generator = new GlassGenerator(
			"master",
			new [] { 
					"/sitecore/templates/Feature/Navigation/_Link",
					},
			(fieldId, fieldOptions) =>
				{
				});
    generator.Run();
    
	WriteLine("These files were generated:");
#>



I am attaching these TT files for your convenience below:

GlassGenerator.tt (3.6KB), GlassMappedClassTemplate.tt (3.8KB) and SitecoreTemplates.tt (1020B).

If you need to change the way how the code is being generated please consider modifying "GlassGenerator.tt" and "GlassMappedClassTemplate.tt" file located in "Configuration\CodeGeneration\Templates\Base\" solution folder.


Notes: you may have Sitecore.CodeGenerator as a part of your solution, or unload from it - code generation still would work, just please make sure template files are referencing the right paths. Samples provided above are configured as at the image above.


Helix: An approach for restoring WebRoot to an initial state of vanilla Sitecore installation

read full post

I have an interesting approach to organizing work with Helix. Imagine a quite standard situation, when you have an already running solution with a code base outside web folder, into which gulp script deploys into. Actually, default Helix scenario.

So, you've been working on some module, then halfway down you've renamed that module or simply decided to remove it from solution. After changes finishing with renaming /deleting, and following redeploying the solution, you may potentially get an error (or not get it), but in any case, sooner or later you find out that your web folder still has an old module in a previous stage, dll and configs. To fix that, a DLL of that module needs to be deleted from bin folder manually, as well as all the corresponding old config files from App_Config/Include folder. Not a nice approach...

Things get even worse if you continuously repeat that delete/rename exercise with other modules, increasing chances of malfunction or false positive, and the far you go with that - the more complicated is to clean up the ends. Can you imagine how many unwanted trails are left in your web folder?

For sanity purpose, it is the best practice to ensure that your code is deployed into a clean copy of Sitecore at webroot so that you can be sure that it functions as expected (or not). So how would you achieve obtaining an exactly the same clean Sitecore as you have installed initially, and make this procedure super quick and repeatable? In case you are working on Sitecore previous to version 9 and have used SIM tool in order to install your Sitecore instance, then the answer is obvious - just use SIM backup / restore for that Sitecore instance, quick and reliable. But what if not? What if you are using the latest Sitecore 9?

The method I am suggesting may be sort of arguably, however, it does the job perfectly, quickly and keeps things consistent and at the same place. What I do - I create an isolated git branch called SitecoreFiles_CM at the same git repository where the main codebase sits, and commit an initial state of Sitecore to that branch. Saying initial I mean exactly the same files as they were installed, before Sitecore was even ever accessed - a clean installation. Of course, not all the files are pushed to remote: things like configurations with my individual (for my local dev instance) passwords do not need to be shared with other developers as well as Sitecore license file. That's why on top of that branch that is pushed to remote, I create one more local commit having these individual settings/changes that I do not push. When I have a need to restore vanilla Sitecore, I navigate to web folder and restore folder to that commit. The next step is git clean command required to get rid of the rest of unversioned files on top of vanilla Sitecore that were deployed from the codebase. As simple, as functional.

  git reset --hard
  git clean -fdx


Both codebase and the corresponding vanilla Sitecore are kept together at the same git repo, for consistency.

Additional benefit: with git diff, you may always see the difference between current web folder state and initial clean install. Not to say, that you can restore each individual file from that set!

When doing platform version upgrades, I upgrade my major codebase along with SitecoreFiles_CM branch, so they again remain consistent Looking ahead, no one stops you from having also a SitecoreFiles_CD branch with a CD set of files, however that is outside of current discussion.

Hope this helps!

Know your tools: The easiest way to install Habitat - Habitat Solution Installer

read full post

Working with Helix often encourages you to perform quick look-ups into the live-standard-implementation - Habitat project. That's why you have to have it installed. I remember the first time I installed Sitecore Habitat in October 2015 and how complicated that seemed at glance.

Luckily we now got a nice tool, that does exactly what it is named for - Habitat Solution Installer written by Neil Shack. So, let's go ahead and install Habitat into a new non-traditional destination using that tool.

Firs of all, let's grab Habitat Solution Installer itself from Sitecore Marketplace. Once downloaded, run HabitatInstaller.exe.


First screen takes three most important Habitat setting that we usually need to change as well as asks for the solution root folder where it will install the code. Once Install is clicked - it will download an archive of master branch from Habitat GitHub repository.


Then it will extract downloaded archive into temporal folder. By the way, you may alternate both path to master archive and your temporal files folder by clicking Settings button on the first (main) screen.


After extracting files, it will run npm install so you need to have node installed as a prerequisite.


Once finished, Habitat Solution Installer will display confirmation box.


So, what it has done - it installed and configured project code folder. But what hasn't it done?

1. It does not install Sitecore. You need to have it installed as another prerequisite, so that you provide Sitecore web folder and hostname to installer as shown on first screenshot. The best way would be to install using SIM (marketplace link). While installing Sitecore, make sure you're installing the right version corresponding to to codebase at Habitat master branch, you may look it up at Habitat Wiki page.

2. Not just to say you need to install Sitecore itself, you also need to install Web Forms for Marketers of the version corresponding to you Sitecore instance. And to ensure WFFM installation not failing, you need to install MongoDB prior to. Luckily that can be done in one click using SIM:


Finally, when all above is done, you may run gulp tasks from Task (View => Other Windows => Task Runner Explorer in Visual Studio 2015). Since npm install was already done for you - tasks are loaded as normal:


That's it! After Sitecore items are deserialised into your Sitecore instance, you'll be able to run Habitat website (however do not forget to publish from master to web unless you run it in live mode). The final result comes in you browser:


Know your tools: The easiest way to add a new project into your Helix solution

read full post

UPDATE: just came across a better solution from kamsar, that generates code, serialization and tests for the module. Please use that one instead of described below, however, the principle described below is exactly the same so you may refer for the sake of understanding.

---------------------------- original text below ----------------------------

I have found a great solution that makes adding a new project into existing Helix / Habitat solution so transparent and painless - Prodigious Helix Generator.

The guidance is well done and describes all thee steps. Is is soooo easy, and removes pain of fixing some forgotten references / namespaces / etc after you accidentally found out. It also optionally creates TDS project skeleton for your module, once you need to serialise items (why not to add Unicorn in addition to?).

So, to make it work you need to install yeoman first and afterwards the generator tool itself:

npm install -g yo
npm install -g generator-prodigious-helix

One important thing to mention is that not to mess with the paths and to get new module generated under the right path - run it at the root of your solution / repo.

You are at a choice of 3 options. In order to add a foundation, type in:

yo prodigious-helix:foundation

to make a feature:

yo prodigious-helix:feature

and for project even simpler:

yo prodigious-helix

As for example, below I am creating a new foundation module called AdminTools for the solution named HelixSolution. That will result in creating me a project called HelixSolution.Foundation.AdminTools within a same name folder (created) at the right path under Foundation folder. All the namespaces, references, settings and naming conventions would be auto-generated and correct!


After answering those 3 questions, you'll get your stuff generated:


That's it!

The only thing I noticed it that it appends solution name under App_config folder: App_Config\Include\HelixSolution\Feature\Feature.AdminTools.config so depending on your setup you may need to move Feature\Feature.AdminTools.config one level up into Include folder.

Hope it will start saving your time as much as it saves mine.