How to Make Usage Analytics Work in SharePoint 2016

In this post I’m going to show you how to fix an issue in SharePoint 2016 that prevents usage reports from being correctly updated.


We have a SharePoint 2016 site and we need to show users the usage reports.


The obvious solution would be to use the out of the box SharePoint 2016 Excel usage reports to do that… Unfortunately when we open them we always find all zeros, no matter how many users have interacted with the site and how many documents have been opened.


Although we have correctly enabled Usage and Health Data Collection for the right events in Central Administration nothing happens.


After some research we found that in our fresh new installation of SharePoint 2016 the usage event receivers assemblies were wrong, version instead of version

A really tricky bug that fortunately in our case was successfully fixed by a few Powershell commands and some patience to wait data to be collected and reports to be updated.


1. Get Usage Analytics Event Receivers

Run the SharePoint 2016 Management Shell as Administrator and run the following Powershell commands to get the “AnalyticsCustomRequestUsageReceiver” and the “ViewRequestUsageReceiver” event receivers.

#Get AnalyticsCustomRequestUsageReceiver event receiver
$ad = Get-SPUsageDefinition | where {$_.Name -like "Analytics*"}
$vru = $ad.Receivers
#Get AnalyticsCustomRequestUsageReceiver event receiver
$pd = Get-SPUsageDefinition | where {$_.Name -like "Page Requests"}
$pru = $pd.Receivers

2. Check Usage Analytics Event Receivers Assembly Version

Once you get the event receivers, check for each one that the “ReceiverAssembly” property contains the correct assembly version.

For a SharePoint 2016 environment the correct assembly version must be as shown in the picture below.


3. Delete the Wrong Event Receivers

If you are in a SharePoint 2016 environment and the event receivers assembly version is not (for example it is you must delete them executing the following Poweshell commands.

#If the receivers assembly is wrong, delete them using the script below

4. Recreate the correct Event Receivers

Now we need to recreate the correct event receivers (assembly version executing the following commands.

#Create the correct event receivers

5. Run Usage Analysis and Import

Finally, we need to start the usage analysis and import as described below.

#Start usage analysis
$a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")
#Get Analysis info. Make sure to wait for 15 minutes as this will allow to complete the timer job properly
#Start the usage log file import timer job
$tj = Get-SPTimerJob -Identity ("job-usage-log-file-import")


Having done all the previous steps, we now need only to let the timer job “Microsoft SharePoint Foundation Usage Data Processing” complete as per its schedule (this will not work if you run it manually).

After that, we should be able to view the usage report data correctly updated.


Rumi’s Point






How to Create a Timer Job with SharePoint Designer 2013

In this post I’m going to show you how to create a no server side code timer job using a SharePoint Designer 2013 site workflow.


We have a SharePoint Online (Office 365) site with a task list in it and we need to send daily alerts to users that have overdue tasks assigned to them.



The obvious solution would be to create a timer job to do that… Unfortunately we are in an Office 365 environment so we do not have the possibility to implement a standard timer job using server side code.



We can simulate the behaviour of a timer job using a Sharepoint Designer 2013 site workflow.


1. Create a New Site Workflow in Sharepoint Designer 2013

Open SharePoint Designer 2013 and create a new site workflow


2. Get All Overdue Tasks that are not Completed

In the first stage (let’s call it “Getting Overdue Tasks”) build a dictionary to store the accept header for our rest request and get results in json:

accept: application/json;odata=verbose



Put the current date in a DateTime variable (let’s call it “today”).


Then call the SharePoint REST Service to get the overdue tasks.
To do so insert the “Call HTTP Web Service” action in the workflow (in an App Step to avoid permission related problems).


The action url must be constructed dynamically  using the “today” variable like this:



Set the action verb to GET, the request headers to the headers dictionary previously built and the response content to a dictionary to store the request result.


Get and store the returned tasks in a separate dictionary to count them and loop through them later.


3. Send Alerts to Overdue Tasks Assignees

Create a second stage (let’s call it “Sending Alerts”) to send alerts and then link the first stage to it.

Insert a loop and inside it extract the task id from the rest response and send an email to each overdue task assignees.

Use a loop index variable to address the current task and remember to initialize it to zero and increment it properly at the end of the loop.



To get the email address of the recipient (the task assignee) use the task id previously extracted from the rest response like this:


4. Wait One Day and Start Over Again

Insert a pause action and configure it to wait one day.

Link the “Sending Alerts” stage to the “Getting Overdue Tasks” stage to start over again.



Here is how the complete workflow will look at the end of the process:


Publish the site workflow and start it (you can access the “Site Workflows” page from the “Site Contents” page).


It will send the alerts daily as expected.



Fabian Williams

4 Steps to Get the Real Error Message from SharePoint 2013 Log Files

In this post I’m going to show you how to find the details of an error in the SharePoint log files.


Sorry, something went wrong. An unexpected error has occurred.

Many of us have seen this non-informative message before, wondering what really happened.


We need a way to get the real SharePoint exception to investigate the problem.


We can do that by using the SharePoint “Unified Logging System” (ULS).

SharePoint saves the details of errors and other diagnostic information in a log file on the server where the error happened.


1. Get Correlation ID

The first step is to note the Correlation ID.
You can directly copy it from the browser in the SharePoint error page.


2. Get and Open ULS Viewer

You can open log files with Notepad or your text editor of choice.

However Microsoft has a tool called ULSViewer that can help you find your error more easily with the use of filters.


3. Open Log File

Now you need to find and open the specific log file that contains your error.

To open a log file click on File -> Open From -> File.


By default SharePoint log files are stored in the SharePoint hive in the LOGS folder:

C:\Program Files\Common Files\microsoft shared\Web Server Extensions\15\LOGS

FindError_LogFolderHowever note that this path can be changed in SharePoint Central Administration.

SharePoint log files are named with the format “SERVERNAME-YYYYMMDD-HHMM.txt” where SERVERNAME is the server’s computer name, YYYYMMDD is the current date and HHMM is the time the log was started using a 24-hour clock.

In single server farms open the log from the day and time which the error happened.

In multi-server farms you’ll need to go to the specific server where the error happened to locate the log. If it’s not immediately obvious where the error happened you’ll need to check every server in the farm.


4. Filter Log Entries

To find your error you will need to filter the log entries by Correlation ID and Level.

To do this click on Edit -> Modify Filter.


Then set the following values in the Filter By dialog.

Correlation ID filter:

Field: Correlation
Operation: Equals
Value: The Correlation ID you copied from the SharePoint Error message
And/Or: And

Level filter:

Field: Level
Operation: Equals
Value: Unexpected


Click OK and ULSViewer will show only the events that have the Correlation ID and Level specified.



Now we can finally see the real error!

If you double click on the log entry, a window will open to show all the error details so that you can begin your investigation.



Jason Warren

How to Estimate User Stories

In this post I’m going to show you the process and techniques to effectively estimate user stories.


In our project we are using an iterative process in conjunction with user stories and we need, or are asked, to provide an estimate of how long the project will take.


Ideal Estimation

The best approach for estimating user stories would be one that:

  • Allows us to change our mind whenever we have new information about a
  • Works for both big and small stories
  • Doesn’t take a lot of time
  • Provides useful information about our progress and the work remaining
  • Is tolerant of imprecision in the estimates

Story Points


An approach that satisfies each of these goals is to estimate in Story Points.

Story Points are relative estimates of the complexity, effort or duration of a story.

A nice feature of story points is that each team defines them as they see fit.
Story points can be entirely nebulous units or ideal time units like “ideal day” or”ideal week”.
Of course, as we’ll eventually need to convert estimates into time, starting with ideal time makes that conversion a little simpler than starting with an entirely nebulous unit.

What to Include

When programmers estimate a story, they should include everything they’ll need to do to complete it.

They need to factor in such things as testing their code, talking to the customer, perhaps helping the customer plan or automate acceptance tests, and so on.

Estimate as a Team

Story estimates need to be owned collectively by the team for two main reasons:

  1. Since the team doesn’t yet know who will work on a story, ownership of the story cannot be more precisely assigned than to the team collectively
  2. Estimates derived by the team, rather than a single individual, are probably more useful and accurate



To estimate user stories as a team:

  1. Implement a baseline story
  2. Compare with the baseline story and other user stories
  3. Split in tasks

From Story Points to Expected Duration


Now we need a way to convert story points into a predicted duration for the project.
How to do that?

The answer is to use team velocity.

Velocity represents the amount of work that gets done by the team in an iteration

Once we know a team’s velocity, we can use it to turn ideal days into calendar days.
For example, if we estimate our project at 100 ideal days, then with a velocity of 25 we can estimate that it will take 100 / 25 = 4 iterations to complete the project.

It is often necessary to estimate a team’s initial velocity.
To do that you can:

  • Use historical values
  • Run an initial iteration and use the velocity of that iteration
  • Take a guess


Mike Cohn

4 Steps to Dynamically Display Locations on a Map in SharePoint 2013

In this post I’m going to show you how to display on a map the locations contained in a SharePoint 2013 list enabling the new Geolocation Field and Map View.

Note: The proposed solution works both for SharePoint 2013 On Premises and Online.


We want to dynamically display the locations contained in a list on a Bing Maps map in a SharePoint 2013 site.



1. Create the list

Create the list that will contain the locations to show on the map.

In our example the list has simply Title and Description fields.
Later in this post we’ll learn how to add the location field.


2. Get a Bing Maps key

You can obtain a valid Bing Maps key from here.


3. Create the location field

To create the location field on our list we need to use a custom form in a page containing a script.

So first create a page where you wish to place the custom form.


On that page add a Script Editor web part that will contain both the script and html for our form.


First add this javascript:

<script type="text/javascript">
var clientContext;
var relativeAdress;
 function AddGeolocationField() {
 clientContext = new SP.ClientContext(relativeAdress); 
 var web = clientContext.get_web();
 var targetList = web.get_lists().getByTitle(document.getElementById('listname_input_id').value);
 var fields = targetList.get_fields();
 "<Field Type='Geolocation' DisplayName='" + document.getElementById('fieldname_input_id').value + "'/>",
 clientContext.executeQueryAsync(function (sender, args) {
 document.getElementById('success_id').innerHTML = "You have succesfully created new geolocation field.";
 function (sender, args) {
 document.getElementById('error_id').innerHTML = "Error: "+args.get_message();
 function SetBingKey() {
 clientContext = new SP.ClientContext(relativeAdress);
 var web = clientContext.get_web();
 var webProperties = web.get_allProperties();
 webProperties.set_item("BING_MAPS_KEY", document.getElementById('bing_key_id').value);
 clientContext.executeQueryAsync(function (sender, args) {
 document.getElementById('success_id').innerHTML = "You have succesfully entered BING map key on "+web.get_title()+" site";
 }, function (sender, args) {
 document.getElementById('error_id').innerHTML = "Error: "+args.get_message();
 function GetBingKey() {
 clientContext = new SP.ClientContext(relativeAdress);
 var web = clientContext.get_web();
 var webProperties = web.get_allProperties();
 clientContext.executeQueryAsync(function (sender, args) {
 document.getElementById('bing_key_id').value = (webProperties.get_fieldValues()["BING_MAPS_KEY"]);
 }, function (sender, args) {
 document.getElementById('bing_key_id').value = "";
 document.getElementById('error_id').innerHTML = "Property not found! Please check your web site relative URL.";
 function ClearNotifications(){
 document.getElementById('success_id').innerHTML = "";
 document.getElementById('error_id').innerHTML = "";
 function GetRelativeAdress(){
 if (document.getElementById('webrelative_url_id').value === "")
 relativeAdress = "/";
 relativeAdress = document.getElementById('webrelative_url_id').value;
 document.getElementById('error_id').innerHTML = "Relative adress has to start with /";

Then add this html:

<table style="width: 480px;">
<td style="width: 200px;">Web relative URL:</td>
<td style="width: 5px;">&nbsp</td>
<td valign="top"><input id="webrelative_url_id" name="relative" type="text" value="/">
<label style="font-size: 8pt;">
* Input web relative URL and select "Get BING key" to check if key is set</label></td>
<td style="text-align: right;" valign="top"><input onclick="GetBingKey()" type="button" value="Get BING key"></td>
<td style="width: 200px;" valign="top">Bing Maps Key:</td>
<td style="width: 5px;">&nbsp</td>
<td valign="top"><input id="bing_key_id" name="bingkey" type="text">
<label style="font-size: 8pt;">
* Input Bing map key and relative url to web site to wich you wish to add the key</label></td>
<td style="text-align: right;" valign="top"><input onclick="SetBingKey()" type="button" value="Set BING key"></td>
<td style="width: 200px;" valign="top">List name:</td>
<td style="width: 5px;">&nbsp</td>
<td valign="top"><input id="listname_input_id" name="listname" type="text">
<label style="font-size: 8pt;">
* Name of the list where you wish to add your new geolocation field</label></td>
<td valign="top">Field name:</td>
<td style="width: 5px;">&nbsp</td>
<td valign="top"><input id="fieldname_input_id" name="fieldname" type="text">
<label style="font-size: 8pt;">
* Name of the new geolocation field you wish to add</label></td>
<td style="text-align: right;" valign="top"><input onclick="AddGeolocationField()" type="button" value="Create field"></td>
<label id="error_id" style="color: red;">
<label id="success_id" style="color: green;">

And now you have your form for adding geolocation fields anywhere on your site collection.


Once you have entered the relative path to your web you can use the Set BING key button to add the BING maps key to the web.
You can optionally use the Get Bing key button to check if you have already a BING maps key placed in that web.


Once entered List name and Field name you can use the Create field button to add a geolocation field with the name you specified to the list on the web.



Note: For SharePoint 2013 On Premises an MSI package named SQLSysClrTypes.msi must be installed on every SharePoint front-end web server to view the geolocation field value or data in a list.  This file is installed by default for SharePoint Online.

More information on this can be found here.

4. Create the Map view

You now have the possibility to create a Map view for your list to show locations.




You can now add items to your list and see their locations appear on the Map view.




Borislav Grgić

How to Export a SharePoint 2013 List with Lookup Columns

In this post I’m going to show you a workaround to an issue that causes data loss when exporting lists with lookup columns to another site.


We have 2 lists in a SharePoint 2013 site, one with a lookup column to the other, and we need to export them to another site without loosing any data.

Export List with Lookup - List A

Export List with Lookup - List B


In the List Settings page of our lists we can use the “Save as Template” functionality to export them.

Export List with Lookup - Save as Template

The list templates are successfully created so we can load them to the destination site collection and recreate the lists from them in the new site.

Note: the destination site language must be the same as the source one

Export List with Lookup - List Templates

However looking closer to List B we note that all the data in the lookup column has been lost…

Export List with Lookup - List B with Lost Data

This happens because the lookup column points to List A GUID, not its path or name…

The GUID of List A in the destination site is different from that of the same list in the source site, so the lookup column is now pointing to a list that does not exist in the new environment.

Unfortunately we can’t change the lookup column definition in the destination site and get our data back because the “Get information from: ” section is locked…

Export List with Lookup - Lookup Column Locked

So, how to solve this problem?


We can change the list template manifest file and set the correct list GUID or path for the lookup column.

Let’s see how to do that in detail…


1. Change the list template file extension

Download the template file of our list with the lookup column (in our example the file is ListBTemplate.stp) and change its file extension from “stp” to “cab”.

Export List with Lookup - List B CAB Extension

2. Extract the files contained in the list template cab package

You can use WinRAR or WinZip to extract the files contained in the cab file

Export List with Lookup - List B Extracted Files

3. Change the list template manifest file

Open the manifest.xml file contained in the list template extracted files (in our example it is the only file) and replace the lookup column list GUID (in our example the GUID of list A in site A, f8088b28-1434-4b51-96d3-a36e945e5146) with the new list GUID or relative path (in our example the GUID or relative path of list A in site B, 265bffc8-dcc9-48e3-99e4-d3ad1d5e2260 or Lists\List A).

Export List with Lookup - Manifest

4. Recreate the list template package

Now that we have changed the content of our manifest file we need to recreate the cab package. To do that we ca use the makecab command in the command prompt using the following syntax:

makecab "<source file>" "<destination file>"

In our example the specific command is as follows:

makecab "C:\List Templates\ListBTemplate\manifest.xml" "C:\List Templates\ListBTemplate\"

Export List with Lookup - makecab

Export List with Lookup - makecab result

Once created the cab file we can change its file extension to “stp”, reload it to the destination site collection and recreate our List B.


This is how our new list with the lookup column should look like:

Export List with Lookup - List B without Lost Data

You can see that our lookup data is there again without any loss.



The 4 Reasons Why User Stories are Better than Requirements Specifications

In this post I’m going to show you why User Stories are a better solution than Requirements Specifications to effectively understand user needs and build successful products.


Software requirements is a communication problem.

How do you decide what a software system is supposed to do?
How do you communicate that decision between the various people affected?


Experience has taught us that Requirements Specifications do not work anymore in today’s  world and that User Stories are a much better solution.

I’m going to explain you why in this post…

What are Requirements Specifications?

A typical Requirements Specifications fragment is the following:


There is a tremendous appeal to the idea that we can think about a system and then write all the requirements upfront.

Requirements Specifications written like that have sent many projects astray because they focus attention on a checklist of requirements rather than user goals.

What Is a User Story?

A user story describes functionality that will be valuable to either a user (who uses the system) or customer (who pays for the system).


The format of a User Story is the following

As a <Role> I want <Function> so that <Benefit>

as in the example below

As an executive, I want to generate a report so that I can understand which departments need to improve their productivity

Our goal with user stories is to write down a short place holding sentence that will remind developers and customers to hold future conversations.

Why User Stories are better?

1. Comprehension

A way that extensive upfront Requirements Specifications can kill a project is through the inaccuracies of written language.

The shift toward verbal communication of User Stories provides rapid feedback cycles, which leads to greater understanding.

2. Planning

A difference between User Stories and Requirements Specifications is that with the latter the cost of each requirement is not made visible until all the requirements are written down.

With stories, an estimate is associated with each one right up front so that they may be conveniently used for release planning.

3. Resiliency

Change always happens in projects…

  • Users and customers do not generally know exactly what they want
  • Many of the details needed to develop the software become clear only as the system is being developed
  • Product and project changes occur
  • People make mistakes

User Stories embrace change because

  • Do not need to be all written before developers can begin coding the first
  • Encourage deferring details because we can write them at whatever level of detail is appropriate in a given moment.

4. Prioritization

It’s hard to prioritize and work with thousands of sentences all starting with “The system shall…”.
That difficulty usually results in having the user considering 95% of the requirements high priority!

On the opposite, User Stories can be easily ordered by priority.


Mike Cohn