Rendering JSON from an XSLTListViewWebPart

I recently built a site on SharePoint 2010 that used lots of JavaScript for custom form validations.  The JavaScript made ajax calls back to listdata.svc to get the data to perform the validations. Things worked fine locally, but when a user from the other side of the globe accessed the site, things were slow.  A quick test using Fiddler’s ‘TimeLine’ tab showed me that after the initial page load, seconds were being spent making  multiple  ajax calls to get the validation data.

My first thought was to switch from rest to csom so that I could batch all the requests together and get the validation data in a single call. But then I began to think ‘Why not just send it down with the initial page?’ I thought about writing a custom webpart that could get data from an SPDataSource and serialize it to JSON and write it to the output stream.

Then it occurred to me that this could probably be done using an XSLTListViewWebPart and some custom XSL.  I found several pages that discussed serializing XML to JSON (Most notably http://controlfreak.net/xml-to-json-in-xslt-a-toolkit/ ).  Tweaking one of the samples in that article gave me the following XSLT which renders an XSLTListViewWebPart’s data into the browser:

<xsl:stylesheet xmlns:xsl=”http://www.w3.org/1999/XSL/Transform&#8221;
version=”1.0″ xmlns:ddwrt2=”urn:frontpage:internal”>
<xsl:param name=”webpartTitle”></xsl:param><!– need to add parameter binding <ParameterBinding Name=”webpartTitle” Location=”WPProperty(Title)” /> –>
<!– Turn off auto-insertion of <?xml> tag and set indenting on –>
<xsl:output method=”text” encoding=”utf-8″ indent=”yes”/>

<!– strip whitespace from whitespace-only nodes –>
<!–CAUSES ERROR Error while executing web part: System.Xml.Xsl.XslTransformException: White space cannot be stripped from input documents that have already been loaded. Provide the input document as an XmlReader instead. –>
<!– <xsl:strip-space elements=”*”/> –>

<xsl:template match=”/dsQueryResponse”>
<xsl:value-of select=”$webpartTitle” />
<xsl:text>&lt;script type=&quot;text/javascript&quot; &gt; debugger;</xsl:text>
<xsl:value-of select=”$webpartTitle” />
<xsl:text>=</xsl:text>

<xsl:apply-templates></xsl:apply-templates>
<xsl:text>&lt;/script&gt;</xsl:text>
</xsl:template>

<!– handles elements –>
<xsl:template match=”*”>
<!– element name –>
<xsl:text>{ &quot;e&quot; : &quot;</xsl:text>
<xsl:value-of select=”name()”/>
<xsl:text>&quot;</xsl:text>
<xsl:variable name=”ctr” select=”count(*)”/>
<xsl:variable name=”actr” select=”count(@*)”/>
<xsl:variable name=”tctr” select=”count(text())”/>
<!– there will be contents so start an object –>
<xsl:if test=”$actr > 0″>
<xsl:text>, &quot;values&quot; : { </xsl:text>
<xsl:apply-templates select=”@*”/>
<xsl:text>}</xsl:text>
</xsl:if>
<!– handle element nodes –>
<xsl:choose>
<xsl:when test=”$ctr = 1″>
<xsl:text>, &quot;data&quot; : </xsl:text>
<xsl:apply-templates select=”*”/>
</xsl:when>
<xsl:when test=”$ctr > 1″>
<xsl:text>, &quot;rows&quot; : [ </xsl:text>
<xsl:apply-templates select=”*”/>
<xsl:text> ]</xsl:text>
</xsl:when>
</xsl:choose>
<!– handle text nodes –>
<xsl:choose>
<xsl:when test=”$tctr = 1″>
<xsl:text>, &quot;$&quot; : </xsl:text>
<xsl:apply-templates select=”text()” />
</xsl:when>
<xsl:when test=”$tctr > 1″>
<xsl:text>, &quot;$&quot; : [ </xsl:text>
<xsl:apply-templates select=”text()” />
<xsl:text> ]</xsl:text>
</xsl:when>
</xsl:choose>
<xsl:text> }</xsl:text>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

<!– this template handle text nodes –>
<xsl:template match=”text()”>
<xsl:variable name=”t” select=”.” />
<xsl:choose>
<!– test to see if it is a number –>
<xsl:when test=”string(number($t)) != ‘NaN'”>
<xsl:value-of select=”$t”/>
</xsl:when>
<!– deal with any case booleans –>
<xsl:when test=”translate($t, ‘TRUE’, ‘true’) = ‘true'”>
<xsl:text>true</xsl:text>
</xsl:when>
<xsl:when test=”translate($t, ‘FALSE’, ‘false’) = ‘false'”>
<xsl:text>false</xsl:text>
</xsl:when>
<!– must be text –>
<xsl:otherwise>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”$t”/>
<xsl:text>&quot;</xsl:text>
</xsl:otherwise>
</xsl:choose>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

<!– this template handles attribute nodes –>
<xsl:template match=”@*”>
<!– attach prefix to attribute names –>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”name()”/>
<xsl:text>&quot; : </xsl:text>
<xsl:variable name=”t” select=”.” />
<xsl:choose>
<xsl:when test=”string(number($t)) != ‘NaN'”>
<xsl:value-of select=”$t”/>
</xsl:when>
<xsl:when test=”translate($t, ‘TRUE’, ‘true’) = ‘true'”>
<xsl:text>true</xsl:text>
</xsl:when>
<xsl:when test=”translate($t, ‘FALSE’, ‘false’) = ‘false'”>
<xsl:text>false</xsl:text>
</xsl:when>
<xsl:otherwise>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”$t”/>
<xsl:text>&quot;</xsl:text>
</xsl:otherwise>
</xsl:choose>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

</xsl:stylesheet>

The XSLT creates a JavaScript global variable with the same name as the webpart and writes the json data into that variable. Note that I needed to add an additional Parameter Binding:

<ParameterBinding Name=”webpartTitle” Location=”WPProperty(Title)” />

to the webpart so that the JavaScript variable could be named the same thing as the Webpart Title.  Also note that after the webpart is added to the page the ‘Enable Asynchronous Load’ option must be turned off, otherwise the JSON object will be written into an IFrame, and not easily accessible by the scripts on the main page.

Now I can just place these modified XSLTListViewWebParts on my forms page and the validation data I was previously retrieving via ajax calls is delivered with the initial page load and can be accessed using my custom JavaScript. No more ajax callbacks!

As an added benefit I can now filter the data send to the browser by editing the XSLTListViewWebParts, adding/removing columns, setting filter criteria etc.

Other possible uses of this technique could be to create views completely in JavaScript or the XSLT could easily be modified to write out Knockout observables. Also, if you Group the data in the Webpart, group data is even written to the JSON so you could render the groups on the client!

Below is the full source code for a webpart that serializes its data to JSON (Note that you should probably store the XSL in a library rather than inline)

<WebPartPages:XsltListViewWebPart runat=”server” Description=”mapping for contracts” PartOrder=”2″ HelpLink=”” AllowRemove=”True” IsVisible=”True” AllowHide=”True” UseSQLDataSourcePaging=”True” ExportControlledProperties=”False” IsIncludedFilter=”” DataSourceID=”” Title=”MAP_Contracts” ViewFlag=”8388613″ AllowConnect=”True” DisplayName=”JSON” FrameState=”Normal” PageSize=”-1″ PartImageLarge=”” AsyncRefresh=”False” ExportMode=”NonSensitiveData” Dir=”Default” DetailLink=”/sites/testing/sitenamehere/Lists/MAP_Contracts” ShowWithSampleData=”False” ListId=”7f481096-9449-48bd-b02d-e3b8dc62ef95″ ListName=”{7F481096-9449-48BD-B02D-E3B8DC62EF95}” FrameType=”Default” PartImageSmall=”” IsIncluded=”True” SuppressWebPartChrome=”False” AllowEdit=”True” ViewGuid=”{A3EE0A01-B7EB-407E-9FC3-4572CB34B207}” AutoRefresh=”False” AutoRefreshInterval=”60″ AllowMinimize=”True” WebId=”00000000-0000-0000-0000-000000000000″ ViewContentTypeId=”0x” InitialAsyncDataFetch=”False” GhostedXslLink=”main.xsl” MissingAssembly=”Cannot import this Web Part.” HelpMode=”Modeless” ID=”g_a3ee0a01_b7eb_407e_9fc3_4572cb34b207″ ConnectionID=”00000000-0000-0000-0000-000000000000″ AllowZoneChange=”True” TitleUrl=”/sites/testing/sitenamehere/Lists/MAP_Contracts” ManualRefresh=”False” __MarkupType=”vsattributemarkup” __WebPartId=”{A3EE0A01-B7EB-407E-9FC3-4572CB34B207}” __AllowXSLTEditing=”true” __designer:CustomXsl=”fldtypes_Ratings.xsl” WebPart=”true” Height=”” Width=””><ParameterBindings>
<ParameterBinding Name=”webpartTitle” Location=”WPProperty(Title)” />
<ParameterBinding Name=”dvt_sortdir” Location=”Postback;Connection” />
<ParameterBinding Name=”dvt_sortfield” Location=”Postback;Connection” />
<ParameterBinding Name=”dvt_startposition” Location=”Postback” DefaultValue=”” />
<ParameterBinding Name=”dvt_firstrow” Location=”Postback;Connection” />
<ParameterBinding Name=”OpenMenuKeyAccessible” Location=”Resource(wss,OpenMenuKeyAccessible)” />
<ParameterBinding Name=”open_menu” Location=”Resource(wss,open_menu)” />
<ParameterBinding Name=”select_deselect_all” Location=”Resource(wss,select_deselect_all)” />
<ParameterBinding Name=”idPresEnabled” Location=”Resource(wss,idPresEnabled)” />
<ParameterBinding Name=”NoAnnouncements” Location=”Resource(wss,noXinviewofY_LIST)” />
<ParameterBinding Name=”NoAnnouncementsHowTo” Location=”Resource(wss,noXinviewofY_DEFAULT)” />
</ParameterBindings>
<XmlDefinition>
<View Name=”{A3EE0A01-B7EB-407E-9FC3-4572CB34B207}” MobileView=”TRUE” Type=”HTML” DisplayName=”JSON” Url=”/sites/testing/GeraldFixPortal/Lists/MAP_Contracts/JSON.aspx” Level=”1″ BaseViewID=”1″ ContentTypeID=”0x” ImageUrl=”/_layouts/images/generic.png”>
<Method Name=”Read List”/>
<Query>
<GroupBy Collapse=”TRUE” GroupLimit=”30″>
<FieldRef Name=”Exchange”/>
</GroupBy>
<Where>
<Eq>
<FieldRef Name=”Category”/>
<Value Type=”Text”>CME_Futures</Value>
</Eq>
</Where>
</Query>
<ViewFields>
<FieldRef Name=”ID”/>
<FieldRef Name=”Category”/>
<FieldRef Name=”Cname”/>
</ViewFields>
<RowLimit Paged=”TRUE”>2000000</RowLimit>
<Aggregations Value=”Off”/>
<Toolbar Type=”Standard”/>
</View>
</XmlDefinition>
<DataFields>
</DataFields>
<Xsl>
<xsl:stylesheet xmlns:xsl=”http://www.w3.org/1999/XSL/Transform&#8221;
version=”1.0″ xmlns:ddwrt2=”urn:frontpage:internal”>
<xsl:param name=”webpartTitle”></xsl:param><!– need to add parameter binding <ParameterBinding Name=”webpartTitle” Location=”WPProperty(Title)” /> –>
<!– Turn off auto-insertion of <?xml> tag and set indenting on –>
<xsl:output method=”text” encoding=”utf-8″ indent=”yes”/>

<!– strip whitespace from whitespace-only nodes –>
<!–CAUSES ERROR Error while executing web part: System.Xml.Xsl.XslTransformException: White space cannot be stripped from input documents that have already been loaded. Provide the input document as an XmlReader instead. –>
<!– <xsl:strip-space elements=”*”/> –>

<xsl:template match=”/dsQueryResponse”>

<xsl:text>&lt;script type=&quot;text/javascript&quot; &gt; debugger;</xsl:text>
<xsl:value-of select=”$webpartTitle” />
<xsl:text>=</xsl:text>

<xsl:apply-templates></xsl:apply-templates>
<xsl:text>&lt;/script&gt;</xsl:text>
</xsl:template>

<!– handles elements –>
<xsl:template match=”*”>
<!– element name –>
<xsl:text>{ &quot;e&quot; : &quot;</xsl:text>
<xsl:value-of select=”name()”/>
<xsl:text>&quot;</xsl:text>
<xsl:variable name=”ctr” select=”count(*)”/>
<xsl:variable name=”actr” select=”count(@*)”/>
<xsl:variable name=”tctr” select=”count(text())”/>
<!– there will be contents so start an object –>
<xsl:if test=”$actr > 0″>
<xsl:text>, &quot;values&quot; : { </xsl:text>
<xsl:apply-templates select=”@*”/>
<xsl:text>}</xsl:text>
</xsl:if>
<!– handle element nodes –>
<xsl:choose>
<xsl:when test=”$ctr > 1″>
<xsl:text>, &quot;rows&quot; : [ </xsl:text>
<xsl:apply-templates select=”*”/>
<xsl:text> ]</xsl:text>
</xsl:when>
</xsl:choose>
<xsl:text> }</xsl:text>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

<!– this template handle text nodes –>
<xsl:template match=”text()”>
<xsl:variable name=”t” select=”.” />
<xsl:choose>
<!– test to see if it is a number –>
<xsl:when test=”string(number($t)) != ‘NaN'”>
<xsl:value-of select=”$t”/>
</xsl:when>
<!– deal with any case booleans –>
<xsl:when test=”translate($t, ‘TRUE’, ‘true’) = ‘true'”>
<xsl:text>true</xsl:text>
</xsl:when>
<xsl:when test=”translate($t, ‘FALSE’, ‘false’) = ‘false'”>
<xsl:text>false</xsl:text>
</xsl:when>
<!– must be text –>
<xsl:otherwise>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”$t”/>
<xsl:text>&quot;</xsl:text>
</xsl:otherwise>
</xsl:choose>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

<!– this template handles attribute nodes –>
<xsl:template match=”@*”>
<!– attach prefix to attribute names –>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”name()”/>
<xsl:text>&quot; : </xsl:text>
<xsl:variable name=”t” select=”.” />
<xsl:choose>
<xsl:when test=”string(number($t)) != ‘NaN'”>
<xsl:value-of select=”$t”/>
</xsl:when>
<xsl:when test=”translate($t, ‘TRUE’, ‘true’) = ‘true'”>
<xsl:text>true</xsl:text>
</xsl:when>
<xsl:when test=”translate($t, ‘FALSE’, ‘false’) = ‘false'”>
<xsl:text>false</xsl:text>
</xsl:when>
<xsl:otherwise>
<xsl:text>&quot;</xsl:text>
<xsl:value-of select=”$t”/>
<xsl:text>&quot;</xsl:text>
</xsl:otherwise>
</xsl:choose>
<xsl:if test=”position() != last()”>
<xsl:text>, </xsl:text>
</xsl:if>
</xsl:template>

</xsl:stylesheet></Xsl>
</WebPartPages:XsltListViewWebPart>

The Image below shows a sample JSON object called MAP_Contracts that was generated using this XSLT. Note that the generated object has an array called rows that contains field called values that contains the data. In this example I have grouped the view on Exchange so the fields Exchange.COUNT.group, Exchange.groupindex, Exchange.newGroup and exchange.urlencoded were added by the webpart and could be used to do grouping on the client side.

jaonvariable

Advertisements
Posted in ajax, javascript, sharepoint | Leave a comment

Automatically retry jquery ajax calls to listdata.svc after HTTP 500 Errors

I had an issue with a SharePoint page that was making  jQuery ajax calls to get the valid values of a Choice Column on a list. It worked fine  usually but once in a while the ajax calls would return an HTTP 500 Internal Server Error, and the user would see this:

error500

This was usually first thing in the morning, so I assume it happens when SharePoint is warming up.  The ULS Logs gave me no clues.   If the user refreshed the page, the error disappeared.

I found a nice jQuery plugin that will automatically retry  an ajax request after an error  here: https://github.com/johnkpaul/jquery-ajax-retry.

After adding that plugin, I just needed to change one line of code:

return $.ajax(ajaxOptions);

to

return $.ajax(ajaxOptions).retry({ times: 3, statusCodes: [500] });

and that fixed the problem.

Posted in ajax, javascript, jquery | Tagged , | 1 Comment

Getting started with Durandal on SharePoint

I recently saw a presentation on using Durandal to build Single Page Applications within a SharePoint-hosted app. You can download the Sildes from the presentation on Andrew Connels blog here). It looked promising, so I wanted to give it a go. The instructions here show how to create a Durandal SPA within a SharePoint-hosted app.

Open Visual Studio 2012 and select File/New/Project/SharePoint 2013 App

Make it app named SharepointDurandalSample

d1

Next, add Durandal to the project

Go to Tools/Library Package Manger/Package Manager Console

Within the Package Manager Console, enter the command:

Get-Package -ListAvailable Durandal

d2

This will display the packages available containing the text ‘durandal’:

We want the starter kit, so enter the command:

install-package Durandal.starterkit

This added a lot of stuff to the project as you can see:

d3

If we build the project now, we get an error on the DurandalController.cs. We don’t need it, since we’re using SharePoint as the backend, so we can just delete the Controllers Folder.

The pages/Default.aspx is the starting page for the SharePoint-hosted app that Visual-studio created for us.  It wants to start the app by calling /Scripts/App.js.  We’re going to change this below so we can delete the following line from Default.aspx:

<script type=”text/javascript” src=”../Scripts/App.js”></script>

We can also delete the file /Scripts/App.js (DO NOT MISTAKENLY DELETE SCRIPTS/DURANDAL/APPS.JS)

Replace the contents of the contentsPaceholderMain inside of Defaut.aspx with

<div id=”applicationHost”>

<div>

<div>

Durandal Starter Kit

</div>

<i></i>

</div>

</div>

<script type=”text/javascript” src=”../Scripts/require.js” data-main=”../App/main”></script>

The <div> whose id is applicationHost is where Durandal will display its output. The <Script> tag tells the browser to load require.js and the start App/Main.js. (I’m not sure about the majic that makes that happen.

If we try to run the app now, we get an error because Durandal wants knockout to be loaded on the page, so we need to add knockout to default.aspx. Add the following to paceholderadditionalpagehead within default.aspx:

<script type=”text/javascript” src=”../Scripts/knockout-2.3.0.js”></script>

Now if we run the app, we get an error that jquery is undefined:

d4

Durandal seems to want jquery 1.9.1, so in the contentplaceholderadditionalpagehead of Default.aspx,replace the tag:

<script type=”text/javascript” src=”../Scripts/jquery-1.8.2.min.js”></script>

with this one:

<script type=”text/javascript” src=”../Scripts/jquery-1.9.1.js”></script>

Running the app now, We see the Durandal sample starter page:

d6

In my case, clicking on the flicker link the first time gave me an exception. But the second time its worked. I’m not sure what’s happening, but I don’t care, I just want to get at some SharePoint data for now.

So let’s create a SharePoint list we can query . in Visual Studio select

Add/New Item/List

Make the list based on the Contacts template and name it My Contacts:

d7

Edit My Contacts/My Contacts Instance/Elements.xml to add some sample data

<?xml version=”1.0″ encoding=”utf-8″?>

<Elements xmlns=”http://schemas.microsoft.com/sharepoint/”&gt;

<ListInstance Title=”My Contacts”

OnQuickLaunch=”TRUE”

TemplateType=”10000″

Url=”Lists/My Contacts”

Description=”My List Instance”>

<Data>

<Rows>

<Row>

<Field Name=”FirstName”>Bill</Field>

<Field Name=”Title”>Gates</Field>

<Field Name=”Email”>bgates@microsoft.com</Field>

</Row>

<Row>

<Field Name=”FirstName”>Steve</Field>

<Field Name=”Title”>Ballmeer</Field>

<Field Name=”Email”>sbalmeer@yahoo.com</Field>

</Row>

</Rows>

</Data>

</ListInstance>

</Elements>

If you run  the app now you should be able to navigate to

https://yourappweb/SharepointDurandalSample/Lists/My%20Contacts/AllItems.aspx

To see the sample data, or go to

https:// yourappweb /SharepointDurandalSample/Lists/My%20Contacts/NewForm.aspx

To add new data.

Now we want to make our Durandal app display this list.

First, we need to modify the app object to remember the appUrl and the webUrl. Add the following function to the App/Main.js to get qyerystring parameters:

function getQueryStringParameter( param )

{

var params = document.URL.split( “?” )[1].split( “&” );

//var strParams = “”;

for ( var i = 0; i < params.length; i = i + 1 )

{

var singleParam = params[i].split( “=” );

if ( singleParam[0] == param )

{

return singleParam[1];

}

}

}

Then, just before appStart add the following code:

app.hostWebUrl = decodeURIComponent( getQueryStringParameter( ‘SPHostUrl’ ) );

app.appWebUrl = decodeURIComponent( getQueryStringParameter( ‘SPAppWebUrl’ ) );

app.requestDigest = $( ‘#__REQUESTDIGEST’ ).val();

(Please note that I am new to Durandal and no doubt this is not the best way to do this)

This will save these properties on the app object so we can use them later.

Next we need to create the ViewModel (or module in Durandal Teminology). In Visual Studio, within the App/viewmodels folder, Add/New Item/Web/Javascript. Name the file MyContacts.js and set the content as follows:

define( [‘plugins/http’, ‘durandal/app’, ‘knockout’], function ( http, app, ko )

{

debugger;

return {

displayName: ‘My Contacts’,

myContacts: ko.observableArray( [] ),

activate: function ()

{

//the router’s activator calls this function and waits for it to complete before proceding

var success = function ( data, status, jqXHR )

{

self.myContacts(data.d.results)

};

var options = {

dataType: “json”,

url: app.appWebUrl + “/_api/lists/getByTitle(‘My Contacts’)/Items”,

headers: {

Accept: “application/json;odata=verbose”

},

success: success

};

var self = this;

return $.ajax(options);

},

};

} );

Then create the view. Within the App/views folder, Add/New Item/Web/HTML File. Name the file MyContacts.html and set the content as follows:

<div>

<h2>Customers</h2>

<table>

<tbody data-bind=”foreach: myContacts”>

<tr>

<td data-bind=”    text: Title”></td>

<td data-bind=”    text: FirstName”></td>

<td data-bind=”    text: Email”></td>

</tr>

</tbody>

</table>

</div>

Finally, Add a route to the new  module inside of the activate function in App/viewmodels/shell.js

activate: function ()

{

debugger;

router.map([

{ route: ”, title:’Welcome’, moduleId: ‘viewmodels/welcome’, nav: true },

{ route: ‘contacts’, title: ‘contacts’, moduleId: ‘viewmodels/MyContacts’, nav: true },

{ route: ‘flickr’, moduleId: ‘viewmodels/flickr’, nav: true }

]).buildNavigationModel();

return router.activate();

}

Now if we run the app, the Contacts link works, and displays the contacts stored in the appweb,

The sourcecode can be downloaded here:http://code.msdn.microsoft.com/Sample-Sharepoint-Hosted-723d7a44

Additional info can be found  at the Durandal site, at  SPServices Stories #13: Durandal SP3: Developing SharePoint SPAs Made Easy, and at at  Durandal SP3: Developing SharePoint SPAs made easy

Posted in Durandal, sharepoint | Tagged , | Leave a comment

Troubleshooting issues with deactivating SharePoint(2010 SP1) Publising feature

I recently had an issue deactivating the Publishing Feature on a site. This document explains how I recreated the issue and eventually resolved it.

I created a new Blank Site called TP

I added a new document library called Images , and put a couple of files in it.

In SharePoint Manager the webs Lists and Folders look like this:

publishing1

Notice that the Root Folder for the Images library is called Images1. This is because there is already a folder called images under the webs RootFolder that gets created automatically when you create a blank site.

Before the next step, I went ahead and set the Trace Logging Level for Web Content Management/Publishing Provisioning to Verbose.

Now I go ahead and enable the Publishing feature on the Web, perhaps  because I want to share navigation with the parent Site. I see this in the trace logs:

Publishing Feature activation event handled.
Calling AreaProvisioner.Provision().
AreaProvisioner.Provision() called with with chromeUrl = ‘null’, welcomePageUrl = ‘null’, availableWebTemplates = ‘null’, availablePageLayouts = ‘null’, defaultPageLayout = ‘null’, newPageUrlToken = ‘null’, themedCssFolderUrl = , alternateCssUrl == null, versioningOnPages = MajorAndMinor, versioningOnDocuments = Major, versioningOnImages = Major, enableModerationOnPages = False, enableModerationOnDocuments = False, enableModerationOnImages = False, enableApprovalWorkflowOnPages = False, enableApprovalWorkflowOnDocuments = False, enableApprovalWorkflowOnImages = False, requireCheckoutOnPages = True, requireCheckoutOnDocuments = False, requireCheckoutOnImages = False, enableSchedulingOnPages = False, enableSchedulingOnDocuments = False, enableSchedulingOnImages = False, allowSpacesInNewPageName = False.
Adding list Url=’WorkflowTasks’, Title=’$Resources:cmscore,ListNameWorkflowTasks;’ for feature Id=’00bfea71-a83e-497e-9ba0-7a5c597d0107′
Adding list with Title=’$Resources:cmscore,ListNameWorkflowTasks;’, does not exist. Creating it now. Exception thrown was: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.AddList(SPListCollection lists, String urlName, String title, String description, Guid featureId, Guid[] previousVersionFeatureIds, Int32 templateType, Boolean& newListCreated)
Adding list with Title=’$Resources:cmscore,ListNameWorkflowTasks;’, does not exist. Creating it now.
Successfully added list Url=’WorkflowTasks’, Title=’$Resources:cmscore,ListNameWorkflowTasks;’ for feature Id=’00bfea71-a83e-497e-9ba0-7a5c597d0107′.
Setting NoCrawl on list ‘Workflow Tasks’ in web ‘http://servername/TP&#8217;.
Successfully set NoCrawl on list ‘Workflow Tasks’ in web ‘http://servername/TP&#8217;.
‘Workflow Tasks’ task library created.
Adding list Url=’Pages’, Title=’$Resources:cmscore,PagesListDisplayName;’ for feature Id=’22a9ef51-737b-4ff2-9346-694633fe4416′
Adding list with Title=’$Resources:cmscore,PagesListDisplayName;’, does not exist. Creating it now. Exception thrown was: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.AddList(SPListCollection lists, String urlName, String title, String description, Guid featureId, Guid[] previousVersionFeatureIds, Int32 templateType, Boolean& newListCreated)
Adding list with Title=’$Resources:cmscore,PagesListDisplayName;’, does not exist. Creating it now.
Successfully added list Url=’Pages’, Title=’$Resources:cmscore,PagesListDisplayName;’ for feature Id=’22a9ef51-737b-4ff2-9346-694633fe4416′.
Created Pages List using ListTemplate
Setting versioning on list ‘Pages’ in web ‘http://servername/TP&#8217; to ‘MajorAndMinor’.
Finished setting versioning on list ‘Pages’ in web ‘http://servername/TP&#8217; to ‘MajorAndMinor’.
Enabling multiple content types on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Successfully enabled multiple content types on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Start AreaProvisioner.RegisterWelcomePageEventsOnPublishingWeb()
RegisterWelcomePageEventsOnPublishingWeb registered ItemDeleting handler.
RegisterWelcomePageEventsOnPublishingWeb registered ItemFileMoving handler.
End AreaProvisioner.RegisterWelcomePageEventsOnPublishingWeb()
Begin AreaProvisioner.RegisterCPVEventsOnPagesList
RegisterCPVEventsOnPagesList registered ItemUpdated handler.
RegisterCPVEventsOnPagesList registered ItemCheckedIn handler.
RegisterCPVEventsOnPagesList registered ItemAdded handler.
RegisterCPVEventsOnPagesList registered ItemDeleting handler.
RegisterCPVEventsOnPagesList registered ItemDeleted handler.
End AreaProvisioner.RegisterCPVEventsOnPagesList
Replacing all content types on list ‘Pages’ in web ‘http://servername/TP&#8217; with content type ‘Page’.
Adding content type (Name=’Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39′) to list ‘Pages’.
Adding content type (Name=’Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39′).
Content type (Name=’Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39′) does not exist, adding it now.
Successfully added content type (Name=’Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39′).
Successfully added content type ‘Page’ to list ‘Pages’.
Successfully replaced all content types on list ‘Pages’ in web ‘http://servername/TP&#8217; with content type ‘Page’.
Adding content type (Name=’Article Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D’) to list ‘Pages’.
Adding content type (Name=’Article Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D’).
Content type (Name=’Article Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D’) does not exist, adding it now.
Successfully added content type (Name=’Article Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D’).
Successfully added content type ‘Article Page’ to list ‘Pages’.
Adding content type (Name=’Welcome Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4′) to list ‘Pages’.
Adding content type (Name=’Welcome Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4′).
Content type (Name=’Welcome Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4′) does not exist, adding it now.
Successfully added content type (Name=’Welcome Page’, Id=’0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4′).
Successfully added content type ‘Welcome Page’ to list ‘Pages’.
Adding content type (Name=’Folder’, Id=’0x0120′) to list ‘Pages’.
Adding content type (Name=’Folder’, Id=’0x0120′).
Content type (Name=’Folder’, Id=’0x0120′) does not exist, adding it now.
Successfully added content type (Name=’Folder’, Id=’0x0120′).
Successfully added content type ‘Folder’ to list ‘Pages’.
Updated start and end date SPField.Hidden values to ‘True’ on list ‘Pages’.
Adding field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Successfully added field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Adding field ‘aea1a4dd-0f19-417d-8721-95a1d28762ab’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Successfully added field ‘aea1a4dd-0f19-417d-8721-95a1d28762ab’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Adding field ‘0f800910-b30d-4c8f-b011-8189b2297094’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Successfully added field ‘0f800910-b30d-4c8f-b011-8189b2297094’ to view on list ‘Pages’ in web ‘http://servername/TP&#8217;.
No publishing approval workflow was found on list ‘Pages’ in web ‘http://servername/TP&#8217;.
Skipped associating the approval workflow with list ‘Pages’ because the feature parameters indicated it should not be associated.
‘Pages’ document library created.
Adding list Url=’PublishingImages’, Title=’$Resources:cmscore,ListNameImages;’ for feature Id=’4bcccd62-dcaf-46dc-a7d4-e38277ef33f4′
List with UrlName ‘PublishingImages’ or title ‘$Resources:cmscore,ListNameImages;’ already exists.
Successfully added list Url=’PublishingImages’, Title=’$Resources:cmscore,ListNameImages;’ for feature Id=’4bcccd62-dcaf-46dc-a7d4-e38277ef33f4′.
Adding field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Successfully added field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Adding field ‘9941082a-4160-46a1-a5b2-03394bfdf7ee’ to view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Field ‘9941082a-4160-46a1-a5b2-03394bfdf7ee’ on list ‘Images’ in web ‘http://servername/TP&#8217; does not exist, not adding.
Adding field ‘7e68a0f9-af76-404c-9613-6f82bc6dc28c’ to view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Field ‘7e68a0f9-af76-404c-9613-6f82bc6dc28c’ on list ‘Images’ in web ‘http://servername/TP&#8217; does not exist, not adding.
Adding field ‘1944c034-d61b-42af-aa84-647f2e74ca70’ to view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Field ‘1944c034-d61b-42af-aa84-647f2e74ca70’ on list ‘Images’ in web ‘http://servername/TP&#8217; does not exist, not adding.
Removing field ‘922551b8-c7e0-46a6-b7e3-3cf02917f68a’ on view on list ‘Images’ in web ‘http://servername/TP&#8217;.
Field ‘922551b8-c7e0-46a6-b7e3-3cf02917f68a’ on list ‘Images’ in web ‘http://servername/TP&#8217; does not exist, could not remove from view.
‘Images’ document library created.
Adding list Url=’Documents’, Title=’$Resources:cmscore,ListNameDocuments;’ for feature Id=’00bfea71-e717-4e80-aa17-d0c71b360101′
Adding list with Title=’$Resources:cmscore,ListNameDocuments;’, does not exist. Creating it now. Exception thrown was: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.AddList(SPListCollection lists, String urlName, String title, String description, Guid featureId, Guid[] previousVersionFeatureIds, Int32 templateType, Boolean& newListCreated)
Adding list with Title=’$Resources:cmscore,ListNameDocuments;’, does not exist. Creating it now.
Successfully added list Url=’Documents’, Title=’$Resources:cmscore,ListNameDocuments;’ for feature Id=’00bfea71-e717-4e80-aa17-d0c71b360101′.
Setting versioning on list ‘Documents’ in web ‘http://servername/TP&#8217; to ‘Major’.
Finished setting versioning on list ‘Documents’ in web ‘http://servername/TP&#8217; to ‘Major’.
Updated start and end date SPField.Hidden values to ‘True’ on list ‘Documents’.
Adding field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Documents’ in web ‘http://servername/TP&#8217;.
Successfully added field ‘3881510a-4e4a-4ee8-b102-8ee8e2d0dd4b’ to view on list ‘Documents’ in web ‘http://servername/TP&#8217;.
No publishing approval workflow was found on list ‘Documents’ in web ‘http://servername/TP&#8217;.
Skipped associating the approval workflow with list ‘Documents’ because the feature parameters indicated it should not be associated.
‘Documents’ document library created.
Start AreaProvisioner.InitializePublishingWebDefaults().
Adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
area.SetLayoutRelatedProperties() starts
MasterUrl = /_catalogs/masterpage/Gerald.master, inheriting = True
CustomMasterUrl = /_catalogs/masterpage/Gerald.master, inheriting = True
area.SetLayoutRelatedProperties() ends
PublishingWeb properties initialized to defaults.
End AreaProvisioner.InitializePublishingWebDefaults().
Begin AreaProvisioner.RegisterCPVEventsOnArea
RegisterCPVEventsOnArea registered WebMoving handler.
RegisterCPVEventsOnArea registered WebMoved handler.
RegisterCPVEventsOnArea registered WebDeleting handler.
End AreaProvisioner.RegisterCPVEventsOnArea
AreaProvisioner.Provision() determinted that current Web does not need Variation Spawning because Variations EnableAutoSpawning is OFF
Publishing Feature activation completed successfully.

Notice that line that says ‘List with UrlName ‘PublishingImages’ or title ‘$Resources:cmscore,ListNameImages;’ already exists’? When the Publishing feature is turned on it tries to create a Library titled ‘Images’ whose Root Folder is titled ‘PublishingImages’. It saw I already had a Library called images, so it skipped this step.

In Sharepoint manager the webs lists and Folders Look like this:

publishing2

Note the ID of the Pages Library (3487541a-893e-4051-b2d5-6bf7e0fac678).

If I now go into PowerShell and run the following script:

Add-PSSnapin microsoft.sharepoint.powershell
$web = get-spweb http://servername/TP
$web.allproperties

I see this:

publihing3

Note that the __PagesListId property of my web is set to the ID of the Pages Library that was created by activating the Publishing feature.

Now I need to save the Site as a template, so I need to deactivate the publishing feature. When I do that I see this:

publishing4

Looking at the Trace Logs I see this

Web Publishing Feature deactivation event handled.
Calling AreaProvisioner.Unprovision().
Adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://servername/TP&#8217;
Deleting list ‘Workflow Tasks’ from web ‘http://servername/TP&#8217; since it is empty.
Empty list ‘Workflow Tasks’ successfully deleted from web ‘http://servername/TP&#8217;.
Attempting to delete ‘WorkflowHistory’ from web ‘http://servername/TP&#8217; but the list was not found so it could not be deleted. Exception thrown was ‘System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteListIfEmpty(SPWeb parentWeb, String listUrl)’.
Deleting doclib ‘Pages’ from web ‘http://servername/TP&#8217; if empty.
Deleting list ‘Pages’ from web ‘http://servername/TP&#8217; since it is empty.
Empty list ‘Pages’ successfully deleted from web ‘http://servername/TP&#8217;.
Finished deleting doclib ‘Pages’ from web ‘http://servername/TP&#8217; if empty.
Deleting doclib ‘PublishingImages’ from web ‘http://servername/TP&#8217; if empty.
Publishing Feature DeActivation failed. Exception: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.CommonUtilities.GetDocLibByUrl(SPWeb web, String webRelativeUrl) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteDocLibIfEmpty(SPWeb web, String webRelativeUrl) at Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.Unprovision() at Microsoft.SharePoint.Publishing.PublishingFeatureHandler.FeatureDeactivating(SPFeatureReceiverProperties properties).

Note that it did delete the Pages Library, but the Deactivation failed because the ‘PublishingImages’ Library (or the Images1 folder) is not empty!

If I run the script to see the web properties now I see this:

publihing6

Note that the __PagesListId points to the list that just got deleted, and that the __PublishingFeatureActivated is set to False.

Back in Sharepoint Manager my site now looks like this:

publihing7

The Pages library got deleted, but not that Documents library or the WorkflowTasks list! Note that the site appears to be working fine at this point, and if one was not monitoring the logs, they may think that all is well. The ‘Save as Template’ link is still not available in the Site Settings, and if I go back in to Manage Site Features, the Sharepoint Server Publishing Feature appears to be inactive (al least the Activate button is enabled). So I click the Activate button to turn it on. I get No errors and the feature appers to be Active. Back in Sharepoint Manager my Site now Looks like this:

publishing8

Notice that it did not create a pages Library. If I run the script to see the web properties now I see this:

publisging9

So now it appears that the feature is active but the ‘__PagesListId’ web property points to a list that does not exist. Now if I deactivate the feature I see this:

publishing4

If I go back into Site settings/ Site features, the Sharepoint server publishing feature appears to be inactive (atleast the Activate button is enabled). Looking at the Event Logs I see this:

Web Publishing Feature deactivation event handled.
Calling AreaProvisioner.Unprovision().
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Publishing Feature DeActivation failed. Exception: Microsoft.SharePoint.Publishing.InvalidPublishingWebException: The site is not valid. The ‘Pages’ document library is missing. at Microsoft.SharePoint.Publishing.PublishingWeb.GetPublishingListWithCleanup(PublishingListType list, Boolean throwExceptionOnInvalidWeb) at Microsoft.SharePoint.Publishing.PublishingWeb.IterateOverAllPages(ProcessPublishingPage callToProcessPublishingPage) at Microsoft.SharePoint.Publishing.VariationsCleanup.RemoveVariationArtifactsForWeb(PublishingWeb pubWeb, Boolean recurse, SPList relationshipList, Boolean cleanWebProperties, Boolean cleanIsSourceHierarchy) at Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.c__DisplayClass4.b__3() at Microsoft.SharePoint.SPSecurity.c__DisplayClass4.b__2() at Microsoft.SharePoint.Utilities.SecurityContext.RunAsProcess(CodeToRunElevated secureCode) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(WaitCallback secureCode, Object param) at Microsoft.SharePoint.SPSecurity.RunWithElevatedPrivileges(CodeToRunElevated secureCode) at Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.Unprovision() at Microsoft.SharePoint.Publishing.PublishingFeatureHandler.FeatureDeactivating(SPFeatureReceiverProperties properties).

That last line says Publishing Feature DeActivation failed. Exception: Microsoft.SharePoint.Publishing.InvalidPublishingWebException: The site is not valid. The ‘Pages’ document library is missing. If you do a seacrch on that message, you’ll see a lot of writeups that say to create the Pages library (if it does not exist) and , run the Following powershell to set the ‘__PagesListId’ property of the web to the proper Value, and then Deactivate the feature.

$web = get-spweb http://servername/TP
$correctId = $web.Lists[“Pages”].ID
$web.AllProperties[“__PagesListId”] = $correctId.ToString()
$web.Update()

So I figure I’ll delete my Images Library, add a Pages Library,  and try again. But the ‘Delete this Library’ link is missing from my Images library settings… the Publishing feature has marked it as not Allowing Deletion:

publishing10

So I change it to Allow Deleition in SPM, save that in SPM, then delete my Images library. Then I create the Pages library, run the script to set the __PagesListId ,then Activate and Deactivate the feature. Again I get This:

publishing4

The ULS Show me this :

Adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Attempting to delete ‘WorkflowTasks’ from web ‘http://spdevtest/TP&#8217; but the list was not found so it could not be deleted. Exception thrown was ‘System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteListIfEmpty(SPWeb parentWeb, String listUrl)’.
Attempting to delete ‘WorkflowHistory’ from web ‘http://spdevtest/TP&#8217; but the list was not found so it could not be deleted. Exception thrown was ‘System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteListIfEmpty(SPWeb parentWeb, String listUrl)’.
Deleting doclib ‘Pages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting list ‘Pages’ from web ‘http://spdevtest/TP&#8217; since it is empty.
Empty list ‘Pages’ successfully deleted from web ‘http://spdevtest/TP&#8217;.
Finished deleting doclib ‘Pages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting doclib ‘PublishingImages’ from web ‘http://spdevtest/TP&#8217; if empty.
Publishing Feature DeActivation failed. Exception: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.CommonUtilities.GetDocLibByUrl(SPWeb web, String webRelativeUrl) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteDocLibIfEmpty(SPWeb web, String webRelativeUrl) at Microsoft.SharePoint.Publishing.Internal.AreaProvisioner.Unprovision() at Microsoft.SharePoint.Publishing.PublishingFeatureHandler.FeatureDeactivating(SPFeatureReceiverProperties properties).

I’m Still getting an error Deleing the PublishingImages Library. So Now I activate the Feature , create a Library called PublishingImages, and one called Pages,  run the script to set the __PagesListId then Activate and Deactivate the feature

And… it WORKS. No error s displayed. Better yet the ULS Logs Show me this:

Finished adding key-value pair to the web-property-bag for ‘http://spdevtest/TP&#8217;
Attempting to delete ‘WorkflowTasks’ from web ‘http://spdevtest/TP&#8217; but the list was not found so it could not be deleted. Exception thrown was ‘System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteListIfEmpty(SPWeb parentWeb, String listUrl)’.
Attempting to delete ‘WorkflowHistory’ from web ‘http://spdevtest/TP&#8217; but the list was not found so it could not be deleted. Exception thrown was ‘System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002) at Microsoft.SharePoint.Library.SPRequestInternalClass.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.Library.SPRequest.GetMetadataForUrl(String bstrUrl, Int32 METADATAFLAGS, Guid& pgListId, Int32& plItemId, Int32& plType, Object& pvarFileOrFolder) at Microsoft.SharePoint.SPWeb.GetList(String strUrl) at Microsoft.SharePoint.Publishing.Internal.Store.GetListByUrl(SPWeb web, String listUrlName) at Microsoft.SharePoint.Publishing.Internal.ProvisioningHelper.DeleteListIfEmpty(SPWeb parentWeb, String listUrl)’.
Deleting doclib ‘Pages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting list ‘Pages’ from web ‘http://spdevtest/TP&#8217; since it is empty.
Empty list ‘Pages’ successfully deleted from web ‘http://spdevtest/TP&#8217;.
Finished deleting doclib ‘Pages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting doclib ‘PublishingImages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting list ‘PublishingImages’ from web ‘http://spdevtest/TP&#8217; since it is empty.
Empty list ‘PublishingImages’ successfully deleted from web ‘http://spdevtest/TP&#8217;.
Finished deleting doclib ‘PublishingImages’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting doclib ‘Documents’ from web ‘http://spdevtest/TP&#8217; if empty.
Deleting list ‘Documents’ from web ‘http://spdevtest/TP&#8217; since it is empty.
Empty list ‘Documents’ successfully deleted from web ‘http://spdevtest/TP&#8217;.
Finished deleting doclib ‘Documents’ from web ‘http://spdevtest/TP&#8217; if empty.
Start AreaProvisioner.RemovePublishingWebProperties().
End AreaProvisioner.RemovePublishingWebProperties().
Web Publishing Feature successfully deactivated.

I can Activate and Deactivate the feature now with no errors reported in the ULS Log,

Posted in sharepoint | Tagged , , | Leave a comment

Timezone Issues when working with dates in SharePoint’s REST services

I was working on a SharePoint vacation planner  that would show employees down the left side of the page and Dates accross the top. The Sharepoint calendar does not provide such a format (as far as I know) so I decided to build it using javascript/jsRender and listdata.svc.  Users click on a cell in the Grid and it shows them a popup to enter info about there absence.

attendance

It worked great until a user  in  Asia accessed the calendar. All the dates were off by One day (i.e. a vacation day scheduled for July 5 would show up as july 4 when Viewed in Asia).

I queried my conrtent database with

SELECT *
FROM [AllUserData]
WITH(NOLOCK)
where [tp_ListId] =(
SELECT tp_ID
FROM [AllLists]
WITH(NOLOCK)
where [tp_WebId]  = (
SELECT  [Id] FROM [AllWebs] WITH(NOLOCK)where FullUrl = ‘HR/Attendance2’)
and tp_Title = ‘Absencses’)
and tp_DeleteTransactionId=0x

and found that the Vacation date was stored as 2013-07-05 04:00:00.000 when It was created from my custom Javascript UI but when I added a new vacation day via the Listforms it was stored as 2013-07-05 00:00:00.000.

As we know SharePoint stores all  its dates in UTC. So when I created a date in my Javascript code it was 2013-07-05 00:00:00 Eastern Time. When SharePoint got that date it converted it to UTC and hence I ended up with 2013-07-05 04:00:00.000 in my content database.

I created the following method to ‘Adjust’  my dates  before sending them to sharepoint so that they would be stored in the database  with the 00 hour

self.fromUtc = function (inputDate) {         debugger;         var localTime = inputDate.getTime();         var localOffset = inputDate.getTimezoneOffset() * 60000;         var utc = localTime – localOffset;         var retval = new Date(utc);         return retval;

};

Now when I created a new vacation date  from my custom Javascript UI it was properly stored  as 2013-07-05 00:00:00.000.

But I stll had issues. Although the dates were being stored properly in the database they were not appearing properly for my frien in Asia. All the days wer still off by one.  When the data above was sent to my browser via json ( it sends down the number of milliseconnds since Jan 1 1970 UTC)  javascript created the date Object in the users local Timezone (which was UTC -8:00). So my vacation day on 2013-07-05 00:00:00.000 was created in his browser as  2013-07-04 16:00:00.000  I created the following method to ‘Adjust’  my dates  after reading them from SharePoint so that they would be displayed ‘properly’ (i.e. as originally entered)

 self.toUtc = function (inputDate) {         var localTime = inputDate.getTime();         var localOffset=inputDate.getTimezoneOffset() * 60000;         var utc = localTime + localOffset;         var retval = new Date(utc);         debugger;         return retval;

};

Now my dates are stored and displayed as I expected.  It’s interesting to note that if SharePoint stored dates with  a time of 12 noon rather than  midnite, then when the timezone adjustment was made in javascript, the date would be correct still. This might make a nice custom Column. (Or maybe somebody has done it already?)

Also, I’m using datajs(http://datajs.codeplex.com/) to read the data from sharepoint. It has a setting called recognizeDates which automatically converts the dates in JSON. It would be nice if this feature were added to that library.

 

IMPORTANT NOTE: I learned after  deploying to production… the regional settings fro you site MUST be set to UTCC for this to work !!!

Posted in javascript, sharepoint | Tagged , , , , | 3 Comments

Using SPPersistedObjects to store custom SharePoint Configuration Settings

SharePoint Applications can store configuration information in a many places (PropertyBags, lists, PersitedObjects, web.config ). PropertyBags and Lists are both stored in the Content Database, so when you copy your production environment down to QA environment, these settings come with it. Persisted Objects are stored in the Configuration Database, so if you copy your production environment down to QA environment, these settings remain intact.

This can be useful in many situations, such as if you wanted your QA SharePoint server to always talk to  a QA version of some external server you are accessing(a QA report server, Web Server or some external Web Service) even after you refreshed SharePoint from Production. The persisted objects can be use most any SharePoint Project (Web parts, Timer Jobs ,event receivers, coded workflows … anywhere we can add code, we can use persisted Objects). If we don’t want the data overwritten when you copy Production to QA we should consider using them.

The nice thing about storing configuration information in a list is that you get a means to maintain that information via the SharePoint UI. While you don’t get a UI to maintain configuration information stored in Persisted objects, it is quite easy to do using PowerShell.

As an example, I have a Timer job that pulls new information from an external database every 5 minutes. I store key information about the Job in a class that inherits from SPPersistedObject.

public class AgressoImporterTimerJobParameters : SPPersistedObject

{

[Persisted]

public DateTime CustomersFromDate;

[Persisted]

public DateTime EntitiesFromDate;

[Persisted]

public DateTime VendorsFromDate;

[Persisted]

public DateTime CounterpartiesFromDate;

[Persisted]

public string ConnectionString;

[Persisted]

public Guid SiteID;

[Persisted]

public Guid WebID;

public AgressoImporterTimerJobParameters() { }

public AgressoImporterTimerJobParameters(string name, SPPersistedObject parent, Guid id)

: base(name, parent, id)

{

}

protected AgressoImporterTimerJobParameters(string name, SPPersistedObject parent)

: base(name, parent)

{

}

}

In the Execute method of the Timer Job the settings are retrieved by calling

 AgressoImporterTimerJobParameters settings = GetChild<AgressoImporterTimerJobParameters>(this.Name); // the parameters are storted as persisted objects under the job. Multiple jobs can eists for multiple sites, so jobname is used to differentiate

I can update the settings from within my timer job by calling

settings.CustomersFromDate = importer.LoadCustomers(settings.ConnectionString, customersList, settings.CustomersFromDate);

settings.Update();

If I need to update some settings from PowerShell, I can use the following script

[System.Reflection.Assembly]::Load(“XXXAgressoImport, Version=1.0.0.0, Culture=neutral, PublicKeyToken=b9c4d28743c1d3be”)

$job=[XXXAgressoImport.AgressoImporterTimerJob] (get-sptimerjob | where {$_.Name -match “XXX Agresso Importer Timer Job “} )

$farm = [Microsoft.SharePoint.Administration.SPFarm]::Local

$parameters = $farm.GetObject($job.name, $job.Id, [XXXAgressoImport.AgressoImporterTimerJobParameters])

$parameters.CounterpartiesFromDate=”3/22/2011 5:17:41 PM”

$parameters.CustomersFromDate=”3/22/2011 5:17:41 PM”

$parameters.VendorsFromDate=”3/22/2011 5:17:41 PM”

$parameters.EntitiesFromDate=”3/22/2011 5:17:41 PM”

#$parameters.ConnectionString=”Datasource= …..”

$parameters.Update()

You need to get the full name of the assembly using Reflector or a sililar Tool.

Posted in Powershell, sharepoint | Tagged , | Leave a comment

Turning SharePoint ListColumns into SiteColumns using CSOM

I had a Library with various columns defined at the list level and did not use site content types. I needed to begin using site content types to better manage the data in the library.   I needed a way to add the new site content types to the library and then change the existing items to the new content types without losing the metadata that was already defined on the Items.

I created the new site columns and content types and added the content types to the library. When I changed the content type of the existing items to one of the new content types, the metadata that was on the item was lost. SharePoint did not know that the field defined on the list called field1, should be moved to the new site column called as field1 in the new content type when I changed the content type of the item.

After researching a bit I realized that when I change the content type of an Item the field values are retained only if the fields are based on the same site column. The fields in my list were defined in the list itself, and were not site columns. Had the fields been site columns, and I used these site columns in my new Content types, the values would have been saved when I change content types.

With this lesson learned, I decided to create a quick WinForms app that would use the Client Object model to copy the original column to a new site column. It would:

  1. Create a temporary list column that was just like the original column
  2. Copy the value of the original list column to the temporary list column.
  3. Delete the original list column.
  4. Add a new Site Column with the same internal name and ID as the original column
  5. Add the new site Column to the list.
  6. Copy the value of the temporary list column to the new site column that was added to the list.
  7. Delete the temporary list column.

It’s worth noting that in my case I had some custom code already deployed that would break if the Internal Column name changed  so I needed to use the temporary column to hold the values while I created the new site column with the same Internal name and ID.

The solution worked fine. I could see the site column on the gallery, and any changes I made to the Site column also updated the list column, as would be expected. (The code can be found in the project on Codeplex).

While checking to make sure that everything was OK in the Object Model after I updated one of my list columns to be a site column, I used SharePoint manager to examine the SchemaXml of the list column both before and after the column was made into a site column. To my surprise, the two were identical! It seemed nothing had changed. I then looked at the SchemaXml of the site column in the Web Fields node in SPM and saw that the SchemaXml there was the same as the SchemaXml of the list column except that it did not have two properties:

  1. ColName – which determines what column in the content database the column is stored
  2. RowOrdinal – Which I think determines where the column is displayed by default within the row.

So the net effect of all the above was that I had just added a new entry to the web.Fields collection whose SchemaXml was the same as the SchemaXml of the List Column without the ColName and RowOrdinal!

Realizing this, I created a second method in the solution (on Codeplex) that just took the SchemaXml from the list column, removed the ColName and RowOrdinal attributes, and then added it to the Web Fields collection using AddFieldAsXml.

Sure enough, the Column appeared in the Web’s Column gallery and any changes made in the gallery update the field in my list, the list column was made a site column just by adding this entry to the seb.Fields collection.   It appears that the only thing that makes a column in a list a site column is that there happens to be an entry in the web’s Fields collection with the same ID.

The Project is located at http://code.msdn.microsoft.com/Convert-Sharepoint-28e28d4d

ListCol2SiteCol

To convert a column enter your Site Url and click Get Lists.

Select the List, then select the column. Check either ‘In Place’ or ‘Copy’

Copy will use the 6 step algorith described above to create a temporary column , etc…. It displays information in the Messages box explaining what its doing. NOTE: ALL items must be checked in to use this option as it will update every item in the list! (I think you may loose your last updated info as well…)

‘In Place’  just adds an entry to the Web’s Fields collection as noted above. I runs instantly no messages are displayed.

I’ve tested it on ‘Single Line of Text’ fields and Lookup Fields and ity worked fine. Other field types may need some Tweaking.

Be sure to test Thouroghly… Use at your Own Risk 🙂

Click ‘Make Site Column’ to convert the Columns

Posted in sharepoint, Uncategorized | Tagged , , , , | 4 Comments