Friday, October 26, 2012

Installing SharePoint 2013: Configuring the Firewall On the SQL Box With a Named Instance


There’s a lot of information out there on how to configure firewalls for SQL Server, but none really spell out exactly what is needed in order to get a connection with the minimal firewall settings. As a result, most people likely throw up their hands and turn off the firewall on the database server “Just to get it running”, and it’s left that way forever.

If you’re interested in getting the database connection for a named connection working with the firewall enabled, here are the steps to take, starting from the default configuration after installing SQL 2012:
  • Launch Sql Server Configuration Manager
  • Under SQL Services, SQL Server Browser is disabled and stopped. This service is required so incoming clients can get the proper port number for your named instance (by default, named instances use dynamic port numbers). Double click SQL Server Browser; On the Service Tab, set start mode to Automatic; on the Log On tab, start the service.
  • Expand SQL Server Network Configuration and select “Protocols for
  • Right Click TCP/IP and select “Enabled”
That is the configuration necessary to get remote connections working in the first place. Now for the firewall – we’ll use Windows Firewall with Advanced Security to create our rules allowing remote connections.

There are two rules you need to add – one for SQL Server itself and one for the browser service used to obtain the current dynamic port setting. For SQL Server:
  • Right Click “Inbound Rules” and select “New Rule…”
  • Select “Program” and click “Next”
  • Under “This program path” click “Browse…” and browse to C:\Program Files\Microsoft SQL Server\MSSQL11.\MSSQL\Binn\sqlservr.exe. Click Next. Note: Documentation I’ve seen for this say to go to MSSQL11.MSSQLSERVER instead of your named instance. This is wrong – that is for the default instance. To set up the firewall for your named instance, you must specify a path including MSSQL11.
  • Ensure “Allow the connection” is selected and click Next
  • Specify the profile. Likely, you’ll just need Domain for SharePoint – I tend to choose Domain and Private so I can connect from machines that aren’t on the domain. Click Next.
  • Name your rule – something like SQL Server for should suffice. Click Finish.
The second rule is for port SQL Browser, and required UDP port 1434:
  • Right click “Inbound Rules” and select “New Rule…”
  • Select “Port” and click “Next”
  • Select “UDP” and enter 1434 in the textbox next to “Specific local ports:”. Click Next
  • Ensure “Allow the connection” is selected and click Next
  • For Profile, match what you did above and click Next
  • Give this rule a name like “SQL Server Browser Service” and click “Finish”.
If you haven’t already, restart the SQL Server service for your named instance – that is necessary to get the TCP/IP protocol enabled earlier actually up and running – you would have received a warning about that when you first enabled it, so if you restarted it then you shouldn’t need to do it again.

Try your connection – hopefully it is successful and you have a secure-as-you-can-get-firewall-wise named instance of SQL 2012 for your SharePoint 2013 installation.

One more note since you’re likely reading this to configure your database for SP2013 – you need to have Max Degree of Parallelism set to 1 for you DB instance. SharePoint Adam provides an excellent explanation of just how to do that (and what happens if you don’t) here: http://sharepointadam.com/2012/07/20/sql-does-not-have-the-required-maxdegree-of-parallelism-setting-of-1/.

Friday, July 22, 2011

Refresh All Published Content Types On Next Update for All Site Collections.

Recently, I had an issue where some content types published to the content type hub did not get published on the first try. The reason is that I had site collections where the Document Set feature had not been enabled for the initial content type publish, so all content types derived from Document Set failed to be published.

It’s easy enough to activate the document set feature for all site collections, right? PowerShell. I can do that in one line:

   1:  (Get-spWebApplication <web app url>).Sites | % {enable-spfeature "DocumentSet" -url $_.url}

Done. But that’s not what this post is about.

Now that the Document Set feature has been enabled, I needed a way to republish the content types that failed. I didn’t want to modify the content types in any way, so my only real option was to go into the content type publishing settings for each site collection and click the “Refresh all published content types on next update” checkbox. This magically makes all the content types re-publish themselves to the site collection the next time the Content Type Subscriber timer job runs.

image

That’s all well and good if you have one or two site collections – go to each site collection, Site Actions –> Site Settings –> Content Type Publishing (under Site Collection Administration) –> Click the checkbox –> Click OK. It’s easy.

But I had a lot of site collections. This was my My Site web application. With personal site collections for a lot of users. 200 or so users. That’s a lot of clicking.

Unfortunately, I could find no apparent way to “click” this checkbox through PowerShell. There is no obvious member or property on either SPSite or SPWeb that seemed like it would do the trick. Frustrated, I turned to ILSpy.

ILSpy is what has replaced Reflector in my toolbox now that Reflector is no longer free. I highly recommend it. But this isn’t a post about ILSpy

The first step in trying to determine what SharePoint does when that checkbox is checked was to open the aspx file to find out where the code behind is located. Here’s the key line in ContentTypeSyndicationHubs.aspx in the Template\Layouts folder:

<%@ Page Language="C#" DynamicMasterPageFile="~masterurl/default.master" Inherits="Microsoft.SharePoint.Taxonomy.OM.CodeBehind.ContentTypeSyndicationHubsPage"       %>

Inherits attribute tells me the namespace I’m looking for, but not the assembly. First I checked Microsoft.SharePoint.dll, but when I didn’t find it there, I checked Microsoft.SharePoint.Taxonomy.dll (in the ISAPI folder). There I found what I was looking for:

image

So if the checkbox is checked, a method called Subscriber.RefreshAllTimeStamps is called for the current site. What does that function do? If I click on it, I get a warning message with the latest version of ILSpy (previous versions would just silently fail…)

image

With “Show internal types and members” selected under the View menu, I find this:

image

Ahhhhhh – now we’re getting somewhere. Even though this is an internal method that can’t be called from PowerShell, now that I know what I’m doing, it’s easy enough to convert:

Function RemoveAllTimeStamps([Microsoft.SharePoint.SPSite] $site)
{
    if ($site -eq $null) { return }
    $rootWeb = $site.RootWeb
    if ($rootWeb.Properties.ContainsKey("MetadataTimeStamp"))
    {
        $rootWeb.Properties["MetadataTimeStamp"] = [string]::Empty
       $rootWeb.Properties.Update()
    }
}

Now that we’ve got that, it’s just a matter of calling RemoveAllTimeStamps for each site collection in the web application.

$webApp = Get-spWebApplication <web app url>
 
$webApp.Sites | % {
    $site = $_
    RemoveAllTimeStamps $site
    $site.Dispose()
}
Now, simply run the Content Type Hub job, followed by the Content Type Subscriber job for your web application. If you have a lot of content types – be prepared to wait. I typed this up this entire blog post while my job went from 9% to 19%.

Tuesday, November 9, 2010

Creating Web Templates From Publishing Enabled Sites in SharePoint 2010

Creating web templates from sites using SharePoint 2010 and Visual Studio 2010 is a fairly straightforward process, so long as you don’t try to do it with a site that has publishing enabled. The basic steps are:

  • Save your site as a template with content included (Site Actions –> Site Settings –> Save Site as Template)
  • Download the saved template from the template gallery
  • In Visual Studio 2010, create a new project of type “Import SharePoint Solution Package
  • Run through the wizard, selecting your saved template as the wsp file when prompted.
  • Right click / Deploy

By default, that will create a sandboxed solution that you can then use to create sites within the site collection where the solution was deployed. With two simple changes, you can make it a farm solution that can then be used to create new site collections as well:

  • Change the Visual Studio solution to a farm solution – either chose Farm solution when creating the project or set the project’s Sandboxed Solution property to False after the project has been created
  • Change the scope of the Web Template Feature (Feature3 by default) to Farm.

Now, your deployed web template will show up in the site template gallery when creating any site or site collection.

Now, what if you want to perform this on a site with publishing enabled? You’ll hit several road blocks…

What do you mean publishing enabled?

First things first – to enable publishing on a site, you need to enable two features, one on the site collection, and one on the web. At the site collection scope (Site Actions –> Site Settings –> Site Collection Features), enable the SharePoint Server Publishing Infrastructure feature. At the web scope (Site Actions –> Site Settings –> Manage Site Features), enable the SharePoint Server Publishing feature.

At this point, modify the site the way you would like it to be for new sites based on the web template. That can include a custom master page, custom page layouts, new publishing pages, web parts, custom content types – anything that you want will (in theory) be included in your web template project.

Roadblock 1: Where did Save site as template go?

The first thing you’ll notice when you go to save your template is that the option for saving the site as a template is no longer there – it gets removed when you enable publishing. There may be a good reason for this – in MOSS for example, you ran the risk of a template getting deployed in another site collection not necessarily having all the content types you would need for the template to work properly. With Web Templates, though, it appears all that content gets stored with the template, so I'm not sure why the feature has been removed.

Before publishing is enabled:

image

After publishing is enabled:

image

There are several ways around this, including writing a feature to re-enable this option on the site settings page, but the easiest thing to do is just enter <site url>/_layouts/savetmpl.aspx in your browser’s address bar – that will take you right to the page that allows you to save your site.

Be sure to click “include content” so everything gets loaded into your SharePoint solution, including content types.

image

image

Once the template is created, click on the solution gallery link and save the template’s WSP file to your local disk.

Now go to Visual Studio and create a new project. The type will be “Import SharePoint Solution Package”. Give it a name and click OK.

image

When prompted for a deployment site, enter a site that you don’t care about. I would create a fresh site collection just for this purpose – the deployment will likely overwrite whatever is there.

image

Next, browse to your saved template file. Click Next.

image

Visual studio will then show you all the things it is about to import from your WSP file. You may be tempted to de-select some of the content, but if you do you’re likely to create errors in your solution that will need to be resolved, and the likely resolution will be to add the content back. Just leave everything selected and click finish.

image

image

Since we made this a Farm solution, we’ll need to modify the scope of the Web Template feature. For my publishing enabled team site, this is Feature 3:

image

You now have a visual studio project that is ready to be deployed and used to create new sites. Or so it would seem…

Roadblock 2: deployment error.

Project –> Right Click –> Deploy. Visual studio will alert you to many deployment conflicts – just let it resolve them automatically. You’re deploying to a site you don’t care about, remember?

image

There will be a lot of errors where SharePoint was unable to delete items to resolve conflicts. Most likely, the definition in the web template is a match for what already exists in SharePoint, namely standard list instances and content types. Nothing to worry about.

What you do need to worry about is the error message after the conflicts are “resolved”:

Error occurred in deployment step 'Add Solution': Dependency feature 'PublishingSite' (id: f6924d36-2fa8-4f0b-b16d-06b7250180fa) is not properly scoped for feature 'PublishingWebTemplate_Feature3' (id: 6fd24c8e-b684-42ab-9bbf-56759dfd5ea2). Its scope 'Site' must be equal to or higher than 'Farm'.

For some reason, when the WSP file was imported and Feature3 was generated, Visual Studio decided to make the SharePoint publishing site a dependency. This really makes no sense because the onet.xml file that was created will activate this feature, so there’s no reason for it to be a dependency.

image

To remove the dependency, double click Feature 3 to load the feature designer, scroll down to the bottom and expand “Feature Activation Dependencies, scroll down again, select the dependency and click “Remove”.

Now that that’s out of the way, right click-deploy again, and you will run into…

Roadblock 3: Deployment Error HRESULT: 0x80070002

File not found. This one has to be a bug. When you deploy, your error list look like this:

image

The warning was mentioned previously – Visual Studio couldn’t resolve all the conflicts. Again, no big deal. The problem is the error. Here, the output window offers no help.

  Deployment conflict resolution for one or more items failed. See the Output Window for details.
  Adding solution 'PublishingWebTemplate.wsp'...
  Deploying solution 'PublishingWebTemplate.wsp'...
Activate Features:
  Activating feature 'Feature1' ...
Error occurred in deployment step 'Activate Features': The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
========== Build: 1 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Deploy: 0 succeeded, 1 failed, 0 skipped ==========

It took me a while to figure out exactly what is causing this. The answer is included in the SharePoint ULS log files:

Cannot find doc C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Template\Features\PublishingWebTemplate_Feature1\Files\Variation Labels\AllItems.aspx    
Failed to add list view and form pages for list "Variation Labels" in web "http://homedev/sites/test". hr = 0x2ecf41d0    
Failed to create list "Variation Labels" in web "http://homedev/sites/test", HRESULT=0x2ecf41d0. List XML: "<List Title="Variation Labels" Direction="none" Url="Variation Labels" BaseType="0" Type="100" AllowDeletion="FALSE" NoCrawl="TRUE" BrowserFileHandling="Permissive" DisableDeployWithDependentList="TRUE" HiddenList="TRUE" FolderCreation="FALSE" Catalog="FALSE" SendToLocation="|" ImageUrl="/_layouts/images/itgen.png" xmlns:ows="Microsoft SharePoint" xmlns:spctf="http://schemas.microsoft.com/sharepoint/v3/contenttype/forms" xmlns="http://schemas.microsoft.com/sharepoint/"/>"    
Failed to instantiate list 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\Template\Features\CustomList\custlist'    
Unknown SPRequest error occurred. More information: 0x80070002    
Leaving Monitored Scope (List Creation: Variation Labels). Execution Time=70.0444709826517    
The element of type 'ListInstance' for feature 'PublishingWebTemplate_Feature1' (id: 14a09d46-89fb-4c97-bc3e-7797944b23d8) threw an exception during activation: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)    
Feature Activation: Threw an exception, attempting to roll back.  Feature 'PublishingWebTemplate_Feature1' (ID: '14a09d46-89fb-4c97-bc3e-7797944b23d8').  Exception: System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)     at Microsoft.SharePoint.Administration.SPElementDefinitionCollection.ProvisionListInstances(SPFeaturePropertyCollection props, SPSite site, SPWeb web, Boolean fForce)     at Microsoft.SharePoint.Administration.SPElementDefinitionCollection.ProvisionElements(SPFeaturePropertyCollection props, SPWebApplication webapp, SPSite site, SPWeb web, Boolean fForce)     at Microsoft.SharePoint.SPFeature.Activate(SPSite siteParent, SPWeb webParent, SPFeaturePropertyCollection props, Boolean fForce)    

Even armed with this information, it isn’t clear what needs to be done to resolve the issue.

Here’s what you need to do:

  • Expand “Other Imported Files\<templatename>\Files\Variation Labels”
  • Copy “AllItems.aspx” to “List Instances\Variation_Labels”
  • On the copied file, set the deployment type to “ElementFile” and Deployment Location path to “Files\Variation Labels” in the properties window (use the Schema.xml file in the same location as your guide).

Re-deploy. Now, at long last, it should deploy without an error. You can finally create a new site based on your new web template. Without further modification, it will be displayed on the “Custom” tab of templates:

image

Go ahead and create a site base on your new template – that should work now as well, but don’t celebrate yet…

Roadblock 4: Error Creating Publishing Pages

On your new site, try to create a publishing page. Select Site Actions –> More Options, select Page in the left hand column, click Publishing Page for the type, and then click Create. Boom:

image

Since one of the main reasons to have a web template for a publishing enabled site is to be able to, you know, create publishing pages, we’re going to need to figure this out too.

Searching for the correlation id in the log files yields the following:

System.ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.  Parameter name: index

Not very helpful, huh? While I can’t remember the exact steps I went through to troubleshoot this the first time, I can tell you that the answer lies in the way the content types are defined for the publishing pages. If you look at the elements.xml files for the master page gallery, you will find several instances where the PublishingAssociatedContentType property value is not set right. For example, Project Page reads like this:

<Property Name="PublishingAssociatedContentType" Value="Project Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8" />

It should be this:

<Property Name="PublishingAssociatedContentType" Value=";#Project Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8;#" Type="string" />

At first glance, you may not even notice the difference, but you’ll note that in the correct string, you’ll find number signs as a delimiter. That’s what’s missing, and that’s what’s causing the crash when you try to create a new publishing page.

I’m not sure what the best way is to provide this information in this format, but you’ll need to find these 5 property elements in your solution (one at a time):

 

<Property Name="PublishingAssociatedContentType" Value="Project Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8" />

<Property Name="PublishingAssociatedContentType" Value="Enterprise Wiki Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C" />

<Property Name="PublishingAssociatedContentType" Value="Redirect Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900FD0E870BA06948879DBD5F9813CD8799" />

<Property Name="PublishingAssociatedContentType" Value="Article Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D" />

<Property Name="PublishingAssociatedContentType" Value="Welcome Page, 0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4" />

and replace them with the correct values:

 

<Property Name="PublishingAssociatedContentType" Value=";#Project Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C0055EF50AAFF2E4badA437E4BAE09A30F8;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Enterprise Wiki Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF39004C1F8B46085B4d22B1CDC3DE08CFFB9C;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Redirect Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900FD0E870BA06948879DBD5F9813CD8799;#" Type="string" />

<Property Name="PublishingAssociatedContentType" Value=";#Article Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF3900242457EFB8B24247815D688C526CD44D;#" />

<Property Name="PublishingAssociatedContentType" Value=";#Welcome Page;#0x010100C568DB52D9D0A14D9B2FDCC96666E9F2007948130EC3DB064584E219954237AF390064DEA0F50FC8C147B0B6EA0636C4A7D4;#" Type="string" />

Be sure to replace the correct corresponding Property Element (i.e. Project page for project page). Use the find and replace with a scope of entire solution – some of these appear more than once.

Still with me? Didn’t think so, but I’ll plow on anyway, because that’s the last road block. Right click-deploy, create a new site, create a publishing page, everything should be working now.

Final steps: Customization

At this point, you should have a working web template solution, but there are a few other things you may want to clean up:

  • Edit the elements.xml file under the Web Templates folder to add a Display Category element. This puts your web template under a tab you name rather than the default “Custom” tab
  • While editing the elements file, give your web template a more appropriate Name and Title
  • Depending on your source web site, you may have path length issues. Visual Studio is pretty good about updating references when you rename folders, so you should be able to find folders that you can reduce (drastically in some cases) the length of the folder name to prevent path length issues.
  • Rename your features. Check the contents of the features to see which feature is doing what and rename appropriately.
  • Make your features hidden – there’s no reason for a user to interact with these features, they’re just used to help deploy content when you create sites based on your web template. Hide them to prevent confusion.

NOTE: one other issue I encountered when writing this was an error during deployment from Visual Studio:

Error occurred in deployment step 'Activate Features': Specified data type does not match the current data type of the property.

This doesn’t seem to affect the web template itself, it’s more a case of the site I’m using as a deployment target is in a state that no longer allows for successful activation of the features. I didn’t bother trying to resolve the issue since it didn’t keep me from successfully creating a web template I could use to create new sites, and the site I was using as a deployment target was just a throwaway anyway.

Wednesday, May 5, 2010

LaunchPickerTreeDialog in SharePoint 2010, and the Importance of ScopeToWeb

You know what I hate – when people start a blog, and then go months or years without publishing a topic. And then they come back and say “sorry I was gone, I’ve been busy, blah blah blah it won’t happen again – this time I’m really committed to updating this blog regularly” – whatever. I may take another 15 months off after this post, but here’s something interesting I just discovered about the behavior of the LaunchPickerTreeDialog javascript function included in SharePoint that I think is worth sharing here since it definitely falls under the “voodoo magic” category.

LaunchPickerTreeDialog – what it does

If you’re unfamiliar with LaunchPickerTreeDialog, others have written about it extensively so I won’t go into the details of how it works here. Google is your friend. I will tell you that it prompts the user to select a website, list, or document library using a graphic interface rather than forcing them to type in a URL. It’s well worth your effort to figure it out and use it in your solutions.

What’s New in SharePoint 2010?

So I’m testing some of my custom code on a site collection that was upgraded from SharePoint 2007 to SharePoint 2010, and I discover that when I call LaunchPickerTreeDialog on my site in SharePoint 2010, it is only showing lists and libraries from the current site. In SharePoint 2007, the same code would display all sub-sites as well in the tree view that the dialog box displays. What’s up with that?

image (click to enlarge)

Where’d my sites go? I know I have sub-sites, and I know they showed up in SharePoint 2007:

image (click to enlarge)

My first step towards figuring out what was going on was to load up IE8’s debugger and look at the script tab. The script I'm looking for is in PickerTreeDialog.js – that hasn’t changed from 2007. I put a breakpoint on the first line of code in the LaunchPickerTreeDialog function:

image (click to enlarge)

I was wondering if this was different from the 2007 version, so I did the same thing in my SharePoint 2007 environment. Here’s what that looked like:

image (click to enlarge)

Notice the difference? Aside from the obvious code difference inside the function, the signature changed in the 2010 version: has two new parameters:

  • ScopeToWeb
  • RequireCT

And now, in a blatant attempt to trick the search engines help people find this post, ScopeToWeb, ScopeToWeb, ScopeToWeb. That’s the key. What does this parameter do? What is the purpose? Where is it documented? No clue. But what I do know is that if you pass in an empty string to this parameter, you get your sub-sites. Pass in anything else, you don’t get your sub-sites. Hopefully someone will add in the comments exactly what this parameter is supposed to do, but for now, that’s all you need to know to fix your code.

Of course, if you’re still supporting SharePoint 2007, you can’t just go adding parameters to the LaunchPickerTreeDialog call because you’ll code won’t work in 2007. So you need to do a version check (blah) to make sure you’re only adding this parameter when you’re running on SharePoint 2010. There are several ways to do that – here’s what I chose to do:

   1: if (Microsoft.SharePoint.Administration.SPFarm.Local.BuildVersion.Major >= 14)
   2: {
   3:     // SharePoint 2010 has two new parameters - 
   4:     // ScopeToWeb and RequireCT. ScopeToWeb needs
   5:     // to be empty string to show sub-sites in the tree
   6:     sb.AppendLine("LaunchPickerTreeDialog('CbqPickerSelectListTitle','CbqPickerSelectListText','listsOnly','',serverUrl, '','','','/_layouts/images/smt_icon.gif','',callbackList,'','');");
   7: }
   8: else
   9: {
  10:     sb.AppendLine("LaunchPickerTreeDialog('CbqPickerSelectListTitle','CbqPickerSelectListText',");'listsOnly','',serverUrl, '','','','/_layouts/images/smt_icon.gif','',callbackList);");
  11: }

Wow, do I hate that. Someone rescue me and tell me there’s a better way in the comments. But in the meantime, if you’re running your LaunchPickerTreeDialog in SharePoint 2010 and trying to figure out why you can no longer see your sub-sites in the tree view, now you know – pass in an empty string for the ScopeToWeb parameter.

Thursday, February 19, 2009

Mounting a Virtual PC disk image with VHDMount in Vista

As a SharePoint developer, I do a lot of work on Virtual machines. I use VMware occasionally, but I mostly use Virtual PC or Virtual Server from Microsoft. Through the years, I’ve accumulated many, many virtual machine images – hard drives are cheap, so cleanup gets delayed at times.

Recently, I came across a need to get a file off of an existing VHD file for one of my virtual machines. I wondered if there was some kind of voodoo magic that would enable me to simply mount the VHD file on my local host machine without having to spin up the virtual machine to just get at a single file. The Google turned me on to this:

http://community.bartdesmet.net/blogs/bart/archive/2006/09/02/4385.aspx

Virtual server contains a command line utility called vhdmount that enables exactly what I want to do. That’s great, if you have virtual server installed. I do have virtual server installed on two different machines in my network, one x86, one x64, but the machine where the VHD file was located only had Virtual PC, and Virtual PC doesn’t come with the vhdmount command.

It turns out, that doesn’t matter – the vhdmount command that works in virtual server can be copied to a machine that has virtual PC installed, and from there it can be used to mount disks just like any machine that has Virtual Server installed. However, there’s an extra step involved…

To get vhdmount working on your Vista machine with only Virtual PC installed, do the following:

  • Copy the vhdmount folder from an existing virtual server installation. It will be located (by default) in c:\Program Files\Microsoft Virtual Server. I’d recommend copying it to c:\Program Files\Microsoft Virtual PC, but it should work from any folder. Be sure to match the bits – if you have x86 vista, copy the files from x86 virtual server.
  • Open up Computer Management (Click the Start Menu, right click “Computer” and select Manage).
  • In the left hand navigation tree, select “Device Manager”
  • From the menu, select “Action >> Add Legacy Hardware”
  • On the Welcome to the Add Hardware Wizard, click Next
  • Select “Install the hardware that I manually select from a list (Advanced) and click Next
  • Leave “Show All Devices” selected and click Next
  • Click “Have Disk”, browse to the Vhdmount folder (C:\Program Files\Microsoft Virtual PC\Vhdmount if you followed my advice earlier – select either INF file, it doesn’t matter), click OK
  • Under Manufacturer, ensure “(Standard system devices)” is selected. Under “Model”, ensure “Microsoft Virtual Server Storage Bus” is selected. Click Next.
  • Click Next to install the hardware. Be patient – it takes a while.
  • Close the wizard when finished.

If all goes well, you’ll get a success message and your new device should be properly installed under System devices in Device Manager.

image

Now, follow the instructions in the blog mentioned earlier and you should be able to mount a VHD drive in Vista without having to install virtual server.

Wednesday, February 11, 2009

Office and SharePoint – Follow Up

I just wanted to beat this topic into the ground follow up on my last post on how office applications interact with SharePoint in an extranet scenario. I created a VPC and ran through installations of all office versions from Office 2000 through Office 2007 to see how the different versions interact with SharePoint. I’ve summarized the results of my testing here.

The following table shows what happens when you click on an office document in a SharePoint document library in various office versions:

 

Word

Excel

PowerPoint

Office 2000

A

A

A

2002 (XP) No updates

B

B

C

2002 (XP) With Updates

B

B

B

2003 (No Updates)

D

D

D

2003 (NU, ActiveX Disabled)

B

E

E

2003 (With Updates)

F

F

F

2003 (Updates, ActiveX Disabled)

B *

B *

B *

2007 (No updates)

G

G

G

2007 (NU, ActiveX Disabled)

H

H

I

2007 (With Updates - Pre SP2)

G +

G +

G +

2007 (Updates, ActiveX Disabled)

H #

H #

H #

2007 (With SP2 Beta)

J

J

J

2007 (SP2 Bata, ActiveX Disabled)

K

L

L

NOTE: The add-on disabled for "ActiveX Disabled" above is the "SharePoint OpenDocuments Class (File: OWSSUPP.DLL)

KEY:

A: File Opens in browser without prompt
B: IE Prompt to open or save file, File opens in browser
C: IE Prompts to open or save file, application opens with authentication error, file does not open
D: Prompt "Some files can harm your computer…", 3 prompts for credentials, application opens but file doesn't open, 3-4 more prompts for credentials
E: Prompt to open or save, 3 prompts for credentials, cancel all 3 - file opens in browser
F: Prompt "Some files can harm your computer…", prompt for credentials, Cancel - authentication error, file does not open, Enter credentials - file opens in application
G: Prompt to open or edit, tries to open file in application, two authentication prompts, cancel both - file does not open. Credentials in either one, file opens in application
H: Prompt to open or save, 1 prompt for credentials, cancel, file opens in application
I: Prompt to open or save, 1 prompt for credentials, cancel, file does not open, enter credentials, file opens
J: Prompt to open or edit, tries to open file in application, four or five authentication prompts, file does not open. Enter credentials in one of the first three prompts, file opens
K: Prompt to open or save, 2 prompts for credentials, cancel, file opens. On close, another prompt for credentials that can be cancelled. Enter credentials on a prompt prevents future prompting
L: Same as K: without prompt on closing application
   
* Office 2003 may try to open the "Shared Workspace" task pane when opening a document in the app or the browser. This will prompt for credentials. Hit cancel - no access to shared workspace.
+ Office 2007 with vista - even if you enter credentials in the first prompt, you still get the second prompt
# Office 2007 with Vista, ActiveX Disabled - you are prompted twice and can hit cancel both times

So, for versions prior to Office 2003, office documents are designed to simply open inside the browser. This makes for a very nice experience, so long as you aren’t trying to edit the documents directly in the document library. For read only users, it would be desirable to have this experience, but alas, once Office 2003 or higher is installed, that doesn’t happen by default anymore. With Office 2003, you can disable the SharePoint OpenDocuments Class add-on to effectively turn off the document library integration features and restore Office XP open-the-document-inside-the-browser functionality:

  • In Internet Explorer, browse to a SharePoint document library. This ensures that the SharePoint OpenDocuments Class add-in gets loaded.
  • Navigate to Tools >> Manage Add-ons >> Enable or Disable Add-ons…
  • In the Show: list, ensure “Add-ons currently loaded in Internet Explorer” is selected.
  • In the list of Add-ons, select “SharePoint OpenDocuments Class”
  • Under Settings, select “Disable” and click OK
  • Click OK on the warning that you may have to restart your browser. You don’t – the effect if disabling the SharePoint OpenDocuments Class Add-on is immediate.

Note that if you do this, you lose the ability to interact with the SharePoint document library through your office applications. If you need to edit a document, either re-enable the add-on or you’ll have to edit locally and re-upload the document when done.

Note that disabling the add-on with Office 2007 doesn’t make documents open in the browser – they still open in the application. They even still prompt for credentials, even though there’s no benefit to entering credentials. This is the FrontPage check I wrote about previously.

For fun, I even tested the Office 2007 SP2 beta, and I kind of wish I hadn’t. There are even more credential prompts. I really feel sorry for the end user that just wants to read a document posted on SharePoint and has to constantly deal with all these credential prompts. Hopefully some of that will be cleaned up before SP2 is released.

One other surprise was that the prompting in Office 2007 behaves a little differently with Windows XP as the client rather than Vista. With XP, if you enter credentials in the first prompt, you don’t get prompted a second time. With Vista, you’re prompted twice no matter what.

Monday, February 2, 2009

Office Applications and their Interaction with SharePoint Document Libraries

One of the cool things about Office 2007 applications is that you can work with SharePoint document libraries from directly within the application. For example, if you click on, say, a PowerPoint document in a document library and open it directly without saving it first, Office will make a connection to the document library and give you additional functionality. Things like the Document Management task page, the enhanced save as dialog, etc. all use this voodoo magic to make your interaction with SharePoint seamless from with your application.

It’s all fun and games while everything is on the same side of the router – as long as you and your SharePoint server are all in an intranet scenario, everything works together nicely and seamlessly – you authenticate once against the SharePoint server, probably using the same domain credentials you used to log onto your own machine, open your document, work with it, save it, manipulate your document library, whatever – all without having to re-authenticate.

What happens when you and your SharePoint server are separated by a router – an extranet scenario? That’s where things get a little dicey. I recently helped set up such a system by exposing a SharePoint site to the internet using ISA Server 2006. I plan to go into detail on how I set this up in a future blog, but for now, know that ISA handles all the authentication using LDAP connections to active directory, and users are then authenticated to SharePoint using NTLM from ISA to the SharePoint server. The login is done with Forms authentication on ISA, and Office applications have no problem performing their own voodoo magic to recognize ISA’s form prompt and turn it into a regular windows prompt when the need arises.

One of the first things my client noticed during testing was that even though they had logged in to the system already, when they opened office documents, they were getting prompted for credentials again. Twice. That’s right – two prompts for credentials, after already logging into the SharePoint server.

Since this was the client and I did most of the ISA server configuration to get this site on the internet, it was up to me to figure out what was going on. As it turns out, that voodoo magic I was actually speaking of that works quite well on the intranet causes all kinds of extra network calls on the internet that are guaranteed to confuse users when they encounter them.

The first thing I did was turn on monitoring in ISA server to find out what was causing the prompts to appear. I tested with a PowerPoint document, but the same problem exists with all Office applications. I ran several experiments to try to figure out what was causing the two prompts to appear, with various combinations of entering credentials and canceling the prompts without entering credentials. Here’s an ISA log entry for the file trying to be retrieved on the first prompt:

Allowed Connection

Log type: Web Proxy (Reverse)

Status: 404 Not Found

Rule: My SharePoint Extranet Rule

Source: External (555.555.555.555)

Destination: (mysharepointsite.mycompany.com 777.777.777.777:443)

Request: POST http://mysharepointsite.mycompany.com/_vti_bin/shtml.exe/_vti_rpc

Filter information: Req ID: 08843636; Compression: client=No, server=No, compress rate=0% decompress rate=0% ; FBA cookie: exists=no, valid=no, updated=yes, logged off=no, client type=unknown, user activity=yes

Protocol: https

User: (LDAP)myusername

Additional information:
1.Client agent: MSFrontPage/12.0
2.Object source: Internet (Source is the Internet. Object was added to the cache.)
3.Cache info: 0x40000008 (Request includes the AUTHORIZATION header. Response should not be cached.)
4.Processing time: 16 ms

The first thing that caught my eye was the client agent: MSFrontPage. I thought we were done with FrontPage already with the rename to SharePoint Designer, but it turns out this was something else. Look at the request – it’s a POST for /_vti_bin/shtml.exe/_vti_rpc. The Google tells me this is a test to see if the site supports FrontPage Server Extensions. A little more research tells me that when Office applications access files in a SharePoint document library, the first thing they do is check to see if the server supports FrontPage extensions. In my case they don’t, but the file PowerPoint is trying to post to exists on a secured server, being accessed over the internet, from a machine not on the domain. Yep – that’s going to require credentials. And there’s no way to turn it off.

So fine, you enter your credentials, hit OK, and PowerPoint tries to open your file. But wait – you’re prompted again! “Didn’t I just enter my credentials” you say to yourself. And you’re right, you did, but Office wants to make extra sure you are who you say you are. Or something like that…

What Office is actually doing is making another connection to the server with a different Client Agent. The credentials from the failed FrontPage server extensions do nothing to help avoid this prompt:

Allowed Connection

Log type: Web Proxy (Reverse)

Status: 200 OK.

Rule: My SharePoint Extranet Rule

Source: External (555.555.5555)

Destination: (mysharepointsite.mycompany.com 777.777.777.777:443)

Request: OPTIONS http://mysharepointsite.mycompany.com /

Filter information: Req ID: 08843643; Compression: client=No, server=No, compress rate=0% decompress rate=0% ; FBA cookie: exists=no, valid=no, updated=yes, logged off=no, client type=unknown, user activity=yes

Protocol: https

User: (LDAP)myuseraccount

Additional information
1.Client agent: Microsoft-WebDAV-MiniRedir/6.0.6001
2.Object source: Internet (Source is the Internet. Object was added to the cache.)
3.Cache info: 0x42220008 (Request includes the AUTHORIZATION header. Response includes the CACHE-CONTROL: PRIVATE header. Response includes the CACHE-CONTROL: MAX-AGE or S-MAXAGE header. Response includes the SET-COOKIE header. Response should not be cached.)
4.Processing time: 47 ms

 

This is the actual magical connection that is going to allow Office to do it’s document library voodoo – good old WebDAV. Or a version of it anyway that appears to be specific to SharePoint.

Ok, so now that we know what the prompts are for, what about supplying credentials for what we’re actually trying to do – open a file. Remember the file? It turns out that you need to enter your credentials on one of the two prompts, it doesn’t matter which one, in order to access your file. The second prompt has the added bonus of giving you access to the document library, but either one will suffice for gaining access to the file. If you hit cancel on both dialogs, you won’t get access to your file.

Unless of course you’ve already successfully supplied credentials during your current session (since the last time you logged on to the computer you’re using to access your document). If that’s the case, your credentials are cached, and you can click cancel to both prompts and still get access to your file. I’ll wait while you scratch your head…

So, the first time you access a file, you have the following four combinations of options:

Cancel both dialogs No access to file or document library
Enter credentials in the first dialog, cancel the second Access granted to the file, not the document library
Hit cancel on the first dialog, enter credentials in the second dialog Access granted to the file, access granted to the document library.
Enter credentials in both dialogs Same as above – Access granted to the file, access granted to the document library

If you look over those options closely, something may jump at you – you can always click cancel on the first prompt. If you get prompted a second time, enter your credentials there. I say If you get prompted a second time, because that second prompt may not be necessary if you’ve already entered your credentials during your current session.

Another way to avoid having to enter credentials into that second prompt is to add an entry in your network passwords:

  • Go to control panel >> user account >> manage your network passwords.
  • Click Add
  • Enter the name of your SharePoint server and your user credentials
  • Select “A web site or program credential

Note these are instructions for Vista, but I believe something similar works for XP.

You’ll still get that first prompt when Office tries to determine if FrontPage extensions exist on the server. If anyone knows how to get rid of that prompt, please let me know.