Presenting at SUGDC annual users conference

I am speaking at the SUGDC annual users conference on June 27th. It should be a good event with a lot of good speakers!

My presentation isn’t on my usual development topics. I am switching it up this time and presenting on the administrative/setup track. My topic is on using Site Collections vs Sites in SharePoint. Yes, I know it doesn’t sound interesting, but I promise I will try to keep everyone interested. I do feel it is one of the most important topics to understand in SharePoint. Almost all the issues I see when I go into my clients stem from an incorrect topology and permission setup. We will cover these situations with a fictitious company walk-through to see how they messed up, what they can do to correct their mistakes and what considerations they need to make.

Below is more information on the conference as a whole:

The SharePoint Users Group of Washington D.C.’s  (SUGDC) Third Annual Regional SharePoint Users Conference is taking place at the Holiday Inn Dulles June 26-27.  With over 40 speakers (including keynoter Bob Mixon), 15 SharePoint software, consulting and resource exhibitors, and four thematic tracks appealing to experts and non-experts alike, this year’s conference is shaping up to be an exciting event!  Anyone interested can check out the conference agenda, and pre-register at the extended early bird rate of $299.

Developing SharePoint WebParts using User Controls and Web Applications

If you’ve read my blogs before, then you probably know I am a fan of WSPBuilder (http://www.codeplex.com/wspbuilder). I like the intuitive nature and flexibility of the product. It really helps with the deployment aspects of SharePoint features and functionality. However, in the end, it is really just a structured way to create a deployment/feature project that will create the wsp install file for SharePoint. It really doesn’t help much when building UI functionality. For example: if you want to build a Web Part in SharePoint, you still have to build the Web Part code out programmatically (instead of using the WYSIWYG features of Visual Studio). The same issue comes up with building master pages, application pages or anything that requires html and a code behind.

In this article I am going to show you how you can utilize the flexible nature of WSPBuilder, ASP.Net Web Applications and post build scripts in order to utilize WSPBuilder as your deployment project for your UI. 

Solution Overview

SharePoint is a dynamically generated website that pulls information out of a database. It also utilizes files on the server and uses those files as templates or actionable files. The combination of these static files on the server and the information in the database creates the web page we see. This architecture is what allows us to create pages, add webparts, modify navigation, etc… all within the SharePoint site itself.

So, when developing against SharePoint, we need to deploy files to this static place on the server and register the files in the database. This static place on the server is called the 12 hive. It is usually found at: C:\Program Files\Common Files\Microsoft Shared\web server extensions\12. A lot of stuff goes on in the 12 hive. You have the templates for the website, images, themes, etc… You also have very special items called Features. Features allow us to deploy custom functions to SharePoint and then activate/deactivate them at our leisure.

WSPBuilder allows us to “mimic” the 12 hive, within a Visual Studio project. As long as you have the same 12 hive structure setup, it can create the SharePoint deployment file (i.e.: the wsp). You do not need WSPBuilder to create wsp files; you can do the same thing by building extra files in your solution called manifest.xml and ddf files. However, for rapid application development, it is easier to use a third party solution like WSPBuilder because it creates those extra deployment files for you.

While WSPBuilder is a great tool to help us build the deployment files, it is not a web application project in Visual Studio. Web application projects help us build code behind and designer files for our server side controls.

Thus, the ideal solution for building UI elements in SharePoint consists of:

  1. ASP.Net Web Application project to build the UI elements
  2. WSPBuilder project to create our deployable wsp file

The ASP.Net Web Application project will contain the UI elements (such as User Controls). The build process will move the appropriate elements from the ASP.Net Web Application project to the WSPBuilder deployment project. Then the deploy will move the files from the wsp to the SharePoint Server.

Deploy

The key to this solution is seperation of concerns. You should build all UI related functionality in the ASP.Net Web Application project. All SharePoint specific functionality (ex: features), should be built in the WSPBuilder project.

 

Create the WSPBuilder deployment project

  1. Create your project in Visual Studio (File – New – Project)
    • Choose the WSPBuilder project. I am utilizing the one under c# for this example.
    • Give it a good name. I am using DemoProject for this example.
    • Make sure it creates a new solution when creating the project.
  2. Add a folder under the 12 folder called “Template”.
  3. Add a folder under the Template folder called “LAYOUTS”.
  4. Add a folder under the Template folder called “FEATURES”.
  5. Add a folder under the Template folder called “CONTROLTEMPLATES”.
  6. Add a folder under the project called “GAC”.

Note: The “GAC” folder in WSPBuilder is a special folder. We can place external dlls in this folder the the resulting WSP will deploy those dlls to the GAC for us.

DemoProject

 

Create the UI project

As I mentioned in the beginning of this article, the point is to create our UI elements in an ASP.Net Web Application project. So, we need to create another project, in the same solution, so that we can develop our UI elements.

  1. Create the UI project (File – Add – New Project)
    • Choose the ASP.NET Web Application Template. I am utilizing the one under c# for this example.
    • Give it a good name. I am using DemoProjectUI for this example.
  2. Delete the Default.aspx
  3. Sign the project (this is because we are going to deploy to the GAC)
    • Go to the properties (right-click the project and choose properties).
    • Go to the Signing tab.
    • Choose “Sign the assembly”.
    • Under the “Choose a strong name key file” – choose <New>
      • Give it a strong name – I usually use the name of my project (for example: DemoProjectUI).
      • Uncheck “Protect my key with a password”.
  4. Add in the post build commands
    • Go to the properties (right-click the project and choose properties).
    • Go to the Build Events tab.
    • Add the following into the post build section:
      xcopy "$(TargetPath)" "$(SolutionDir)DemoProject\GAC\" /Y /s
      xcopy "$(ProjectDir)*.ascx" "$(SolutionDir)DemoProject\12\TEMPLATE\CONTROLTEMPLATES"\" /s /Y /R

Let’s recap what we did in the steps above. First we created a WSPBuilder project called DemoProject. This is our deployment project. It will create wsp files that we can deploy to SharePoint. Then we created an ASP.Net Web Application project called DemoProjectUI. This is where we will create all our UI elements. This will allow us to create user controls with html and code behind files. Lastly, we made sure that we moved the dll and ascx files, from the DemoProjectUI project, to the appropriate place in the DemoProject project.

 

Create a Web Part

Our next step is to create a Web Part. As anyone who has developed in SharePoint before knows, Web Parts are complete code files. They are not the html with code behind files we are used to when developing in ASP.Net. Some people are fine with developing Web Parts completely programmatically. However, it is much easier to create UI elements when you have WYSIWYG editors, html and code behind.

One way to get the normal ASP.Net web development experience, when developing Web Parts, is to use the SmartPart. The SmartPart is a very clever Web Part, developed by Jan Tielens, which can render .Net user controls in Web Parts. I really like the SmartPart, especially for people learning to build SharePoint Web Parts as user controls. However, I like more control over what I do and there are some limitations to the SmartPart. It is not my intent to go over those limitations in this article, but you can read them here: http://weblogs.asp.net/erobillard/archive/2008/03/04/what-to-know-about-smartpart-and-loadcontrol.aspx

In the end, you can accomplish the same thing as the SmartPart using the “LoadControl” method in .Net. Thus, this article will show how to create a Web Part, which will load the user control from our UI project.

  1. Create the Web Part using WSPBuilder
    • Right-click on the DemoProject project
    • Go to Add – New Item
    • Choose WSPBuilder – Web Part Feature
    • Give it a good name. For this example I am going to use DemoFeature
    • A popup will come up with Title, Description and Scope. Since we are developing a Web Part, you must choose “Site” for the scope. This is because we need the Web Part to deploy to the Web Part gallery of our Site Collection.

    Notice that WSPBuilder did two things for you:
    - It created the feature in the features folder
    - It created the Web Part code in a folder called WebPartCode

  2. Modify the Web Part code to use “LoadControl”
    • Open up the DemoFeature.cs file in the WebPartCode folder
    • Remove the MyProperty property and attribute for now. This is just WSPBuilder showing you how you can use properties. We aren’t going to use them for this demo.
    • Find the CreateChildControls method and find the comment that says “Your code here…”
      • Remove the line under it.
      • Replace it with this: this.Controls.Add(Page.LoadControl(”~/_controltemplates/DemoControl.ascx”));

      Your CreateChildControls method should look like this:
      protected override void CreateChildControls()
      {
          if (!_error)
          {
             try
             {
                base.CreateChildControls();
                this.Controls.Add(Page.LoadControl("~/_controltemplates/DemoControl.ascx”));
             }
             catch (Exception ex)
             {
               HandleException(ex);
             }
          }
      }

  3. Add the control to the UI project
    • Right-click on the DemoProjectUI project
    • Go to Add – New Item
    • Choose Web – Web User Control
    • Give it a good name. For this example I am going to use DemoControl.ascx

Now, when you build your solution, the DemoControl.ascx will move to the ControlTemplates folder in the DemoProject project. The SharePoint Web Part will look for the control by using the _controltemplates path.

Note: SharePoint can find any control in the ControlTemplates folder by using the _controltemplates path because of a mapping it creates in IIS. SharePoint maps the _controltemplates path to the servers 12 hive at 12/Template/ControlTemplates.

 

Utilizing the Code Behind

We now have our basic solution setup. We have our UI project and can build our user control there. We have our WSPBuilder deployment project that will create our SharePoint install file. But, we aren’t ready to deploy just yet. We still need to tell our user control how to talk to its code behind. Because we are utilizing the GAC for our assemblies, we need to put a fully qualified domain in our ascx file. There are a couple of techniques for figuring out this fully qualified domain. What I like to do is deploy the project and go to the GAC to get the properties.

  1. Deploy the WSP, so that the assembly gets added to the GAC, so that we can pull out the assembly information.
    • Right-click on the DemoProject project
    • Click WSPBuilder – Build WSP (wait for it to finish)
    • Right-click on the DemoProject project
    • Click WSPBuilder – Deploy WSP (wait for it to finish)
    • Get the assembly information from the GAC
      • Usually found at C:\Windows\assembly
      • Right-click on the DemoProjectUI assembly and click properties
      • Note the public key token and version ( I suggest copying the public key token at this point because we are going to use this information in the next step).
  2. Add the assembly information to the ascx
    • Go to the DemoControl.ascx file in the DemoProjectUI project
    • Add an Assembly reference as the first line in this file. Below is an example. However, you cannot copy this example. Your assembly reference must have the correct information about your assembly (including your public key information).

<%@ Assembly Name =”DemoProjectUI, Version=1.0.0.0, Culture=neutral, PublicKeyToken=772ab5f02712b819″%>

 Now we are ready to test our solution!

 

Deploy the Solution

Now that we are finished setting everything up, we can deploy the solution. This is real easy with the use of WSPBuilder as long as we are developing on a SharePoint server.

Note: Please make sure you have a web application and site collection setup in SharePoint before doing the following steps. When you deploy the WSP solution, it will deploy to all web applications on your development server. Thus, the web application must exist before you deploy.

  1. Right-click the DemoProject project
  2. Click WSPBuilder – Build WSP (wait for it to finish)
  3. Right-click on the DemoProject project
  4. Click WSPBuilder – Deploy WSP (wait for it to finish)
  5. Open up your SharePoint site.
  6. Note: if you go to the SharePoint site at this point and it says “Server Not Available”, then just keep refreshing the page until it shows up. The WSP install recycled the application pools because it needs to every time a dll in the GAC is added or modified. It sometimes takes a few seconds for this process to finish.

  7. Go to Site Actions – Site Settings
  8. Go to Site Collection Features
  9. Activate the DemoFeature
  10. Go back to the site
  11. Go to Site Actions – Edit Page
  12. Click “Add a Web Part” in one of your web zones
  13. Find your WebPart. It should be under “MyGroup” unless you changed the group name in your feature. The elements.xml file in the feature folder of the DemoProject lets you configure this information. I
  14. Add your Web Part to the page

At this point your Web Part is empty because we didn’t add any html or code to it. However, you solutions architecture is ready to go. Now you can go back to your User Control, do your development, deploy the solution and it will show up on your page. Let’s give it a try.

  1. Go to the DemoProjectUI project
  2. Open the DemoControl.ascx
  3. Put a label tag on the control
  4. <asp:Label runat=”server”  ID=”test” />

  5. Go to the code behind (i.e.: DemoControl.ascx.cs)
  6. In the Page_Load type the following
    protected void Page_Load(object sender, EventArgs e)
     {
          test.Text = "Hello World";
     }
  7. Right-click the DemoProject project
  8. Click WSPBuilder – Build WSP (wait for it to finish)
  9. Right-click on the DemoProject project
  10. Click WSPBuilder – Deploy WSP (wait for it to finish)
  11. Open your SharePoint site and see your Web Part. It should say “Hello World”.

Remember: if it says “Server Not Available”, then keep refreshing the page until the site shows up again.

Wow, that was a lot of setup just to create a user control that can render in a Web Part. But, the great thing is, you only have to do the setup once. As a SharePoint Architect, I set this up for my team and they can just concentrate on building the user controls. It runs very smooth!

Hopefully I can find some time to write another article in which I can expand on this solution. I could show how to utilize images correctly in this scenario. Or, I could show how to do the same thing with application pages. I’ll have to think about what I want to show next.

‘IsReference’ property specified was not found

The other day I got an error message that said “’IsReference’ property specified was not found”. I started doing some research on it and most people were referring to WCF. The interesting thing in my case was:
1. I wasn’t using WCF
2. This wasn’t happening on the development environment. It only happened when we moved to staging.

So, I had to start figuring out what was going on. I narrowed the issue down to DataContractSerialization. Basically, the situation I was dealing with was that I was serializing Linq to Sql objects. If you have worked with Linq to Sql before, you would know that they are not easily serialized. In fact, they are not marked with the serializable attribute. Instead, they are marked with the Data Contract attribute. So, if you want to serialize them for any reason (my reason was that I wanted to store them in Session), then you have to use the DataContractSerializer to do the serialization. The DataContractSerializer was introduced for WCF – thus the reason why most of the references I saw about this error were talking about WCF.

So, now that I narrowed the issue down to my serialization, I had to figure out what was happening. I noticed that my Linq to Sql objects were marked with the Data Contract attribute and a property called “IsReferenced”. Ok, now I know this marking is causing the issue. But, why was this issue working in my environment and not staging.

Well, the answer ended up being the version of .net. I was told that the staging servers were upgraded to .net 3.5 sp1. However, when I looked closely, I noticed that one of the front end servers (in our case there were many front end servers) was only .net 3.5. That’s what I get trusting IT guys, I have to double check everything they do :) .

It turns out that even though the DataContractSerializer was around in the .net 3.5, the marking of “IsReference” wasn’t introduced until .net 3.5 sp1. So, if you get this error, you might want to check out your version of .net before starting to debug.

Cleaning up your WebPart Gallery

Deploying webparts in SharePoint can be difficult (when you are learning) because of the many ways to do it. You can manually add webparts to the WebPart gallery, you can add WebParts through features and you can even manually add them to certain file system folders.

If you have developed extensively in SharePoint, you probably choose the feature way to deploy WebParts. I am not going to go into this conversation right now, but it is well known that the best practice for building webparts is to deploy them with features. However, one thing that nobody really talks about is removing the WebPart from the WebPart Gallery. If a WebPart is placed in the WebPart gallery when the feature is activated, shouldn’t it be removed from the WebPart gallery when the feature is de-activated? In this article we are going to go over the basics of how WebParts are added to the WebPart gallery during activation of the feature and then we are going to show how you can clean up your gallery when the feature is deactivated.

Overview of WebParts and Features

The premise of WebParts is simple – you are adding an xml file to a special list in SharePoint. This special list is called the Web Part Gallery and the xml file you add is called a .webpart file or a .dwp file. The .webpart or .dwp file “describes” the properties of the WebPart and points to a code file where the .net code runs that builds the WebPart controls. If you want to visit this special WebPart Gallery you can go to your SharePoint site and put “/_catalogs/wp” after your url (i.e.: http://server:port/_catalogs/wp). If I have already confused you at this point, you probably want to start looking into some beginner guides to WebParts in SharePoint. I will do a small overview of WebParts in this article (that might be helpful for beginners), but the point of this article is to show some advanced techniques to how to clean up your WebPart Gallery.

When deploying webparts in SharePoint I always use a Feature to deploy the WebPart to the WebPart gallery. Then I always wrap these features up in a WSP solution in order to deploy a single solution to my SharePoint farm. This is best practice and is really easy once you have done it a couple of times. Personally, I choose to use WSPBuilder to build my WSP solutions (but the process is the same if you use Visual Studio Extensions or if you build your WSPs manually).

If you are using WSPBuilder you can just click Add – New Item – WSPBuilder – WebPart Feature. This will create a .webpart file, feature.xml, element.xml  and code file for you. And, I choose “Site” scope so that the WebPart is added to my Site Collection WebPart Gallery. If you don’t use WSPBuilder, the process is the same, you still have to build the .webpart file, feature.xml, element file and code file (WSPBuilder just does it for you).

So, let’s talk about what these files do:

1.       Feature.xml – this file defines “how” the feature will get deployed and the general settings for the feature. Technically, this is the only file required for a feature to work.

 

The important part of this file is the “Scope”. The scope defines where the feature will be deployed (Site, Web, Web Application, Farm). For webparts, we usually choose “Site” in order for the WebPart to be added to the WebPart Gallery. The other important part of this file is the “ElementManifest”. This tells the feature where the element file is located. The element file is the actually actionable file in a feature.

<?xml version=1.0 encoding=utf-8?>

<Feature  Id=7801227a-326a-4a65-8769-6e56ade76b66

          Title=Custom Webpart

          Description=This is my custom webpart Webpart

          Version=12.0.0.0

          Hidden=FALSE

          Scope=Site

          DefaultResourceFile=core

          xmlns=http://schemas.microsoft.com/sharepoint/ >

  <ElementManifests>

    <ElementManifest Location=elements.xml/>  

  </ElementManifests>

</Feature>

 

2.       Elements file – usually we call this file elements.xml, but you can call it whatever you want. You just need to make sure it is an xml file and you need to make sure that the name you call it is the same as the reference in the feature.xml.

 

This file tells SharePoint “what” to do with the feature. There are a lot of different things you can do with SharePoint Features. So, you have to do your research to understand the different xml combinations. In the case of the WebPart, we are using a “Module” action. Module actions move files to different galleries (or lists) in SharePoint. With webparts, we want to move the .webpart gallery to the WebPart gallery list. We do this by setting the “url” property to the “_catalogs/wp” path. That is the path for the WebPart gallery list. The other thing to notice in this file is the “Group” and “QuickAddGroups”. These define the grouping when you click “Add New Webpart” on your SharePoint site.

<?xml version=1.0 encoding=utf-8 ?>

<Elements xmlns=http://schemas.microsoft.com/sharepoint/>

  <Module Name=WebPartPopulation Url=_catalogs/wp RootWebOnly=TRUE>  

         <File Url=CustomWebpart.webpart Type=GhostableInLibrary>

                <Property Name=Group Value=RDA></Property>

                <Property Name=QuickAddGroups Value=RDA />

         </File>    

  </Module>

</Elements>

 

3.        .webpart file – this is the file that “describes” the properties of the WebPart. This could also be a .dwp file (it all depends on the version of the WebPart you are using). SharePoint supports .net webparts and SharePoint webparts. That is why you can have .dwp and .webpart files. So, any property you can set in your WebPart, can be set in this file. I am not going to show an example here because the set of properties is different for every WebPart.

 

This file also contains the assembly path to the code file. This is important, because if you ever change information about the Code file (name, version, etc…), then you need to change the path in your .webpart file.

 

4.       Code file – this is the file that actually runs the webpart code. These files inherit from one of two classes depending on the version of WebPart you are building (.net webpart or SharePoint webpart). For example, a .net framework webpart will inherit from Microsoft.SharePoint.WebPartPages.WebPart. Once again, I am not going to show an example of this, because it is different for every customization. The basis of this class is that you build out your html controls programmatically and add them to the Control collection.

 

Side Note: On a side note that is not really part of this article, don’t think the SharePoint webpart (i.e.: dwp) is better to use than the .net webpart (i.e.: .webpart) just because one is labeled a SharePoint webpart. If you don’t plan on using SharePoint webpart connections or part caching, I actually recommend using a .net webpart in SharePoint. Also, you might hear .dwp webparts referred to as Version 2 webparts and .webparts referred to as Version 3 webparts. This is because .dwp webparts were originally built just for SharePoint 2.0 and are backwards compatible. But, in SharePoint 3.0 we got the ability to use the asp.net webparts.

 

Cleaning up your WebPart Gallery

Ok, so you know how to deploy a WebPart to the WebPart Gallery. But, you notice that when you de-activate your Feature the WebPart is still hanging around in the Gallery. Some developers don’t really care about this. But, other than the obvious problem of keeping a clean Gallery for maintaince, there is another big issue. Let’s say you are using versioning for your assembly for the WebPart code. The reference for the assembly is located in the .webpart file. Thus, if you don’t remove the WebPart from the gallery before you make version changes, then your new assembly version will never make it to the gallery.

So, the easiest way to do this is to just go to the WebPart Gallery and remove it before you install and activate your updated feature. However, as most SharePoint developers know, you want to stay away from manual processes as much as possible. That makes your customizations almost impossible to maintain as your code base grows.

But, the answer to this problem is easy. Create a feature receiver to remove the webpart from the WebPart Gallery when the feature deactivates. I saw this done for the first time when I was reviewing the code for the Forms Based Authentication project in the Community Toolkit for SharePoint:  http://www.codeplex.com/CKS. I modified the code a little to meet my needs, but it is basically the same. Here is how you do it.

Create a Generic Class

This class is a generic class that will work for any WebPart feature receiver. The idea behind this class is to implement the Feature Deactivating event and remove the WebPart. Here is the code for this class:

     class GenericWebPartFeatureReceiver : SPFeatureReceiver

    {

 

        /// <summary>

        /// Remove the webpart from the webpart gallery. This is to fix a shortcoming of SharePoint

       ///       that it doesn’t remove

        /// the webpart when the feature gets deactivated (even though it adds it when the feature

       ///       gets activated).

        /// </summary>

        /// <param name=”properties”></param>

        public override void FeatureDeactivating(SPFeatureReceiverProperties properties)

        {

            SPSite site = null;

            try

            {

                site = properties.Feature.Parent as SPSite;

                using (SPWeb web = site.RootWeb)

                {

                    SPList list = web.Lists["Web Part Gallery"];

 

                    // go through the items in reverse

                    for (int i = list.ItemCount – 1; i >= 0; i–)

                    {

                        // format name to look like a feature name

                        string webpartName = list.Items[i].Name;

                        webpartName = webpartName.Substring(0, webpartName.IndexOf(‘.’));

 

                        // delete web parts that have been added

                        if (properties.Feature.Definition.DisplayName == webpartName)

                        {

                            list.Items[i].Delete();

                            break;

                        }

                    }

                }

            }         

            finally

            {

                if (site != null)

                    site.Dispose();

            }

        }

 

        public override void FeatureActivated(SPFeatureReceiverProperties properties){ }

 

        public override void FeatureInstalled(SPFeatureReceiverProperties properties){ }

 

        public override void FeatureUninstalling(SPFeatureReceiverProperties properties) { }

    }

 

Call Feature Receiver

After your Generic Feature Receiver is built, then you just need to call it from the feature.xml file for all your WebParts. Here is some sample code. It is the same as the feature.xml file I showed above, except for the fact that I placed a reference to a RecieverAssembly and a ReceiverClass.

<?xml version=1.0 encoding=utf-8?>

<Feature  Id=7801227a-326a-4a65-8769-6e56ade76b66

          Title=Custom Webpart

          Description=This is my custom webpart Webpart

          Version=12.0.0.0

          Hidden=FALSE

          Scope=Site

          DefaultResourceFile=core

          xmlns=http://schemas.microsoft.com/sharepoint/

   ReceiverAssembly=MySolution, Version=1.0.0.0, Culture=neutral, PublicKeyToken=3cbe19ecf7353dae

   ReceiverClass=MySolution.GenericWebPartFeatureReceiver>

  <ElementManifests>

    <ElementManifest Location=elements.xml/>  

  </ElementManifests>

</Feature>

Note: You cannot just copy the xml above. The reference to the features ReceiverAssembly and ReceiverClass is specific to my implementation. When you deploy the assembly that contains the “GenericWebPartFeatureReceiver” to your BIN or GAC, then you need to get the information necessary to build out these two lines. Make sure the name of the Assembly and Namespaces match your implementation. And, make sure that the PublicKeyToken matches your public key for your assembly.

 

That’s it; you now have a Generic WebPart Feature Receiver that will remove the WebPart from the WebPart Gallery whenever your WebPart Feature is deactivated. This should help you keep your gallery clean and will help eliminate possible issues with assembly versioning.

Re-attaching objects – My Foray into Linq to Sql and the Entity Framework (part 2 of 2)

This is the second part of a two part series on Linq to Sql and the Entity Framework. In the first part we talked about the two different frameworks. Specifically we talked about detached objects. This part of the series will discuss the options to re-attach objects. We will be discussing this in terms of Linq to Sql. However, the “ideas” will work for the Entity Framework also.

How do I Re-attach my objects?

During my research I found 3 different techniques for re-attaching objects. Each of these techniques has pros and cons, so I will attempt to explain each one. I will be explaining these techniques in terms of Linq to Sql. However, the architectural decisions are the same with Linq to Sql or the Entity Framework.

Technique 1: Re-Query

The re-query technique is basically re-querying the database in your save routine. The reason to use this is to get the “original” values back before you save. However, there are some serious downsides to this technique.

Here is an example of this technique in action. Let’s say we have already done our retrieve and then hit the save button on our asp.net application. A postback ran and we filled in an “Employee” object. Now we have to attach that employee object to our context to save. However, if we just attach it, then the framework will always thing the object is “New”. So, we will re-query to get the original values so the framework knows we are doing an update.

//Instantiate a context to do our save

MyDataContext saveContext = new MyDataContext();

 

//Run a query to get the “original” objects

MyDataContext origContext = new MyDataContext();

Employee origEmp = origContext.Employees.Single(e => e.EmployeeID == 1);

 

//Attach the new employee object to the save context and pass in the original context

//so it can do concurrency checking

saveContext.Employees.Attach(newEmp, origEmp);

saveContext.SubmitChanges();

 

The above technique will seem like it works. However it has two issues:

1.       Performance – there is no need to run that extra retrieve when saving

2.       Concurrency – this is not really checking concurrency, it is doing a fake check

Let me explain a little more why this is doing a fake check. Let’s take an example of two users in a system. This system has a status field and the users really need to know the current status when they are changing data because another user might have already made the changes that they wanted to make for this particular status.

1.       User 1 retrieves the data

2.       User 2 retrieves the data

3.       User 2 updates the data

4.       User 1 updates the data

That is a concurrency issue on step 4. When User 1 updates the data, they should know that someone else slipped in an updated the data before them and they never saw that updated data. Well, in the above approach where you query the data in the save logic, just to get the original values, you are not really solving the underlying problem. Because User 1 just retrieved User 2’s data when they were saving and never saw the changed data. Concurrency checks will all pass and User 1 will never know there was an issue.

So, I don’t ever recommend this technique if concurrency is something you need in your system.

 

Technique 2: Session

Using session to store your objects across PostBacks is a valid technique to keep original data around. The basic steps are to retrieve the data in load, set that data to session, run the save PostBack, get the data out of session and reset the fields with the data from your user controls.

This sounds easy, but there are many considerations when using this technique.

Consideration 1

Session can hurt performance of a server and eat up resources. You have to be very careful when using Session on an asp.net application. If your objects are too large, then you don’t want to store them in session. Also, if you have many users accessing the system at the same time, session can become an issue. So, weigh in the normal architectural decisions of using Session when validating this approach.

Consideration 2

You have now disconnected the original object and want to re-attach it to the context. Don’t you lose all “state” information (i.e.: new, modified, deleted, etc…)? Well, yes, you do. But, this is where that Code Plex project I mentioned earlier comes in to play http://www.codeplex.com/LINQ2SQLEB. This keeps the state and original values in the base class for the objects, instead of just keeping them in the context. And, it reattaches the objects to a new context and sets all this information. Thus, I can serialize my objects, reattach them to my data context and still have my original values and state around for optimistic concurrency checks.

Consideration 3

Linq to Sql objects are not marked as Serializable. Thus, they can’t just be set into session easily. However, the Entity Framework objects are marked that way, so this is only an issue with Linq to Sql.  Luckily, there is a trick to this. Linq to Sql does support DataContracts. Thus, you can use the DataContactSerializer to store Linq to Sql objects in session http://www.squaredroot.com/post/2008/01/30/Storing-LINQ-Objects-in-SQL-Based-Session-State.aspx

 

This technique was the approach I picked for my last project. We had other architectural decisions in which we were storing objects in session anyways, so it seemed like a perfect fit. And, because of this technique, and the base class I found (LINQ2SQLEB) is the only reason I picked Linq to Sql over the Entity Framework. If there was something available like this for the Entity Framework, I would use it in a second. Hum, maybe I will create something like that.

Note on Technique 2

Be careful with this technique and the concurrency checking mechanism of Linq to Sql. When you re-attach your object, to do an update, here is what the update statement will look like:

@originalValue1 = 02/11/2009 12:33:31 PM

@originalValue2=”testing”

@newValue1 = 02/13/2009 12:33:31 PM

@newValue2=”test”

UPDATE [MyTable]

SET [lastUpdatedTime] = @newValue1, [field1] = @newValue2

WHERE ([lastUpdatedTime] = @originalValue) AND ([field1] = @originalValue2)

 

This is fine and the update statement will do optimistic concurrency perfectly. However, there is a performance consideration here. The update statement builds a check on every field in the “Where” clause. This can make a bloated sql statement when you are updating lots of fields. If you want to clean up these sql statements, here is my suggestion. Add a field to your table in the database that always changes, I recommend a timestamp field in sql server. Then, go to your Linq to Sql designer and turn the concurrency check from “Always” to “Never” on all your fields except this timestamp field. This way, the timestamp field is the only field that will be checked in the “Where” clause. You will still have the same optimistic concurrency and a much quicker update statement.

capture

 

Technique 3 – Manually attaching with timestamp

The last technique I want to explain is how to manually attach an object with a timestamp field. In this technique we retrieve our data in the load, set a timestamp field to ViewState, run the save on PostBack and use that timestamp field to set the original values.

The technique is actually quite simple when seen in code. Below I am showing the save code. Assume we have already loaded the data and set the timestamp field to a ViewState attribute marked _timestamp.

 //Instantiate a context to do our save

MyDataContext saveContext = new MyDataContext();

 

//Create a new employee object to fill in

Employee emp = new Employee();

 

//reset the timestamp viewstate field “before” I attach to set the original value

emp.timeStamp = _timeStamp;

 

//Attach the object after setting the timestamp.

//This creates an object in state “New”

saveContext.Employees.Attach(emp);

 

//set the values from our user controls

Emp.FirstName = txtFirstName.Text;

Emp.MiddleName = txtMiddleName.Text;

Emp.LastName = txtLastName.Text;

 

saveContext.SubmitChanges();

 

Keys to making this work:

1.       Set the original value, from ViewState, before you attach the object.

2.       Set the new values, from your user controls, after you attach the object.

3.       Turn the “Update Check” to “Never” on all the fields except for the timestamp field in the Linq to Sql designer.

capture

 

How this works:

Basically, you create a new Employee object. At this point, all the fields (timestamp, FirstName, MiddleName, LastName) are set to null and the state is “New”. Then you set the timestamp field from ViewState. This sets that value to the “original” value we got when we loaded. The state is still “New” at this point. Then we attach to the context. Once the object is attached to the context, any change we make turns the state from “New” to “Modified”. So, when we set the FirstName, MiddleName and LastName we are telling the context that we modified data. This will build an update statement like this:

 

@originalTimeStamp = “3289t5u23940uewu903285”;

@originalFirstName=null;

@originalMiddleName=null;

@originalLastName=null;

@newTimeStamp = “3289t5u23940uewu903285”;

@newFirstName=”Greg”

@newMiddleName=”Brian”;

@newLastName=”Galipeau”;

UPDATE [MyTable]

SET [TimeStamp] = @newTimeStamp, [fName] = @newFirstName, [lname] = @newLastName, [mName] = @newMiddleName

WHERE ([TimeStamp] = @originalTimeStamp)

 

The issue with this approach is that it will always think that it needs to update the fields in the database. Because the original value for the First, Middle and Last names always are null, it always thinks we changed them. This is not necessarily a bad thing. You will have to decide that for your architecture. If it is a bad thing, and you really want to use this approach, then you have to set those values in ViewState when you load also. That way you can set the original values, before you attach, and the tool can do a real check against the value to see if you actually made any changes. This process of setting everything into ViewState could become quite cumbersome on larger forms.

The following example assumes I stored the original first, middle and last names in ViewState on the page load.

//Instantiate a context to do our save

MyDataContext saveContext = new MyDataContext();

 

//Create a new employee object to fill in

Employee emp = new Employee();

 

//reset the viewstate fields “before” I attach to set the original value

emp.timeStamp = _timeStamp;

emp. FirstName = _firstName;

emp. MiddleName = _middleName;

emp. LastName = _lastStamp;

 

 

//Attach the object after setting the timestamp.

//This creates an object in state “New”

saveContext.Employees.Attach(emp);

 

//set the values from our user controls

Emp.FirstName = txtFirstName.Text;

Emp.MiddleName = txtMiddleName.Text;

Emp.LastName = txtLastName.Text;

 

saveContext.SubmitChanges();

 

  

Conclusion

So far these 3 techniques are the best ones I have found for re-attaching disconnected objects using Linq to Sql or the Entity Framework. If you have any other suggestions please leave a comment. And, let’s just “hope” Microsoft one day realizes this scenario and gives us better options, such as the option to have a mini-datacontext that we can store in Session. That would be nice!

My Foray into Linq to Sql and the Entity Framework (Part 1 of 2)

Recently I had a project in which I knew that I wanted to use one of the new O/RM tools from Microsoft. If you don’t know what an O/RM tool does, then let me give my brief explanation. An O/RM tool creates mappings from the database to an object models. This really cuts down on manual mappings and manual creation of objects and is a great option for Rapid Application Development.

Why are there two tools?

So, the most obvious question is – why are there two tools for O/RM in the Microsoft stack? Well, I am not really sure, but I know the timeline a little – so you can draw your own conclusions.

Linq to Sql came out first and Linq to Sql was never even supposed to exist. Well, at least that is what Matt Warren’s blog says: http://blogs.msdn.com/mattwar/archive/2007/05/31/the-origin-of-linq-to-sql.aspx.  So, what happens at your company when a developer builds something and it gets released even though it wasn’t spear headed by the company – politics begin? I am not saying that happened here, but soon after Linq to Sql comes out we have a new O/RM tool from Microsoft, the Entity Framework. This new tool does almost everything Linq to Sql does, and it does it in similar ways. But, it was created separately and it scales a little more. Even though some people think that some of the smaller nuisances in Linq to Sql are actually better than the Entity Framework, but that debate is not one I will get into during this blog.

This new O/RM tool, the Entity Framework, has the ability to hook into more database types. It also has some more advanced mapping configurations. It decouples the database structure from the logical data model better than Linq to Sql. And, it is easier to use with stored procedures. However, it is not as “lightweight” as Linq to Sql and is suppose to work with all databases, thus it doesn’t take advantage of some of the unique features of particular databases. But, hopefully some better adapters will come out soon for particular databases that will increase performance.

I heard Microsoft was killing off Linq to Sql?

Well, I am not 100% sure yet on this statement. From what I hear Microsoft will continue to “support” Linq to Sql, however we might not see any extra features added to it. I have seen statements that Microsoft will continue to invest in Linq to Sql based on customer feedback, but the Entity Framework is the recommended data access solution for .net 4.0. What this tells me is that when .net 4.0 comes out we will see enhancements to the Entity Framework and we will continue to see enhancements to the Entity Framework from there on out. But, at the same time the word “dead” might not be appropriate for Linq to Sql. While we might not see many improvements, Microsoft should continue to support this technology.

Please refer to this article by Jonathan Allen on InfoQ explaining these thoughts a little more: http://www.infoq.com/news/2008/11/DLINQ-Future

Also, here is the Microsoft blog that clarifies the future of Linq To Sql: http://blogs.msdn.com/adonet/archive/2008/10/31/clarifying-the-message-on-l2s-futures.aspx 

Choosing one or the other

Well, based on my previous statements you would think it is obvious to choose the Entity Framework. However, on my last project I actually still chose Linq to Sql. Why would I do that? It has to do with established patterns and tools, let me explain. I was building an asp.net application and I wanted to use a repository pattern.

The repository pattern allows me to set up Retrieve, Update, Insert, etc… statements in a repository class. This class takes care of using my object to attach to the data context in Linq to Sql and then runs the respective Linq to Sql data access operation. It is a great pattern and one I recommend for asp.net projects and multi-tiered projects in Linq to Sql.

Well, one big feature of the repository pattern is re-attaching disconnected objects. On this particular project we had a real short deadline and needed every advantage we could get. Plus, this project had a very well defined database that mapped perfectly to the object model. It was really setup for a lightweight O/RM tool. Luckily, I found a project on CodePlex specifically designed to re-attach disconnected objects in Linq to Sql – http://www.codeplex.com/LINQ2SQLEB. This project is basically a base class for Linq to Sql that has the ability to re-attach disconnected objects. I couldn’t find anything similar for the Entity Framework at this point (probably because it is newer). So, that is why I decided to go with Linq to Sql.

What do you mean when you keep mentioning Disconnected Objects?

Ok, I have mentioned Disconnected Objects a bunch in the last section. So, if you don’t understand what I mean by that, let me explain the architecture of both Linq to Sql and the Entity Framework a little. Both of these tools use a graphical interface that creates your objects. However, they actually create two things:

1.       Context

2.       Objects

So, the first thing to understand is what is a context versus an object? The easiest one to understand is the objects. These are the actual objects that represent the tables in your database. These are dynamically built by the frameworks, but they are basically classes with properties (just like we would do if we did the mapping manually ourselves). So, there is nothing too fancy about the actual objects that get created.

The fancy stuff in these frameworks is the Context. The Context is what you use to populate the objects. For example, you want to run a retrieve, you tell the Context to retrieve the data and it populates the objects. Or, you want to save, you tell the Context to save, and it saves the objects that were previously retrieved. The Context also remembers “state”. State is important because it remembers if the object is New, Modified, Deleted, etc… This becomes really important during save operations. The framework needs to know the state to know what kind of sql statement to run (Update, Insert, Delete). The Context also keeps the original values from the initial load and can use these initial values to do optimistic concurrency checks when saving. This architecture is great for WinForM or Silverlight applications, but can produce issues with Postback scenarios in asp.net (we will explain this later in the article).

So, hopefully I adequately explained what the Context does and what the Objects are for. But, I still haven’t explained Disconnected Objects. Disconnected Objects can happen in many ways, but I will explain in terms of asp.net. Let’s say you use the Context, to call the framework, to populate your objects. And, let’s say you are doing this in the OnLoad of your asp.net page. Now let’s say the normal asp.net processing takes place and when you finish the user hits the Save button. The Save button does a PostBack at this point. Well, there is your issue. As soon as you do the PostBack you have lost the reference to your Context and your objects.

Ok, I have lost my references, how do I get them back. Well, you could just instantiate new objects and fill them in. However, that is all you will have, new objects. This is now a Disconnected Object scenario. Your objects have been disconnected from your Context.

In the next part of this series we will discuss the different options for reattaching those disconnected objects.

Site Information Viewer

Recently in the SharePoint Development Forum on msdn, there was a thread going around about viewing certain things from the SharePoint API. These threads dealt with viewing the users associated to sub-sites and viewing the template the sub-site was created with. Of course, myself and others replied with code snippets of how to do it. During the discussion someone asked me if I could post the webpart I had (in which I got the code snippet from) to CodePlex. I was about to put it on CodePlex “as is” and I realized it was really too simple to put it on CodePlex “as is”. So, I decided to build a simple administrative WebPart called the “Site Information Viewer”. This WebPart enumerates the sub-sites of the site collection it is on and shows a tree view. You can click on any site in the tree view and it will show you valuable information from the SharePoint API about the particular site.

A more detailed description of the webpart can be found here: http://www.codeplex.com/RDACollaboration/Wiki/View.aspx?title=Site%20Information%20Viewer%20WebPart

The release for the webpart is with the other RDA CodePlex releases found here: http://www.codeplex.com/RDACollaboration/Release/ProjectReleases.aspx?ReleaseId=15218

2

I can’t retract my solution!

Ok, I have seen this issue a bunch and the first time I ran across it I didn’t know what to do. I should have written a blog about it then, but forgot. Then, the other day a colleague of mine ran across the same issue. So, I decided to write a blog and hopefully help someone else in the future.

Have you ever tried to retract a solution in Central Administration and it then sits on “Retracting”. Then, after it is done it still says “Deployed”. And, to make matters worse, you don’t see an error message. Why does this happen?

Well, I know of one way that is very common. If you originally deployed to all webs, instead of specifying the web applications to deploy to, then you can get this issue. In fact, if you use a tool like WSPBuilder, and you use their deploy command in Visual Studio, you are deploying to all web applications.

Here is the issue, after you do a retract and use “allcontenturls”, which is the default if you just say retract all web applications in Central Administration, then it has an issue. The issue is that it won’t retract the solution from Central Administration itself. In order to retract the solution from Central Administration, you have to choose it specifically to retract. You can do this through stsadm and point directly to the central administration url. Or, you can go into Central Administration and when you go to the retraction screen you can pick the Central Administration web application from the dropdown list.

Usually I just do my retractions in Central Administration. So, when I originally do a “deploy to all web applications”, I then have a 2 step process to retract. Step 1: Retract from all Web Applications. Step 2: Retract specifically from Central Administration.

Hopefully this helps someone in this situation. I am sure there are more reasons why a web application won’t retract, but this is a common scenario I have seen.

I’m back

Ok, if there actually is anyone who follows this blog, then I have to apologize for not writing a blog in a while. I have been super busy at a client and just couldn’t find the time. But, the good news is we ran into so many issues at the client I have a lot to talk about :) . So, I should be writing some new blogs in the upcoming weeks. These will revolve around linq to sql (because that was a huge part of my last project). And, of course they will revolve around SharePoint (man, I do too much SharePoint :) ). So, keep checking in and hopefully I will have some good stuff. I will also be presenting at the FedSPUG on March 5th. This is the Federal SharePoint Users Group in Washington, D.C. My topic will be on Information Architecture. So, if you are part of that group, then please stop by, I think I have an entertaining presentation this time.

Incorporating a Master Page into a SharePoint site definition

My sixth article in a 6 part series on Master Pages and SharePoint has been published to www.sharepointmagazine.net. You can view the article here: http://sharepointmagazine.net/technical/development/incorporating-a-master-page-into-a-sharepoint-site-definition-master-pages-and-sharepoint-part-6-of-6

 

This article is the sixth part of a larger series of articles:
1. Introduction to Master Pages.
2. Examining the out of the box Master Pages in SharePoint.
3. Developing a custom Master Page for SharePoint.
4. Deploy a Master Page for a production ready system.
5. Customizing the Application.master Page.
6. Incorporating a Master Page into a SharePoint site definition.