RSS Feeds and Google Sitemaps for ASP.NET MVC with LINQ To XML

For the majority of personal web sites, two uses of XML are commonplace: creating an RSS feed and a Google Sitemap. Here, I look at how to create both of those using LINQ to XML for an ASP.NET MVC web site.

I've already looked at both RSS Feeds and Google Sitemaps before, using an XMLTextWriter object to generate the output, and it can be a fairly laborious task. LINQ to XML requires far less code to achieve the same thing. Both of the documents will be generated on demand - that is as a result of someone requesting the appropriate URL. This is so that the documents incorporate the most up-to-date content as articles are added and amended regularly. The content will be drawn from a database and the Entity Framework is the mechanism I have chosen to act as my data access technology.

My full blown Article class consists of the text of the article and a lot of meta-data that just isn't necessary for RSS or a sitemap, and I have already created a couple of classes to cater for the retrieval of a small subset of Article data, so I shall use those. The actual data retrieval is in a class called ArticleRepository, which is in the Model area of the application. Starting with the RSS feed, I have created a method that lists the most recent 20 items that have been added:


public IEnumerable<ArticleSummary> GetRSSFeed()
{
  return (de.ArticleSet
              .OrderByDescending(a => a.DateCreated)
              .Select(a => new ArticleSummary
                              {
                                ID = a.ArticleID,
                                Head = a.Headline,
                                Intro = a.Abstract,
                                CreatedDate = a.DateCreated
                              }))
              .Take(20)
              .ToList();
}

I also need to create a Controller to handle this (and the sitemap), so I shall call it XMLController:


using System;
using System.Linq;
using System.Web.Mvc;
using System.Xml.Linq;
using MikesDotnetting.Helpers;
using MikesDotnetting.Models;


namespace MikesDotnetting.Controllers
{
  public class XMLController : Controller
  {
    private IArticleRepository repository;

    public XMLController() : this(new ArticleRepository())
    {

    }

    public XMLController(IArticleRepository rep)
    {
      repository = rep;
    }


  }
}

This controller is making use of the Repository pattern, in that instead of invoking the ArticleRepository and calling it directly, I am programming against an interface instead. If I decide to get fed up with Entity Framework, and choose to use LINQ to SQL, or even ADO.NET code for data access, I will only have to change this class in one place, rather than have to go through every method and unhook the concrete ArticleRepository class from them. So now I need to add a method to generate the RSS feed, which is actually just a streamed XML document. We will look at the code that does that, then examine it:


public ContentResult RSS()
{
  const string url = "http://www.mikesdotnetting.com/Article/Show/{0}/{1}";
  var items = repository.GetRSSFeed();
  var rss = new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
    new XElement("rss",
      new XAttribute("version", "2.0"),
        new XElement("channel",
          new XElement("title", "Mikesdotnetting News Feed"),
          new XElement("link", "http://www.mikesdotnetting.com/rss"),
          new XElement("description", "Latest additions to Mikesdotnetting"),
          new XElement("copyright", "(c)" + DateTime.Now.Year + ", Mikesdotnetting. All rights reserved"),
        from item in items
        select
        new XElement("item",
          new XElement("title", item.Head),
          new XElement("description", item.Intro),
          new XElement("link", String.Format(url, item.ID, UrlTidy.ToCleanUrl(item.Head))),
          new XElement("pubDate", item.CreatedDate.ToString("R"))

        )
      )
    )
  );
  return Content(rss.ToString(), "text/xml");
}

If you compare LINQ to XML with using the XmlTextWriter, you can see that a lot less code is required. The savings mainly derive from not having to explicitly close elements in the document, and not having to write out the individual parts of an element using different methods such as WriteElementString(), WriteStartElement() etc. Not only that, but you can almost see the outline of the finished XML document from the code. The link node within the item elements has an odd method applied to the item.Head to create a link. UrlTidy.ToCleanUrl takes the existing article title, and replaces spaces with dashes etc to give a clean SEO-friendly url. The code for the method is available in this previous article (first block of code). The result of the action is returned as a ContentResult, which allows for any type of data. In this case, the content type is also specified as text/xml. I have seen some examples of RSS feed that use application/rss+xml. This works for a large number of rss readers, but cannot be relied upon all the time - application/rss+xml is NOT a standard MIME type.

Now to the second action on the controller:


public ContentResult Sitemap()
{
  XNamespace ns = "http://www.sitemaps.org/schemas/sitemap/0.9";
  const string url = "http://www.mikesdotnetting.com/Article/Show/{0}/{1}";
  var items = repository.GetAllArticleTitles();
  var sitemap = new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
      new XElement(ns + "urlset",
          from item in items
          select
          new XElement("url",
            new XElement("loc", string.Format(url, item.ID, UrlTidy.ToCleanUrl(item.Head))),
            item.DateAmended != null ?
                new XElement("lastmod", String.Format("{0:yyyy-MM-dd}", item.DateAmended)) :
                new XElement("lastmod", String.Format("{0:yyyy-MM-dd}", item.DateCreated)),
            new XElement("changefreq", "monthly"),
            new XElement("priority", "0.5")
            )
          )
        );
  return Content(sitemap.ToString(), "text/xml");
}

The main differrence between this method and the previous one is the presence of a namespace in the XML. The RSS sepcification doesn't require one to be present, but the Sitemap 0.9 specification does:


<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">

Lots of developers see a namespace in XML and treat it in the same way as attributes, with code similar to this:


string ns = "http://www.sitemaps.org/schemas/sitemap/0.9";

var sitemap = new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
    new XElement("urlset",new XAttribute("xmlns", ns),

And then they see an exception message along these lines:

The prefix '' cannot be redefined from '' to 'http://www.sitemaps.org/schemas/sitemap/0.9' within the same start element tag.

Namespaces are different to attributes, and need to be defined as an XNamespace object before being passed in to the document.

Now that the actions have been created, all that is needed is to register a couple of routes to point to them:


routes.MapRoute(
  "Sitemap", "sitemap",
  new { controller = "XML", action = "Sitemap" });
  
routes.MapRoute(
  "RSS", "rss",
  new{controller="XML", action="RSS"});

And we are done.

There are some improvements that can be made such as use of caching or perhaps even generating a file that sits on disk. This will save having the sitemap or rss feed being generated anew unnecessarily when there have been no modifications to it since the last request. Also, building on the comments from Jim Wooley and John Sheehan below, the approach I have taken does in fact technically result in the View being built within the Controller action, which is OK if you have a simple requirement. You could almost see LINQ to XML as acting as an Html.Helper extension method in this case (which you would usually find in Views). However, as Jim points out, passing a strongly typed collection to a View and then formatting the results there is a much better way of doing things if you intend to expose multiple feeds. John's approach using the Argotic Syndication Framework to build a custom ViewResult is particularly sweet, and, like mine, means there is no View in the project.

Date Posted: Friday, July 10, 2009 10:43 PM
Last Updated: Friday, October 10, 2014 9:11 PM
Posted by: Mikesdotnetting
Total Views to date: 35475

12 Comments

Saturday, July 11, 2009 5:37 PM - Jim Wooley

Correct me if I'm wrong, but it appears that you are formatting your VIEW in your Controller. If you were to create a common RSS view, your controller could project from the actual data repository into a structure that the RSS can consume. In your case, you could project into an IEnumerable<SyndicationFeedITem> and then consume that in your View. This is particularly helpful if you plan to expose different kinds of RSS feeds (filtered by category, comments, etc).

Another recommendation regarding the namespace issue. If you use XNamespace ns = XNamespace.GetNamespace("http://SomeDomain.org"); then you could use the overloaded + operator to combine the namespace with the name as follows: new XAttribute(ns + "xml).

Saturday, July 11, 2009 11:01 PM - Mike

@Jim

No you are not wrong. I am definitely formatting the View in the controller. But your point about the potential shortcomings of this approach is well made.

Sunday, July 12, 2009 7:53 PM - John Sheehan

This would be a great situation for a custom ActionResult. As an example I created a RssResult that takes an Argotic feed object and handles spitting out the proper content type, etc. It's a little more reusable than specifying the params in a ContentResult every time. Example here: http://john-sheehan.com/blog/using-argotic-syndication-framework-with-aspnet-mvc/

Tuesday, July 14, 2009 9:45 PM - Mike

@John,

I like that approach, John. I wasn't aware of Argotic, but I'm playing with it now.

Thanks!

Friday, July 24, 2009 5:24 AM - Pete

Was just wondering how I go about pulling in feeds from multiple unspecified sites by keyword rather than from a specific site.

Monday, July 27, 2009 2:03 PM - Mike

@Pete,

This article is about generating feeds, not consuming them.

Wednesday, July 29, 2009 2:49 PM - Huey

I got this error message in Sitemap action. What's wrong with it?

Exception Details: System.NotSupportedException: Only parameterless constructors and initializers are supported in LINQ to Entities.

Line 65: var sitemap = new XDocument(new XDeclaration("1.0", "utf-8", "yes"),

Thursday, July 30, 2009 7:39 AM - Mike

@Huey,

I'm not an expert on LINQ to XML or LINQ to Entities, but all I do know is that some of the error messages are completely useless, obscure and will send you on a wild goose chase. Probably, further down the code, you have forgotten a comma or something similar.

Try closing the document off in code as early as you can and comment out the remainder. Then walk your closure down the code, each time adding a bit more to the document until you get the error again. Whichever bit you have just added will contain the cause of the problem.

Saturday, August 1, 2009 7:23 PM - Ryan

The timing of this post showing up on the ASP.NET site is comical to me - I saw it literally 2 minutes after publishing an xml sitemap project to codeplex: http://mvcxmlsitemap.codeplex.com.

I think this is a great approach for simple sitemaps involving one controller and a known path, however, the code could get rather ugly for anything more than that.

My project uses an "opt in" approach whereby a developer can include actions in the sitemap by decorating them with a [Sitemap] attribute. My blog post here describes how to use it:
http://www.rjygraham.com/archive/2009/08/01/using-the-mvc-xml-sitemap-in-your-aspnet-mvc-project.aspx


I'd appreciate any feedback you may have if you decide to check it out.

Ryan

Saturday, August 1, 2009 8:43 PM - Mike

@Ryan

My feedback is that you should submit your post to the ASP.NET site. Nice work!

Friday, December 11, 2009 11:06 AM - Jeremy

Thanks for the article Mike.

I've set this up on my site, but Google is detecting the following error.

Incorrect namespace
Your Sitemap or Sitemap index file doesn't properly declare the namespace.

Parent tag: urlset
Tag: url

Did / Do you have the same problem? If so how did you solve it?

Thanks.

Tuesday, December 15, 2009 3:40 AM - Jeremy

Here's the fix:
You need to add the XNamespace to every XElement.

public ContentResult Sitemap()
{
XNamespace ns = "http://www.sitemaps.org/schemas/sitemap/0.9";
const string url = "http://www.mikesdotnetting.com/Article/Show/{0}/{1}";
var items = repository.GetAllArticleTitles();
var sitemap = new XDocument(new XDeclaration("1.0", "utf-8", "yes"),
new XElement(ns + "urlset",
from item in items
select
//Add ns to every element.
new XElement(ns + "url",
new XElement(ns + "loc", string.Format(url, item.ID,
rlTidy.ToCleanUrl(item.Head))),
item.DateAmended != null ?
new XElement(ns + "lastmod", String.Format("{0:yyyy-MM-dd}",
tem.DateAmended)) :
new XElement(ns + "lastmod", String.Format("{0:yyyy-MM-dd}", item.DateCreated)),
new XElement(ns + "changefreq", "monthly"),
new XElement(ns + "priority", "0.5")
)
)
);
return Content(sitemap.ToString(), "text/xml");
}
Add your comment

If you have any comments to make about this article, please use this form to do so. Make sure that your comment relates specifically to the article above. More general comments can be posted through the form on the Contact page.

Please note, all comments are moderated, and some may not be published. The kind of things that will ensure your comment is deleted without ever seeing the light of day are as follows:

  • Not relevant to the article
  • Gratuitous links to your own site or product
  • Anything abusive or libellous
  • Spam
  • Anything in a language I don't understand including gibberish.

I do not pass email addresses on to spammers, so a valid one will assist me in responding to you personally if required.