Displaying Google Analytics Data in ASP.NET

If you have a Google Analytics account, you can use the Data Export API provided by Google to access your visitor stats, and retrieve them for use elsewhere. Here's how to display some stats in ASP.NET.

The approach detailed below has been superceded by the introduction of .NET libraries for accessing Google Analytics Reporting. You should use those libraries instead.

When I first put this site together, it was in a bit of a rush. I decided to keep a count of the number of visits for each article, but did it very simply. All I did was increment a database field called Views by one each time a page was requested. Consequently, the only metric I get is all-time page views. I wondered how I would be able to get the number of page views within the last 7 or 30 days, and it crossed my mind to redesign the database, and the code to track all this. Then I came across the Google AnalyticsData Export API, which provides access to Google Analytics accounts. I have been using Google for my web stats since day one, so I set to work.

Using the API is not that difficult. It's a 3 stage process:

  1. Log in via an HTP POST request and obtain an authorisation key
  2. Query the stats via an HTTP GET request (passing the key as a header) and obtain a data feed
  3. Parse the feed for display

Digging around in the API reference, we see that the core to getting the right data is querying for the correct combination of Dimensions and Metrics. We also see that feeds are returned as XML. The reference site also provides a Query Explorer, with which you can test a few queries, and see how the urls for feed requests are constructed.

I could simply write some procedural code that runs from top to bottom to get the job done quickly, which is what I initially did. However, heavily inspired by Jacob Reimar's most excellent Google Analytics Reader, I decided to build something more extensible so that I can add other reports more easily if required in the future.

I am going to start off with a couple of helper methods. One will make HTTP POST requests and the other will perform the same task using HTTP GET requests:


public static string HttpPostRequest(string url, string post)
{
  var encoding = new ASCIIEncoding();
  byte[] data = encoding.GetBytes(post);
  WebRequest request = WebRequest.Create(url);
  request.Method = "POST";
  request.ContentType = "application/x-www-form-urlencoded";
  request.ContentLength = data.Length;
  Stream stream = request.GetRequestStream();
  stream.Write(data, 0, data.Length);
  stream.Close();
  WebResponse response = request.GetResponse();
  String result;
  using (var sr = new StreamReader(response.GetResponseStream()))
  {
    result = sr.ReadToEnd();
    sr.Close();
  }
  return result;
}


public static string HttpGetRequest(string url, string[] headers)
{
  String result;
  WebRequest request = WebRequest.Create(url);
  if (headers.Length > 0)
  {
    foreach (var header in headers)
    {
      request.Headers.Add(header);
    }
  }
  WebResponse response = request.GetResponse();
  using (var sr = new StreamReader(response.GetResponseStream()))
  {
    result = sr.ReadToEnd();
    sr.Close();
  }
  return result;
}

These are actually utility methods I have in a static class called HttpRequests and use a fair amount. They are fairly standard uses of the WebRequest and WebResponse classes.

Now we need a way to manage the Dimensions and Metrics. Since they are constants, an Enumeration fits the bill nicely in each case, so I create a file called Dimensions.cs, and then simply add the following to it:


public enum Dimension
{
  pagePath,
  pageTitle
}

Then I do the same for Metrics:


public enum Metric
{
  pageviews
}

I could at this stage have added all the dimensions and metrics that the Google Reference lists as Jacob has, but for the purposes of this exercise, decided to keep the content to the barest minimum. Having played with the Query Explorer I linked to earlier, I know these are all the ones I need at this stage for the page view data. The final Enumeration I need is for the sort direction I want to apply to the data. This one is very simple:


public enum SortDirection
{
  Ascending,
  Descending
}

Here's where things get a little more complicated. I want to work with strongly typed data, but the data that Google Analytics provides is in an XML document. Clearly that will need some work done on it. The other thing is that I might create additional reports in the future, so I need that strongly typed base data to be as generic as possible in the first instance. I add a class called BaseData as follows:


public class BaseData
{
  public IEnumerable<KeyValuePair<Dimension, string>> Dimensions { get; set; }
  public IEnumerable<KeyValuePair<Metric, string>> Metrics { get; set; }
}

Each item of BaseData can have a collection of Key/Value pairs containing Dimension data (page titles, page paths etc) and its corresponding value as a string, and a collection of Key/Value pairs containing Metrics and their corresponding values as strings. In the case of the report I want, the Dimensions property will hold two items, whereas the Metrics property will have one key/value pair. However, this approach allows for reuse nicely, as the other future reports may need to make use of many more dimensions or metrics. At the risk of running ahead of myself, here's how that actually looks when BaseData is generated and viewed in the Locals window of the VS debugger:

However, before we get there, we have to generate the data. This requires the following steps:

  1. Get authenticated
  2. Get the XML document
  3. Convert it into BaseData objects

All of this is going to be the job of one class, which I have called GAReporter. It will have 3 methods: one to get authenticated by Google, one to obtain the raw data, and one to convert that to BaseData items. To begin with, I declare a number of string constants:


private const string AuthenticationUrl = "https://www.google.com/accounts/ClientLogin";
private const string AuthenticationPost =
  "accountType=GOOGLE&Email=xxx@gmail.com&Passwd=xxx&service=analytics&source=xxx-xxx";
private const string PageViewReportUrl = 
  "https://www.google.com/analytics/feeds/data?ids={0}&dimensions={1}&metrics={2}&start-date={3}&end-date={4}&sort={5}&max-results={6}";

Not all reports require the number of parameters as in the Page View report. Additional reports may require additional constants to be added to this class. If you fiddle about with the Query Explorer, you can soon find which reports will require similar patterns, and possibly set your constants up accordingly. The first two constants will be the same regardless. All you need to do is to add your own account details. I am using the ClientLogin method of authorisation. For the source parameter, I provide mikesdotnetting-mikesdotnetting-1.0.

The method to authenticate is private to the class. It's only used within it. All it needs to do is to return a string:


private static string Authentication()
{
  string key = null;
  string result = HttpRequests.HttpPostRequest(AuthenticationUrl, AuthenticationPost);
  var tokens = result.Split(new string[] { "\n" }, StringSplitOptions.RemoveEmptyEntries);
  foreach (var item in tokens)
  {
    if (item.StartsWith("Auth="))
      key = item;
  }
  return key;
}

You can see that this method makes use of the HttpPostRequest helper method I talked about earlier. When you attempt to log in, you will receive a response which consists of a series of 3 values. You need to get hold of the one that starts with "Auth=". This value needs to be passed back to Google when the report data is requested. The next private method takes care of requesting and returning the XML report data:


private static XDocument getXMLData(string account, IEnumerable<Dimension> dimensions, 
		IEnumerable<Metric> metrics, DateTime from, DateTime to, Metric sort, SortDirection direction, int maxrecords)
{
  XDocument doc = null;
  var key = Authentication();
  if (key.Length > 0)
  {
    var dimension = new StringBuilder();
    for (var i = 0; i < dimensions.Count(); i++)
    {
      dimension.Append("ga:" + dimensions.ElementAt(i));
      if (i < dimensions.Count() - 1)
        dimension.Append(",");
    }
    var metric = new StringBuilder();
    for (var i = 0; i < metrics.Count(); i++)
    {
      metric.Append("ga:" + metrics.ElementAt(i));
      if (i < metrics.Count() - 1)
        metric.Append(",");
    }
    var sorter = "ga:" + sort;
    if (direction == SortDirection.Descending)
      sorter = "-" + sorter;
    var fromDate = from.ToString("yyyy-MM-dd");
    var toDate = to.ToString("yyyy-MM-dd");
    var url = string.Format(PageViewReportUrl, "ga:" + account, dimension, metric, fromDate, toDate, sorter, maxrecords);
    var header = new[] { "Authorization: GoogleLogin " + key };
    doc = XDocument.Parse(HttpRequests.HttpGetRequest(url, header));
  }
  return doc;
}

This method is responsible for obtaining the XML document form Google. Having obtained the authentication key from the Authenticate() method, it loops through the collection of Dimensions and Metrics that are passed into it, and together with the other parameters, constructs a valid url with query string which contains the details of the data I want.

The final method is the public method which calls the getXMLData() method. It is reponsible for taking the XDocument object generated by the getXMLDoc() method and parsing it before returning a collection of BaseData objects:


public static IEnumerable<BaseData> GetBaseData(string account, IEnumerable<Dimension> dimensions, 
		IEnumerable<Metric> metrics, DateTime from, DateTime to, Metric sort, SortDirection direction, int maxrecords)
{
  IEnumerable<BaseData> data = null;
  XDocument xml = getXMLData(account, dimensions, metrics, from, to, sort, direction, maxrecords);
  if (xml != null)
  {
    XNamespace dxp = xml.Root.GetNamespaceOfPrefix("dxp");
    XNamespace dns = xml.Root.GetDefaultNamespace();
    data = xml.Root.Descendants(dns + "entry").Select(element => new BaseData
    {
      Dimensions =
        new List<KeyValuePair<Dimension, string>>(
        element.Elements(dxp + "dimension").Select(
          dimensionElement =>
          new KeyValuePair<Dimension, string>(
            dimensionElement.Attribute("name").Value.Replace("ga:", "")
            .ParseEnum<Dimension>(),
            dimensionElement.Attribute("value").Value))),
      Metrics =
        new List<KeyValuePair<Metric, string>>(
        from metricElement in element.Elements(dxp + "metric")
        select new KeyValuePair<Metric, string>(
          metricElement.Attribute("name").Value.Replace("ga:", "")
            .ParseEnum<Metric>(),
          metricElement.Attribute("value").Value))
    });
  }
  return data;
}

There's a lot of angle brackets going on in here, but it isn't as complex as it looks. It will probably help to see a snippet of the XML provided by Google that this code is actually working on:

I've highlighted one entry element. If you look at it, you can see that it contains elements prefixed with dxp: - dxp:dimension and dxp:metric. The LINQ to XML code in the method targets these elements. xml.Root.Descendants(dns + "entry") returns a collection of <entry> nodes. Within each of those nodes, dxp:dimension nodes are selected, and the value of their name attribute (minus the leading "ga:") is assigned to a Dimension object, followed by the value of their value attribute, which is assigned to the string part of the Key/value pair that makes up a BaseData object. This happens until all dxp:dimension nodes have been exhausted, building up an List of key/value pairs. Then the dxp:metric nodes are subjected to the same treatment.

If you are familier with Enumerations, you might be wondering what that ParseEnum<T>() method is all about. If you are not familiar with Enumerations and copy and paste this code as-is, you will definitely wonder why the compiler complains about it. It's an extension method I use to wrap the Enum.Parse() method:


public static T ParseEnum<T>(this string token)
{
  return (T)Enum.Parse(typeof(T), token);
}

That takes care of all the base methods and classes. Now I need a specific class for the values in the Page Views report:


public class PageViewReportData
{
  public string Url { get; set; }
  public string Title { get; set; }
  public int Views { get; set; }
}

And a method to generate it:


public class PageViewReporter
{
  public static IEnumerable<PageViewReportData> GetPageViewReport(string account, DateTime from, DateTime to, int max)
  {
    var dims = new Dimension[] { Dimension.pagePath, Dimension.pageTitle };
    var mets = new Metric[] {Metric.pageviews};
    var sort = Metric.pageviews;
    var order = SortDirection.Descending;
    IEnumerable<BaseData> data = GAReporter.GetBaseData(account, dims, mets, from, to, sort, order, max);
    return data.Select(d => new PageViewReportData
                  {
                    Url = d.Dimensions.First(dim => dim.Key == Dimension.pagePath).Value,
                    Title = d.Dimensions.First(dim => dim.Key == Dimension.pageTitle).Value,
                    Views = Convert.ToInt32(d.Metrics.First(met => met.Key == Metric.pageviews).Value)
                  });
  }
}

Like Jacob, I put this method in its own class. If I want different page view reports that require more Dimensions, for example, I can simpy add another method to this class, while keeping similar reports together. When you call this method, whether in an MVC Controller action or a code-behind, you will retrieve a strongly typed collection of objects which can be passed to a Model in MVC, or simply bound to a control in a web form:

One final thing - if you plan to show Google report data on a public page on your web site together with "local" data, I would advise using javascript to load it asynchronously after the rest of the page has rendered. Waiting for remote data can otherwise delay the rendering of your page considerably. In the case of this site, I have used jQuery which targets a controller action called GetGoogleData that returns a partial view:


public ActionResult GetGoogleData(int days)
{
  DateTime toDate = DateTime.Now.AddDays(-days);
  DateTime fromDate = DateTime.Now;
  IEnumerable<PageViewReportData>  data = PageViewReporter.GetPageViewReport("xxxxxx", toDate, fromDate, 15);
  return View("VisitorStatsPartial", data);
}

And the jQuery that shows a "loading" image before populating the div earmarked for the stats data is as follows:


$(document).ready(function() {
  $("#analyticsdata7").html("<img src=\"../../images/loading.gif\" />");
  $.ajax({
    type: "GET",
    contentType: "text/html",
    url: "/Article/GetGoogleData/7",
    success: function(response) {
    $("#analyticsdata7").empty();
    $("#analyticsdata7").html(response);
    }
  });
 });

Summary

If you look at the code that Jacob Reimers provides, you will see that mine doesn't deviate very much from it in terms of structure.  That's because it is nice and solid, and allows for extensibility.  What I hope I have added to it in this article is a detailed explanation of how it works so that you can extend it as you like.

Date Posted:
Last Updated:
Posted by:
Total Views to date: 67311

23 Comments

- Enes TAYLAN

Mike, where is the code download of this post?

- Mike

@Enes

There isn't one.

- jp

There is an .NET API for this kind of thing - no point re-inventing the wheel?

http://code.google.com/p/google-gdata

- Enes TAYLAN

If you add code to this post, I think, it will be very helpful. At least for me.

- Mike

@jp

If you could point me to the .NET client library for Google Analytics Data, I'd be grateful. Only, I can't see one. I can see them for Java, PHP, Ruby etc, but no .NET one.

- Mike

@Enes,

All the code you need is detailed in the article.

- matt

hello,
I can obtain the auth token and forward it in the data feed web request however, I still get 401 unauthorized errors in the response.

- Rami Vemula

Good one...

- bharti

Even i get 401 unauthorized errors in the response.

IS there anything am missing please suggest.

- Mike

@matt and bharti

Are you sure you are passing a valid account profile ID? My code has xxxxx to obscure mine. Use the Google Query Analyser to test what you are passing is valid.

- Swami K

Nice article..

It would have been nice if you provide source code in your site or host it in codeplex so that we can use it easily..

- John Sheehan

This would be a great situation to use RestSharp: http://restsharp.org. It's new so I'm not sure it supports everything needed yet, but I'll test it out and add whatever is needed to do it.

- Wesley Bakker

You can use the WebClient class instead of creating your own helper class:
http://msdn.microsoft.com/en-us/library/system.net.webclient.aspx

And you might want to consider using serialization instead of manual parsing a bunch of xml.

Cheers,
Wes

- Yuthavong

Thanks Mike :)

- Hans_v

"If you could point me to the .NET client library for Google Analytics Data, I'd be grateful"

Mike, I think this is the link:

http://google-gdata.googlecode.com/files/Google%20Data%20API%20Setup%281.4.0.2%29.msi

- Mike

@Hans_v

That's where I first looked, but couldn't find anything for Analytics. Jacob's original code is basically a port (and tweak) of this: http://code.google.com/p/ga-api-java-samples/source/browse/trunk/src/v1/DataFeedExample.java which doesn't seem to have made it into the Google Data SDK yet. And a search for Analytics within the documentation throws up nothing.

- Denis

thank you! saved me from a bunch of headaches, very useful piece of code, nicely explained so that even I was able to make it work within a few minutes. Thanks again - you got some positive karma now :)

- albert

where the code crazy man

- Lisong

Nice article, it is very helpful!

- Jack <x

Great!

- Ansuya

Above code is not working.

- Mike

@Ansuya,

Google provide an official API for this now. You should use that instead.

- Wouter

hello, this method doenst work anymore ...

Recent Comments

Semil 7/1/2015 7:03 AM
In response to iTextSharp - Drawing shapes and Graphics
I have created a rectangle using above methode. Now I want to add a text in the center of this How I...

Satyabrata Mohapatra 6/30/2015 6:12 PM
In response to Reading Excel Files Without Saving To Disk In ASP.NET
Ahh.....this is awesome. Happy to see after a long time you wrote a article on web form :D...

Marty 6/30/2015 7:16 AM
In response to Posting Data With jQuery AJAX In ASP.NET Razor Web Pages
Mike, you're the Man! Another great article. So incredibly helpful. I'm definitely going to buy your...

Rohan 6/30/2015 5:32 AM
In response to ASP.NET MVC 5 with EF 6 - Working With Files
Very good and helpful tutorial. Thanks. Just wanted to know what would be the max file size limit we...

Fernando 6/30/2015 1:59 AM
In response to Programmatically accessing data from DataSource controls
What if I want to pass parameters natively using the DataSourceSelectArguments object, instead of be...

pankaj 6/29/2015 3:13 PM
In response to How to retain carriage returns or line breaks in an ASP.NET web page
very nice i'm use this in my code thank you.... ...

Mike 6/29/2015 2:22 AM
In response to MVC 5 with EF 6 in Visual Basic - Sorting, Filtering and Paging
This is the first example that I have found that works....

Marty 6/28/2015 4:57 AM
In response to Posting Data With jQuery AJAX In ASP.NET Razor Web Pages
Mike, what if I don't want to render back the text to the browser, but I want to send it some other...

Mike 6/27/2015 4:00 PM
In response to Migrating Classic ASP To ASP.NET Razor Web Pages Part One- Razor Syntax And Visual Basic
have you used any of the code converters to convert classic asp to c#? If so, which one do you have...

deepan 6/27/2015 12:49 PM
In response to How To Send Email In ASP.NET MVC
Your article is very very useful for me...thanks for giving this article ...this is very simple to u...


© 2006 - 2015 Mike Brind