Tuesday, 17 September 2013

Monitoring & Analyzing Error Pages (404s) Using Analytics

I recently wrote a post on the Google Analytics + page about monitoring error pages on websites. The post was well received and generated a healthy discussion on Google+, so I decided to write a more detailed article on the subject here.

First of all, what exactly is an error or 404 page? According to Wikipedia “The 404 or Not Found error message is a HTTP standard response code indicating that the client was able to communicate with the server, but the server could not find what was requested.” Or, in more general terms, the 404 is the error you get when the page you are looking for does not exist, usually because the link you clicked was broken.
Another important question is: why should I care? Often times the 404 is forgotten and no one cares to prioritize its optimization. I believe the answer to prioritization lies on section 2 of this post: by monitoring the percentage of users that arrive at this page you will be in a better position to know if (and how quickly) you should optimize your 404 page. In any case, even if the number of people viewing this page is low, you should at least have a page in the lines of your brand and try to add the elements described in section 1 below; after all, you don’t want to disappoint your customers!
In this post I propose a few techniques to help improve error pages, engage visitors and improve the website experience. The questions I will try to answer are the following:
  1. How to build your 404 page?
  2. How to monitor your 404 page traffic efficiently?
  3. How to analyze & optimize 404 page success?

1. Error Pages Best Practices

Before we dive into Google Analytics, let’s take a look into some ways to create a great 404 page from the beginning. Following are some good usability insights proposed in a book called Defensive Design for the Web. The authors advise us to offer customized "Page Not Found" error pages; and they provide an interesting insight into how to create error pages:
Instead of merely saying a page is not found, your site needs to explain why a page can't be located and offer suggestions for getting to the right screen. Your site should lend a hand, not kick people when they are down. Smart things to include on your 404 page:
  1. Your company's name and logo
  2. An explanation of why the visitor is seeing this page
  3. A list of common mistakes that may explain the problem
  4. Links back to the homepage and/or other pages that might be relevant
  5. A search engine that customers can use to find the right information
  6. An email link so that visitors can report problems, missing pages, and so on

2. Monitoring Error Page Traffic

Suppose a prominent blog links to your site and the link is broken, this will cause a negative experience to users (which will not find what they expected) and to search engines (which will not crawl the right content). How long will it take until you notice it? How often do you check the traffic to your 404 page? Chances are you don’t do it every day, but you should! Or at least you should have someone look at it: why not let Google Analytics do it for you? 
Create an Alert on Google Analytics
In the screenshot below you will see how to set an alert on Google Analytics that will let you know each time your 404 pageviews increases above a certain threshold. This will enable you to do the work once and be alerted every time there is a problem. 
The alert below is based on the increase in error pageviews, but if you decide to create a goal (as suggested below) you could create the alert based on the goal too. Note that you can opt in to receive an email or a text message when the condition is met (404 pageviews increases more than 15% compare to previous day). Also note that I decided to define the 404 page based on the title of the page, very often there is no indication of an 404 page on the URL (read more about this below). 
To learn how to set a Custom Alert check this help center article.

Measure your 404 Page as a Goal
Setting the 404 page as a goal on Google Analytics will surface important information that can be achieved only through goals, e.g. the last three steps before getting to this page. Below is a screenshot on how to do it, but note that you would need to have an identifier on your URL (or trigger an event) in order to set your 404 as a Goal.
Add Your 404 Content Report to Your Dashboard
Every report on Google Analytics can be added to the dashboard. By adding the 404 page to your dashboard you will be able to constantly monitor the trend of visits to your 404 page. Learn more about customizing dashboards.

3. Analyzing & Optimizing Error Pages

Monitoring your 404 pages is important, but useless if you don't take action using this information. Taking action means doing all you can to decrease the number of people getting missing pages. Below I provide a few tips on how to find and fix both internal and external broken links.
Check Your Navigation Summary Report
This will help you understanding where did visitors come from from inside your site, i.e. it will tell you which pages contain internal broken links. You will also be able to understand what is the percentage of visitors that arrive to the 404 page from internal and external sources; the internal sources will be listed on this report. See navigation summary screenshot below:

Check 404 Page URLs
Learning which URLs are producing the errors is a great way to get rid of them. If you learn, for example, that 100 visitors a day get an error when they visit the page “/aboutS” you can infer that there is a broken link leading to it; sometimes it might not be possible to find the source of the error to fix the link, but you can add a redirect from that page to “/about”, which looks to be the right page. 
In order to do that you will need to find the report below, but please keep in mind that some assumptions were made to arrive at it. Most CMS (Wordpress, Drupal, and others) will return an error for non-existing pages on the actual content section, but they will keep the original URL; however, they will have a page title with the word 404 in it. So check your site to know if that is the case before you try the report below.
Once you find this report, click on the first entry and you will get a list of all the URLs that triggered an error page. Good luck with the redirects!
Measure Internal Searches From this Page
If you do not have a search box on your 404 page, you should seriously consider adding one. Through searches performed in this page you will be able to understand what people were expecting to find there and you will get insights on which links you should add to the page. If you don’t have Internal Site Search enabled on Google Analytics check this help center article.
Below are the metrics you will be able to analyze if you use this feature:
  • Total Unique Searches: the number of times people started a search from the 404 page. Duplicate searches within a single visit are excluded.
  • Results Pageviews/Search: the average number of times visitors viewed a search results page after performing a search.
  • % Search Exits: the percentage of searches that resulted in an immediate exit from your site.
  • % Search Refinements: the percentage of searches that resulted in another search (i.e. a new search using a different term).
  • Time after Search: The average amount of time visitors spend on your site after performing a search.
  • Search Depth: The average number of pages visitors viewed after performing a search.
Closing Thoughts
As we mentioned above, errors happen, and we must be prepared for them. We must give a hand to our visitors when they are most frustrated and help them feel comfortable again. The level of online patience and understanding is decreasing and users have a world of choices just one click away, so website owners cannot let one small error get on their way.
Posted by Daniel Waisberg, Analytics Advocate

Broadening Google Patents



Cross-posted with the US Public Policy Blog, the European Public Policy Blog, and Inside Search Blog.

Last year, we launched two improvements to Google Patents: the Prior Art Finder and European Patent Office (EPO) patents. Today we’re happy to announce the addition of documents from four new patent agencies: China, Germany, Canada, and the World Intellectual Property Organization (WIPO). Many of these documents may provide prior art for future patent applications, and we hope their increased discoverability will improve the quality of patents in the U.S. and worldwide.

So if you want to learn about a Chinese dual-drive bicycle, a German valve for inflating bicycle tires, attach a Canadian trailer to your bike, or read the WIPO application for pedalling with one leg, those and millions of other inventions are now available on Google Patents.

Thanks to Google Translate, all patents are available in both their original languages and in English, and you can search across the world’s patents using terms in any of those languages. When there are multiple submission languages, you can move between them with a single click on the tabs at the top of the page, as shown in the screenshot below:


Happy patent searching!

Tuesday, 10 September 2013

We are joining the Open edX platform



A year ago, we released Course Builder, an experimental platform for online education at scale. Since then, individuals have created courses on everything from game theory to philanthropy, offered to curious people around the world. Universities and non-profit organizations have used the platform to experiment with MOOCs, while maintaining direct relationships with their participants. Google has published a number of courses including Introduction to Web Accessibility which opens for registration today. This platform is helping to deliver on our goal of making education more accessible through technology, and enabling educators to easily teach at scale on top of cloud platform services.

Today, Google will begin working with edX as a contributor to the open source platform, Open edX. We are taking our learnings from Course Builder and applying them to Open edX to further innovate on an open source MOOC platform. We look forward to contributing to edX’s new site, MOOC.org, a new service for online learning which will allow any academic institution, business and individual to create and host online courses.

Google and edX have a shared mission to broaden access to education, and by working together, we can advance towards our goals much faster. In addition, Google, with its breadth of applicable infrastructure and research capabilities, will continue to make contributions to the online education space, the findings of which will be shared directly to the online education community and the Open edX platform.

We support the development of a diverse education ecosystem, as learning expands in the online world. Part of that means that educational institutions should easily be able to bring their content online and manage their relationships with their students. Our industry is in the early stages of MOOCs, and lots of experimentation is still needed to find the best way to meet the educational needs of the world. An open ecosystem with multiple players encourages rapid experimentation and innovation, and we applaud the work going on in this space today.

We appreciate the community that has grown around the Course Builder open source project. We will continue to maintain Course Builder, but are focusing our development efforts on Open edX, and look forward to seeing edX’s MOOC.org platform develop. In the future, we will provide an upgrade path to Open edX and MOOC.org from Course Builder. We hope that our continued contributions to open source education projects will enable anyone who builds online education products to benefit from our technology, services and scale. For learners, we believe that a more open online education ecosystem will make it easier for anyone to pick up new skills and concepts at any time, anywhere.

Make Your Websites More Accessible to More Users with Introduction to Web Accessibility



Cross-posted with Google Developer's Blog

You work hard to build clean, intuitive websites. Traffic is high and still climbing, and your website provides a great user experience for all your users, right? Now close your eyes. Is your website easily navigable? According to the World Health Organization, 285 million people are visually impaired. That’s more than the populations of England, Germany, and Japan combined!

As the web has continued to evolve, websites have become more interactive and complex, and this has led to a reduction in accessibility for some users. Fortunately, there are some simple techniques you can employ to make your websites more accessible to blind and low-vision users and increase your potential audience. Introduction to Web Accessibility is Google’s online course that helps you do just that.



You’ll learn to make easy accessibility updates, starting with your HTML structure, without breaking code or sacrificing a beautiful user experience. You’ll also learn tips and tricks to inspect the accessibility of your websites using Google Chrome extensions. Introduction to Web Accessibility runs with support from Google content experts from September 17th - 30th, and is recommended for developers with basic familiarity with HTML, JavaScript, and CSS.

There’s a lot to learn in the realm of web accessibility, and a lot of work to be done to ensure users aren’t excluded from being able to easily navigate the web. By introducing fundamental tips to improve web usage for users with visual impairments, Introduction to Web Accessibility is a starting point to learn how to build accessibility features into your code. Registration is now open, so sign up today and help push the web toward becoming truly universally accessible.

Thursday, 5 September 2013

A Comparison of Five Google Online Courses



Google has taught five open online courses in the past year, reaching nearly 400,000 interested students. In this post I will share observations from experiments with a year’s worth of these courses. We were particularly surprised by how the size of our courses evolved during the year; how students responded to a non-linear, problem-based MOOC; and the value that many students got out of the courses, even after the courses ended.

Observation #1: Course size
We have seen varying numbers of registered students in the courses. Our first two courses (Power Searching versions one and two) garnered significant interest with over 100,000 students registering for each course. Our more recent courses have attracted closer to 40,000 students each. It’s likely that this is a result of initial interest in MOOCs starting to decline as well as students realizing that online courses require significant commitment of time and effort. We’d like other MOOC content aggregators to share their results so that we can identify overall MOOC patterns.

*based on surveys sent only to course completers. Other satisfaction scores represent aggregate survey results sent to all registrants.

Observation #2: Completion rates
Comparing these five two-week courses, we notice that most of them illustrate a completion rate (measured by the number of students who meet the course criteria for completion divided by the total number of registrants) of between 11-16%. Advanced Power Searching was an outlier at only 4%. Why? A possible answer can be found by comparing the culminating projects for each course: Power Searching consisted of students completing a multiple choice test; Advanced Power Searching students completed case studies of applying skills to research problems. After grading their work, students also had to solve a final search challenge.

Advanced Power Searching also differed from all of the other courses in the way it presented content and activities. Power Searching offered videos and activities in a highly structured, linear path; Advanced Power Searching presented students with a selection of challenges followed by supporting lessons. We observed a decreasing number of views on each challenge page similar to the pattern in the linear course (see figure 1).
Figure 1. Unique page views for Power Searching and Advanced Power Searching

Students who did complete Advanced Power Searching expressed satisfaction with the course (95% of course completing students would recommend the course to others, compared with 94% of survey respondents from Power Searching). We surmise that the lower completion rate for Advanced Power Searching compared to Power Searching could be a result of the relative difficulty of this course (it assumed significantly more foundational knowledge than Power Searching), the unstructured nature of the course, or a combination of these and other factors.

Even though completion rates seem low when compared with traditional courses, we are excited about the sheer number of students we’ve reached through our courses (over 51,000 earning certificates of completion). If we offered the same content to classrooms of 30 students, it would take over four and a half years of daily classes to teach the same information!

Observation #3: Students have varied goals
We would also like to move the discussion beyond completion rates. We’ve noticed that students register for online courses for many different reasons. In Mapping with Google, we asked students to select a goal during registration. We discovered that
  • 52% of registrants intended to complete the course
  • 48% merely wanted to learn a few new things about Google’s mapping tools
Post-course surveys revealed that
  • 78% of students achieved the goal they defined at registration
  • 89% of students learned new features of Google Maps
  • 76% reported learning new features of Google Earth
Though a much smaller percentage of students completed course requirements, these statistics show that many of the students attained their learning goals.

Observation #4: Continued interest in post-course access
After each course ended, we kept many of the course materials (videos, activities) available. Though we removed access to the forums, final projects/assessments, and teaching assistants, we have seen significant interest in the content as measured by Google and YouTube Analytics. The Power Searching course pages have generated nearly three million page views after the courses finished; viewers have watched over 160,000 hours (18 years!) of course videos. In the two months since Mapping with Google finished, we have seen over 70,000 unique visitors to the course pages.

In all of our courses, we saw a high number of students interested in learning online: 96% of Power Searching participants agreed or strongly agreed that they would take a course in a similar format. We have succeeded in teaching tens of thousands of students to be more savvy users of Google tools. Future posts will take an in-depth look at our experiments with self-graded assessments, community elements that enhance learning, and design elements that influence student success.

Monday, 12 August 2013

Google Research Awards: Summer 2013



Another round of the Google Research Awards is complete. This is our biannual open call for proposals on computer science-related topics including machine learning and structured data, policy, human computer interaction, and geo/maps. Our grants cover tuition for a graduate student and provide both faculty and students the opportunity to work directly with Google scientists and engineers.

This round, we received 550 proposals from 50 countries. After expert reviews and committee discussions, we decided to fund 105 projects. The subject areas that received the highest level of support were human-computer interaction, systems and machine learning. In addition, 19% of the funding was awarded to universities outside the U.S.

We noticed some new areas emerging in this round of proposals. In particular, an increase of interest in neural networks, accessibility-related projects, and some innovative ideas in robotics. One project features the use of Android-based multi-robot systems which are significantly more complex than single robot systems. Faculty researchers are looking to explore novel uses of Google Glass such as an indoor navigation system for blind users, and how Glass can facilitate social interactions.

Congratulations to the well-deserving recipients of this round’s awards. If you are interested in applying for the next round (deadline is October 15), please visit our website for more information.

Wednesday, 7 August 2013

Computer Science Teaching Fellows Starting Up in Charleston, SC



Google recently started up an exciting new program to ignite interest in computer science (CS) for K12 kids. Located in our South Carolina data center, the Computer Science Teaching Fellows is a two-year post graduate fellowship for new STEM teachers and CS graduates. The goal is to bring computer science and computational thinking to all children, especially underrepresented minorities and girls, and close the gap between the ever-increasing demand in CS and the inadequate supply. We hope to learn what really works and scale those best practices regionally and then nationally.

The supply of CS majors in the pipeline has been a concern for many years. In 2007, the Computer Science education community was alarmed by the lack of CS majors and enrollments in US colleges and universities.

Source: 2009-2010 CRA Taulbee Survey (http://www.cra.org/resources/)

This prompted the development of several programs and activities to start raising awareness about the demand and opportunities for computer scientists, and to spark the interest of K12 students in CS. For example, the NSF funded curriculum and professional development around the new CS Principles Advanced Placement course. The CSTA published standards for K12 CS and a report on the limited extent to which schools, districts and states provide CS instruction to their students. CS advocacy groups, Computing in the Core and Code.org have played an instrumental role in adding provisions to the reauthorization of the Elementary and Secondary School Act to support CS education. More generally, we have seen innovations in online learning with MOOCs, machine learning to provide personalized learning experiences, and platforms like Khan Academy that allow flipped classrooms.

All of these activities represent a convergence in the CS education space, where existing programs are ready for scale, and technological advancements can support that scale in innovative ways. Our Teaching Fellows will be testing after school programs, classroom curriculum and online CS programs to determine what works and why. They’ll start in the local Charleston area and then spread the best programs and curriculum to South Carolina, Georgia, North Carolina (where we also have large data centers). They are currently preparing programs for the fall semester.

We are very excited about the convergence we are seeing in CS education and the potential to bring many more kids into a field that offers not only great career opportunities but also a shot at really making a difference in the world. We’ll keep you posted on the progress of our Teaching Fellows.