Google Flu Trends gets a brand new engine

Posted by Christian Stefansen, Senior Software Engineer

Each year the flu kills thousands of people and affects millions around the world. So it’s important that public health officials and health professionals learn about outbreaks as quickly as possible. In 2008 we launched Google Flu Trends in the U.S., using aggregate web searches to indicate when and where influenza was striking in real time. These models nicely complement other survey systems—they’re more fine-grained geographically, and they’re typically more immediate, up to 1-2 weeks ahead of traditional methods such as the CDC’s official reports. They can also be incredibly helpful for countries that don’t have official flu tracking. Since launching, we’ve expanded Flu Trends to cover 29 countries, and launched Dengue Trends in 10 countries.

The original model performed surprisingly well despite its simplicity. It was retrained just once per year, and typically used only the 50 to 300 queries that produced the best estimates for prior seasons. We then left it to perform through the new season and evaluated it at the end. It didn’t use the official CDC data for estimation during the season—only in the initial training.

In the 2012/2013 season, we significantly overpredicted compared to the CDC’s reported U.S. flu levels. We investigated and in the 2013/2014 season launched a retrained model (still using the original method). It performed within the historic range, but we wondered: could we do even better? Could we improve the accuracy significantly with a more robust model that learns continuously from official flu data?

So for the 2014/2015 season, we’re launching a new Flu Trends model in the U.S. that—like many of the best performing methods [1, 2, 3] in the literature—takes official CDC flu data into account as the flu season progresses. We’ll publish the details in a technical paper soon. We look forward to seeing how the new model performs in 2014/2015 and whether this method could be extended to other countries.

As we’ve said since 2009, "This system is not designed to be a replacement for traditional surveillance networks or supplant the need for laboratory-based diagnoses and surveillance." But we do hope it can help alert health professionals to outbreaks early, and in areas without traditional monitoring, and give us all better odds against the flu.

Stay healthy this season!

Learning Statistics with Privacy, aided by the Flip of a Coin

Posted by Úlfar Erlingsson, Tech Lead Manager, Security Research

(Cross-posted on the Chromium Blog and the Google Online Security Blog)

At Google, we are constantly trying to improve the techniques we use to protect our users' security and privacy. One such project, RAPPOR (Randomized Aggregatable Privacy-Preserving Ordinal Response), provides a new state-of-the-art, privacy-preserving way to learn software statistics that we can use to better safeguard our users’ security, find bugs, and improve the overall user experience.

Building on the concept of randomized response, RAPPOR enables learning statistics about the behavior of users’ software while guaranteeing client privacy. The guarantees of differential privacy, which are widely accepted as being the strongest form of privacy, have almost never been used in practice despite intense research in academia. RAPPOR introduces a practical method to achieve those guarantees.

To understand RAPPOR, consider the following example. Let’s say you wanted to count how many of your online friends were dogs, while respecting the maxim that, on the Internet, nobody should know you’re a dog. To do this, you could ask each friend to answer the question “Are you a dog?” in the following way. Each friend should flip a coin in secret, and answer the question truthfully if the coin came up heads; but, if the coin came up tails, that friend should always say “Yes” regardless. Then you could get a good estimate of the true count from the greater-than-half fraction of your friends that answered “Yes”. However, you still wouldn’t know which of your friends was a dog: each answer “Yes” would most likely be due to that friend’s coin flip coming up tails.

RAPPOR builds on the above concept, allowing software to send reports that are effectively indistinguishable from the results of random coin flips and are free of any unique identifiers. However, by aggregating the reports we can learn the common statistics that are shared by many users. We’re currently testing the use of RAPPOR in Chrome, to learn statistics about how unwanted software is hijacking users’ settings.

We believe that RAPPOR has the potential to be applied for a number of different purposes, so we're making it freely available for all to use. We'll continue development of RAPPOR as a standalone open-source project so that anybody can inspect and test its reporting and analysis mechanisms, and help develop the technology. We’ve written up the technical details of RAPPOR in a report that will be published next week at the ACM Conference on Computer and Communications Security.

We’re encouraged by the feedback we’ve received so far from academics and other stakeholders, and we’re looking forward to additional comments from the community. We hope that everybody interested in preserving user privacy will review the technology and share their feedback at rappor-discuss@googlegroups.com

DAA SF Presents ‘Getting Data Right: Optimizing Performance Using Quantitative & Qualitative Practices’

This year the annual Digital Analytics Association (DAA) San Francisco Symposium is taking place on Thursday November 13^th  and will be hosted at University of San Francisco. The theme is "optimizing performance using quantitative and qualitative practices". The DAA SF chapter has lined up industry leaders who will be sharing their thoughts and experiences. We are expecting a great afternoon of discussion followed by a networking reception. Following are the event details:

Our lineup of wonderful speakers is focused on presenting real-world solutions to the optimization challenges we all face everyday. They will be discussing the principles of qualitative data collection, optimization and the relation to quantitative data they've put into practice. This year we are piloting a new, more interactive format: 15 minutes of content from speakers followed by 10 min of Q&A from the audience, so come ready to participate in a day of analytics sharing and learning, and leave with ideas to put into practice. Our lineup includes:

• Sarah Cho, Survey Monkey
• Jason Conrad, Foresee
• Joe Megibow, Chief Digital Officer, American Eagle
• Krista Seiden, Analytics Advocate, Google
• Jim Sterne, Founder, eMetrics
• Rahul Todkar, Intuit
• Jimmy Wong, LinkedIn
• Ryan Zander, Sport Vision

By attending you will:

• Learn how industry leaders use quantitative AND qualitative data to improve their businesses
• Discover how your analytics can drive savvy optimization across multiple channels
• Uncover how marketing attribution's data output leads to more informed marketing decisions
• Learn the latest on optimization best practices from practitioners and vendors 

Theme: Optimizing performance using Quantitative AND Qualitative practices

When: November 13th, registration starts at 12:30, program 1:00pm to to 5:30pm, followed by a networking reception

Where: University of San Francisco, 2130 Fulton Street, San Francisco, CA 94117

Cost: $25 for DAA members/$75 for non-members

Event website and registration: register here.

Space is limited so register early!

This Symposium is organized by local DAA members and volunteers. We encourage you to become a member of the DAA and join our local efforts. Become a member and reach out to one of the local chapter leaders, Krista, Charles or Feras.

HDR+: Low Light and High Dynamic Range photography in the Google Camera App

Posted by Marc Levoy, Google[x] Software Engineering Manager and Professor Emeritus, Stanford University

As anybody who has tried to use a smartphone to photograph a dimly lit scene knows, the resulting pictures are often blurry or full of random variations in brightness from pixel to pixel, known as image noise. Equally frustrating are smartphone photographs of scenes where there is a large range of brightness levels, such as a family photo backlit by a bright sky. In high dynamic range (HDR) situations like this, photographs will either come out with an overexposed sky (turning it white) or an underexposed family (turning them into silhouettes).

HDR+ is a feature in the Google Camera app for Nexus 5 and Nexus 6 that uses computational photography to help you take better pictures in these common situations. When you press the shutter button, HDR+ actually captures a rapid burst of pictures, then quickly combines them into one. This improves results in both low-light and high dynamic range situations. Below we delve into each case and describe how HDR+ works to produce a better picture.

Capturing low-light scenes

The camera on a smartphone has a small lens, meaning that it doesn't gather much light. If a scene is dimly lit, the resulting photograph will contain image noise. One solution is to lengthen the exposure time - how long the sensor chip collects light. This reduces noise, but since it's hard to hold a smartphone perfectly steady, long exposures have the unwanted side effect of blurring the shot. Devices with optical image stabilization (OIS) sense this "camera shake” and shift the lens rapidly to compensate. This allows longer exposures with less blur, but it can’t help with really dark scenes.

HDR+ addresses this problem by taking a burst of shots with short exposure times, aligning them algorithmically, and replacing each pixel with the average color at that position across all the shots. Averaging multiple shots reduces noise, and using short exposures reduces blur. HDR+ also begins the alignment process by choosing the sharpest single shot from the burst. Astronomers call this lucky imaging, a technique used to reduce the blurring of images caused by Earth's shimmering atmosphere.

A low light example is captured at dusk. The picture at left was taken with HDR+ off and the picture at right with HDR+ on. The HDR+ image is brighter, cleaner, and sharper, with much more detail seen in the subject’s hair and eyelashes. Photos by Florian Kainz

Capturing high dynamic range scenes

Another limitation of smartphone cameras is that their sensor chips have small pixels. This limits the camera's dynamic range, which refers to the span between the brightest highlight that doesn't blow out (turn white) and the darkest shadow that doesn't look black. One solution is to capture a sequence of pictures with different exposure times (sometimes called bracketing), then align and blend the images together. Unfortunately, bracketing causes parts of the long-exposure image to blow out and parts of the short-exposure image to be noisy. This makes alignment hard, leading to ghosts, double images, and other artifacts.

However, bracketing is not actually necessary; one can use the same exposure time in every shot. By using a short exposure HDR+ avoids blowing out highlights, and by combining enough shots it reduces noise in the shadows. This enables the software to boost the brightness of shadows, saving both the subject and the sky, as shown in the example below. And since all the shots look similar, alignment is robust; you won’t see ghosts or double images in HDR+ images, as one sometimes sees with other HDR software.

A classic high dynamic range situation. With HDR+ off (left), the camera exposes for the subjects’ faces, causing the landscape and sky to blow out. With HDR+ on (right), the picture successfully captures the subjects, the landscape, and the sky. Photos by Ryan Geiss

Our last example illustrates all three of the problems we’ve talked about - high dynamic range, low light, and camera shake. With HDR+ off, a photo of Princeton University Chapel (shown below) taken with Nexus 6 chooses a relatively long 1/12 second exposure. Although optical image stabilization reduces camera shake, this is a long time to hold a camera still, so the image is slightly blurry. Since the scene was very dark, the walls are noisy despite the long exposure. Therefore, strong denoising is applied, causing smearing (below, left inset image). Finally, because the scene also has high dynamic range, the window at the end of the nave is blown out (below, right inset image), and the side arches are lost in darkness.

Click here to see the full resolution image. Photo by Marc Levoy

HDR+ mode performs better on all three problems, as seen in the image below: the chandelier at left is cleaner and sharper, the window is no longer blown out, there is more detail in the side arches, and since a burst of shots are captured and the software begins alignment by choosing the sharpest shot in the burst (lucky imaging), the resulting picture is sharp.

Click here to see the full resolution image. Photo by Marc Levoy

Here's an album containing these comparisons and others as high-resolution images. For each scene in the album there is a pair of images captured by Nexus 6; the first was was taken with HDR+ off, and the second with HDR+ on.

Tips on using HDR+

Capturing a burst in HDR+ mode takes between 1/3 second and 1 second, depending on how dark the scene is. During this time you'll see a circle animating on the screen (left image below). Try to hold still until it finishes. The combining step also takes time, so if you scroll to the camera roll right after taking the shot, you'll see a thumbnail image and a progress bar (right image below). When the bar reaches 100%, your HDR+ picture is ready.

Should you leave HDR+ mode on? We do. The only times we turn it off are for fast-moving sports, because HDR+ pictures take longer to capture than a single shot, or for scenes that are so dark we need the flash. But before you turn off HDR+ for these action shots or super-dark scenes, give it a try; we think you'll be surprised how well it works!

At this time HDR+ is available only on Nexus 5 and Nexus 6, as part of the Google Camera app.

Helping teachers teach computer science

Posted by Karen Parker, Education Program Manager and Jason Ravitz, Education Evaluation Manager

(Cross-posted on the Google for Education Blog)

Since 2009, Google’s CS4HS (Computer Science for High School) grant program has connected more than 12,000 computer science (CS) teachers with skills and resources to teach CS in fun and relevant ways. An estimated 600,000 students have been impacted by the teachers who have completed CS4HS professional development workshops so far. Through annual grants, nearly 230 colleges and universities have hosted professional development workshops worldwide.

Grantees use the funds to develop CS curriculum and professional development workshops tailored for local middle and high school teachers. These workshops expose teachers to CS curriculum using real-world applications that spark students’ curiosity. As feedback from those teachers rolls in, we want to share some highlights from what we’ve learned so far.

What went well:

• 89% of participants reported they would recommend their workshop to others
• 44% more participants reported a “high” or “very high knowledge” of CS after their workshop vs. before
• More than half of participants said they would use “most” or “all” of the activities or resources presented during their workshop.
• In 2014 the number of teachers who took part in a CS4HS professional development workshop increased by 50%, primarily due to the funding of multiple MOOCs.

Ways to make a bigger impact:

• Just 53% of participants said they felt a sense of community among the other workshop participants. Research by Joyce & Showers (2002) and Wiske, Stone, & Levinson (1993) shows that peer-to-peer professional development, along with ongoing support, helps teachers implement new content, retain skills, and create lasting change. We’ll explore new ways to build community among participants as we plan future workshops.
• 83% of participants reported being Caucasian, which is consistent with the current demographics of CS educators. This indicates a need to increase efforts in diversifying the CS teacher population.
• Outcome measures show us that the most knowledge gains were among teachers who had no prior experience teaching CS or participating in CS professional development -- a population that made up just 30% of participants. While we see that the workshops are meeting a need, there remains an opportunity to develop materials geared toward more experienced CS teachers while also encouraging more new teachers to participate.

We know there are many challenges to overcome to improve the state of CS teacher professional development. We look forward to sharing new ideas for working in partnership with the CS education community to help address those challenges, in particular by helping more teachers teach computer science.

At the University of Sydney CS4HS workshop teachers are learning how to teach

Computer Science without a computer during a CS Unplugged activity.

Online Retailers: The Secret to Seasonal Success in 2014

Plan ahead.

Over the past few years, we’ve seen the holiday rush begin earlier and earlier. And we’ve also noticed that the shopping frenzy is extending beyond the traditional season, with transaction rates in 2013 boasting lifts even after Christmas. These trends make for a lot of opportunity for online retailers, but you need to play your cards right. The most important thing you can do to ensure seasonal success is to plan your digital strategy now.  We’ve analyzed transaction behavior from a portion of our Google Analytics accounts over the 2013 holiday season to develop a guide for seasonal success. 

Which days drive the most transactions?

In order to develop a successful holiday retail strategy, it’s important to first understand the days that drive the most sales for your business. Once you understand this, you can craft a strategy that optimizes your media and promotion not only for these days, but for the entire holiday season.

As digital retailers are well aware, Cyber Monday is THE digital shopping day of the year.  It generates the most transactions of any single day and, in 2013, saw a transaction rate lift of 170% over average.  In second place is Black Friday, a day that has increased in digital importance over the last three years. By 2013, the transaction rate on Black Friday was 114% higher than average.  Beyond these two stars, the table below shows you the top days in 2013 by transaction volume and the lift in transactions rates on each day.

In general, beyond Cyber Monday and Black Friday, the Mondays and Tuesdays before Christmas in December tend to generate the highest volume of transactions.  Interestingly, the highest transaction days are not correlated to the days with the most sessions (traffic to your site), so avoid using the top session days as a proxy for the top transaction days.

How can I drive sales on the top transaction days?

The holiday season generates some of the highest transaction rate spikes and the lowest dips for the entire year. In order to drive digital sales successfully, it’s important that you adjust your bids for auction-based media, such as search ads, appropriately to account for higher transaction rates on key dates and throughout the season.  As you navigate the holiday season, use the lift in transaction rates over the average transaction rate as your bid multiplier for auction-based media (learn more about bid adjustments). This adjustment schedule reflects the increased value of clicks that are more likely to convert, and helps ensure that you stay ahead of competitors and  get in front of the right consumers. The transaction rate lift for the top days are shown in the table above, while the chart below gives you an idea of the lift for the weeks surrounding the holiday season.

As you can see, the top days generate some of the biggest transaction rate lifts, but there’s also plenty of upside in the weeks preceding Thanksgiving as well as right before Christmas.  A smart retailer will generate a bid schedule for the entire season, starting 4-5 weeks before the Thanksgiving holiday.

Marketing to consumers is notoriously tricky and often trend-driven, making the holiday season a difficult and uncertain time for retailers. However, with proper pre-season preparation, digital retailers can set themselves up for seasonal success in 2014.  If you’re a Google Analytics user, you can tailor this analysis and approach to your business, using your own data and the data available in our benchmarking tool.  For more strategies for the holiday season, check out our holiday tips blog post and best practices checklist.

About the Data

In order to perform this analysis, we looked at billions of sessions across millions of Google Analytics accounts. We used session and transaction trends; and we looked at the percentage of sessions that included a transaction to calculate transaction rates. The data includes only accounts that have authorized Google to share website data in an anonymous way (read more). For questions, comments, or praise please contact us at gaqi@google.com 

Posted by Daniel Waisberg and Jocelyn Whittenburg from the Google Analytics team.

Making tag management more accessible and powerful

Today we are happy to introduce improvements to Google Tag Manager that will make both marketing and IT teams happy: 

• New APIs that tailor the power of Google Tag Manager to your unique needs
• A new intuitive interface to help you launch and edit tags even faster
• More 3rd-party templates to make tagging easier

Many large enterprises use Google Tag Manager to streamline and simplify website and mobile app tagging. It helps marketers control the end-to-end process of adding website tags, while IT departments save time they can spend on more strategic projects. InsureandGo has been using Google Tag Manager for all their tagging needs:

“Before, we missed opportunities because tag changes required a website release. Since we’ve enabled Google Tag Manager on the site, it’s enabled the marketing team to measure more on-site actions. For example, using Google Tag Manager to track on-page events such as specific clicks and form submissions helps us understand more granular customer actions, how to market and what to sell. We can make decisions much quicker and see within a few weeks whether the strategy has worked, whereas before it would have taken six to nine months.” Simon Everett, Head of Marketing

Let's look at the new features.

Introducing Google Tag Manager API

Sometimes you just want things your own way. We understand! The new full-featured Google Tag Manager API lets you customize the infrastructure to suit your needs, whether that means building your own tools or better integrations with your existing workflow. From creating and managing users to previewing and publishing containers and tags, the API provides all the power of the web interface.

For example, the new API makes it easy to manage user access in bulk. It's easy to set permissions for many users at once, or set up your own role-based permissions and let the API give the right level of access to the right people in your organization.

Agencies can use the API to easily manage large tagging setups for their clients: create a master container template, specify variations (such as the domain, or the ad campaign ID) in a Google Sheets doc, and use the API to automatically deploy to multiple containers and keep those containers in sync. 

Our partner Novartis has been able to scale more easily by using Google Tag Manager APIs:

We have a strong data-driven culture at Novartis and thus in the digital space we’re naturally interested in using data and insights to improve the usability and experience of our websites. With many brands and websites across the globe, collecting web analytics data can become time consuming. Two challenges we have faced are data consistency and tagging implementation across many sites. We developed a process where we use the Google Tag Manager API to eliminate a manual, error-prone, process and thus were able to shift our attention from several low-value tasks to determining how to create a great digital experience for our customers.

Angela Grammatas - Digital Analytics Manager for Novartis

More coverage for 3rd-party tags

Starting in the next few weeks, you'll see more 3rd-party templates in the tag creation flow. We've made it easier for marketers to add tags and minimize errors while doing so. When adding a new tag of your own, you'll select from a list of 3rd-party providers and be underway in just a few clicks. We now offer support for tags from AdRoll, Marin, Comscore, Bizo, Clicktale, Neustar, Distillery, Turn, Mediaplex, VisualDNA, quantcast, Criteo and many more to come soon. Don't see the tag you need? No problem: you can add it immediately as a custom HTML tag. You can also ask to have a new tag template included in future releases, as Tag Manager will continue to add new tag templates. You'll find the full list of tag templates in our help center.

Creating a new tag (click image for full-size).

A more intuitive interface

We think tag management should be easy even for non-technical users. Even if you're new to Google Tag Manager, you'll be using the improved interface within minutes. Tasks are intuitive and structured much the same way as in AdWords and Google Analytics. Our new updates include:

• A default workflow that's simpler and clearer
• Instant search and autocomplete that can help you find anything in your Google Tag Manager containers
• New keyboard shortcuts to simplify life for power users

The goal: enable marketing managers to easily add and update tags.

The new container overview page (click image for full-size).

We are confident you'll find the new Google Tag Manager easier to use and a more powerful solution for your web and app tagging needs. If you are already using Google Tag Manager, you can try out the new user interface today by logging in your accounts and following the instructions. New to Google Tag Manager? Get started today!

Posted by Lukas Bergstrom, Product Manager Google Tag Manager

Smart Autofill - Harnessing the Predictive Power of Machine Learning in Google Sheets

Posted by Konstantin Davydov, Software Engineer and Afshin Rostamizadeh, Research Scientist

Much of Google’s work on language, speech, translation, and visual processing relies on machine learning, where we construct and apply learning algorithms that make use of labeled data in order to make predictions for new data. What if you could leverage machine learning algorithms to learn patterns in your spreadsheet data, automatically build a model, and infer unknown values?

You can now use machine learning to make predictions in Google Sheets with the newly launched Smart Autofill Add-on. With a single click, Smart Autofill predicts the missing values of a partially filled column in your spreadsheet by using the data of other related columns. Smart Autofill uses the non-missing data to learn patterns and differs from the standard "Auto-fill" feature of Sheets, which attempts to fill in only simple patterns that it already knows (e.g. calendar dates, days of the week, ordered numbers).

As an example, in the screenshots below, we give four very simple characteristics of used vehicles (year, number of miles, number of doors, and type: car or truck) as well as the price for some of the vehicles. Since the prices are probably correlated with the characteristics of the vehicle, we can use Smart Autofill to estimate what the missing prices should be. The rows that do contain a price will be used as examples to learn from in order to fill in the rows with a missing price.

Smart Autofill uses Google's cloud-based machine learning service Prediction API, which trains several linear as well as non-linear classification and regression models. The best model is automatically chosen for your problem by finding the one with the smallest misclassification error (for categorical data) or root-mean-squared error (for numeric data) calculated by using cross-validation on the labeled (non-empty) set of examples.

To use Smart Autofill, after following the installation procedure, simply select "Add-ons > Smart Autofill > Start" which will open a sidebar. Select a block of data that includes the column to Autofill and click "Next". Finally, from the selected data, choose a target column to Autofill and click "Start" (Figure 1). Now just sit back as Smart Autofill does its work and fills in the missing values (Figure 2).

Figure 1: Highlighting the dataset and selecting the target column.

Figure 2: After clicking "Start" a model is trained and applied to automatically fill in the missing values of the target column. Note, the estimated error of the model is reported in the sidebar.

An estimate of the error-rate of the model (based on the non-missing data) is shown in the sidebar after the missing values are filled. The accuracy of Smart Autofill (as well as the accuracy of the estimated error) depends on many factors, including the amount and quality of the data provided. While not all datasets will be ideally suited for machine learning, we hope our more in-depth tutorial will provide an idea of the range of problems where Smart Autofill can be effective.

While the vehicle pricing example is relatively simple (in reality used vehicle prices are a function of more than just four variables), more complex datasets could have many more non-target columns as well as data rows. Also, the target column does not need to be numeric, since Smart Autofill can also predict categorical values (i.e. in the car example the target column value could have contained the categories "expensive", "moderate", "affordable" instead of price). Other illustrative scenarios include:

• You have a spreadsheet that holds the results of a customer survey, but one of the columns (e.g. "overall satisfaction 1-5") has some missing values. If the other columns of the survey can help indicate overall satisfaction then you can try using Smart Autofill to estimate the missing values.
• You keep a spreadsheet of restaurants that you've visited and their characteristics (type: Italian, ambiance: quiet, cost: $$$, etc.) and whether you enjoyed the restaurant or not. Now you can add the characteristics of new restaurants to your spreadsheet and use Smart Autofill to guess at which ones you might enjoy.

The example dataset and more detailed tutorial for the add-on can be found here. We hope you discover new and useful ways to incorporate the predictive power of machine learning with your data.

All the News that's Fit to Read: A Study of Social Annotations for News Reading

Posted by Chinmay Kulkarni, Stanford University Ph.D candidate and former Google Intern, and Ed H. Chi, Google Research Scientist

News is one of the most important parts of our collective information diet, and like any other activity on the Web, online news reading is fast becoming a social experience. Internet users today see recommendations for news from a variety of sources; newspaper websites allow readers to recommend news articles to each other, restaurant review sites present other diners’ recommendations, and now several social networks have integrated social news readers.

With news article recommendations and endorsements coming from a combination of computers and algorithms, companies that publish and aggregate content, friends and even complete strangers, how do these explanations (i.e. why the articles are shown to you, which we call “annotations”) affect users' selections of what to read? Given the ubiquity of online social annotations in news dissemination, it is surprising how little is known about how users respond to these annotations, and how to offer them to users productively.

In All the News that’s Fit to Read: A Study of Social Annotations for News Reading, presented at the 2013 ACM SIGCHI Conference on Human Factors in Computing Systems and highlighted in the list of influential Google papers from 2013, we reported on results from two experiments with voluntary participants that suggest that social annotations, which have so far been considered as a generic simple method to increase user engagement, are not simple at all; social annotations vary significantly in their degree of persuasiveness, and their ability to change user engagement.

News articles in different annotation conditions

The first experiment looked at how people use annotations when the content they see is not personalized, and the annotations are not from people in their social network, as is the case when a user is not signed into a particular social network. Participants who signed up for the study were suggested the same set of news articles via annotations from strangers, a computer agent, and a fictional branded company. Additionally, they were told whether or not other participants in the experiment would see their name displayed next to articles they read (i.e. “Recorded” or “Not Recorded”).

Surprisingly, annotations by unknown companies and computers were significantly more persuasive than those by strangers in this “signed-out” context. This result implies the potential power of suggestion offered by annotations, even when they’re conferred by brands or recommendation algorithms previously unknown to the users, and that annotations by computers and companies may be valuable in a signed-out context. Furthermore, the experiment showed that with “recording” on, the overall number of articles clicked decreased compared to participants without “recording,” regardless of the type of annotation, suggesting that subjects were cognizant of how they appear to other users in social reading apps.

If annotations by strangers is not as persuasive as those by computers or brands, as the first experiment showed, what about the effects of friend annotations? The second experiment examined the signed-in experience (with Googlers as subjects) and how they reacted to social annotations from friends, investigating whether personalized endorsements help people discover and select what might be more interesting content.

Perhaps not entirely surprising, results showed that friend annotations are persuasive and improve user satisfaction of news article selections. What’s interesting is that, in post-experiment interviews, we found that annotations influenced whether participants read articles primarily in three cases: first, when the annotator was above a threshold of social closeness; second, when the annotator had subject expertise related to the news article; and third, when the annotation provided additional context to the recommended article. This suggests that social context and personalized annotation work together to improve user experience overall.

Some questions for future research include whether or not highlighting expertise in annotations help, if the threshold for social proximity can be algorithmically determined, and if aggregating annotations (e.g. “110 people liked this”) help increases engagement. We look forward to further research that enable social recommenders to offer appropriate explanations for why users should pay attention, and reveal more nuances based on the presentation of annotations.

Announcing the Google CS Engagement Small Awards Program

Posted by Leslie Yeh Johnson, University Relations

(cross-posted on the Google for Education blog)

College students are more interested than ever in studying computer science. There has been an unprecedented increase in enrollment in Computer Science undergraduate programs over the past six years. Harvard University’s popular introductory CS course CS50 has recently claimed the spot as the most enrolled course on campus. An astounding 50% of Harvey Mudd’s graduates received engineering degrees this year. However, while the overall number of students in introductory computer science courses continue to climb, the number of students who go on to complete undergraduate degrees in this field, particularly among women and under-represented minorities, does not match this increase in individual course enrollment (2013 Taulbee Survey).

Recent findings show that while students may begin a CS degree program, retaining students after their first year remains an issue. Research indicates that one of the strongest factors in the retention of students in undergraduate CS degrees is early exposure to engaging courses and course material, such as high quality assignments that are meaningful and relevant to the student’s life or classroom activities that encourage student-to-student interaction. When an instructor or department imbeds these practices into the introductory CS classroom, students remain excited about CS and are more likely to complete their undergraduate CS degree.

At Google we believe in the importance of preparing the next generation of computer scientists. To this end, we’ve created the CS Engagement Small Grants Program to support educators teaching introductory computer science courses in reaching their engagement and retention goals. We’ll give unrestricted gifts of $5,000 to the selected applicants’ universities, towards the execution of engaging CS1 or CS2 courses in the 2014-2015 school year. We encourage educators who are teaching CS1 and CS2 courses at the post-secondary level to apply to the Google CS Engagement Small Grants Program. Applications will be accepted through November 15, 2014 and will be evaluated on an ongoing basis. If you’re interested in applying, please check out the Call for Proposals.

Dynamic remarketing now available to advertisers across all verticals

This post originally appeared on the Inside AdWords Blog.

Over the next few weeks we are rolling out dynamic remarketing to all verticals including hotels, flights, real estate, classifieds, jobs, auto, finance and education. Since we launched dynamic remarketing for retailers, many advertisers like Bebe Stores, Netshoes, and Build Direct have been driving better results and more profits from their remarketing campaigns.

Highlighting what matters most to your customers

Dynamic remarketing shows site visitors tailored ads that feature the products they viewed on your website, and related products they might be interested in. Let’s say you sell cars and offer hundreds of makes and models in multiple cities. With dynamic remarketing, you can build one ad that will dynamically show tailored messaging to your site visitors, like the cars they engaged with on your site, and similar cars in that city and price range. Beta clients across multiple verticals reported a 2x increase in conversion rates and 60% reduction in CPA, on average, when they added dynamic ads to their remarketing campaigns.*

Display ads built for mobile, optimized for conversion value 

• When advertisers add mobile targeting to their remarketing campaigns, we’ve seen conversion volume increase by 15% on average at the same price. That's why all our dynamic remarketing templates are mobile-optimized to deliver ads seamlessly across screens. Learn more. 
• Dynamic remarketing with automated bidding can boost performance by calculating optimal bids for each impression in real-time. This means if you sell an $800 camera and a $20 flashlight, AdWords will automatically prioritize winning more of the $800 conversions than the $20 conversions. This helps to maximize the total value of your conversions, not just the number of conversions.  Learn more. 

Customers driving success with dynamic remarketing  

In travel, Hotel Urbano built a single dynamic ad that showed each prospective traveler the package, hotel or cruise most relevant to them. Dynamic ads improved their return on ad spend by 38%, and drove 415% more revenue compared to standard remarketing. Mariana Filippo, senior marketing analyst at Hotel Urbano, says it made business more efficient “since we don’t have to change creatives every day across over 4,000 packages across 183 different countries. It keeps up with our frequency so we can deliver the right ad with the right user.”

In flights, Jet Airways customized ads based on where and when people were looking to fly, so someone browsing flights from New York to London could see a special offer on business class tickets on the exact day they’re looking to fly. They doubled conversions at a 65% lower CPA by adding dynamic ads to their remarketing strategy.




In local deals, India's largest online B2B marketplace IndiaMART used dynamic remarketing to recommend new suppliers to existing customers, increasing lead volume by 400% at a 60% lower CPA. Saugata Halder, product marketing manager at IndiaMART offers this advice: "For marketplaces like us, it’s imperative to showcase the wide range of products available to our buyers. Dynamic remarketing allowed us to reach a larger segment of our customers with personalized ads and to maximize the impact of our marketing efforts.”

Join us for a Hangout on Air

Learn how you can get started with Dynamic Remarketing. Join us for a live Hangout on Air with Google product managers and experts on Thursday, October 9th at 9:00am PT/ RSVP here.

For more information visit the AdWords Help Center. If you’re a Google Analytics user, you can visit the Google Analytics Help Center to learn how to use your existing tags to get started with dynamic remarketing.

Posted by Jyoti Vaidee, Product Manager Dynamic Display Ads

---

*Campaign results may vary across advertisers