January, 2009

This morning's spam filter issue

Posted by Brad Taylor, Software EngineerThis morning there was a problem with the implementation of Google's malware filters. Gmail's spam engine uses those filters (among hundreds of other signals) to help protect our users from malware, and so between 6:00 a.m. PST and 8:00 a.m. PST, we mistakenly sent some legitimate mail to people's spam folders.We're working to roll out an automated fix to put these legitimate messages back into your inboxes, and we expect this to happen within a day. In the meantime, if you were expecting a critical message this morning, please check your spam folder. (We tune our spam filters well enough that ordinarily you should never have to check your spam folder.)We're very sorry for the inconvenience. We'll update this post as we have more information to share.Update (2/1): We've rolled out a fix that has restored these messages to most people's inboxes, though to be on the safe side we'd still recommend that you check your spam folder if there was a critical message you expected to receive between 6am and 8am PST on Saturday.

This morning's spam filter issue

This morning there was a problem with the implementation of Google's malware filters. Gmail's spam engine uses those filters (among hundreds of other signals) to help protect our users from malware, and so between 6:00 a.m. PST and 8:00 a.m. PST, we mistakenly sent some legitimate mail to people's spam folders.We're working to roll out an automated fix to put these legitimate messages back into your inboxes, and we expect this to happen within a day. In the meantime, if you were expecting a critical message this morning, please check your spam folder. (We tune our spam filters well enough that ordinarily you should never have to check your spam folder.)We're very sorry for the inconvenience. We'll update this post as we have more information to share.Update (2/1): We've rolled out a fix that has restored these messages to most people's inboxes, though to be on the safe side we'd still recommend that you check your spam folder if there was a critical message you expected to receive between 6am and 8am PST on Saturday.Posted by Rishi Chandra, Google Apps Senior Product Manager

"This site may harm your computer" on every search result?!?!

If you did a Google search between 6:30 a.m. PST and 7:25 a.m. PST this morning, you likely saw that the message "This site may harm your computer" accompanied each and every search result. This was clearly an error, and we are very sorry for the inconvenience caused to our users.

What happened? Very simply, human error. Google flags search results with the message "This site may harm your computer" if the site is known to install malicious software in the background or otherwise surreptitiously. We do this to protect our users against visiting sites that could harm their computers. We maintain a list of such sites through both manual and automated methods. We work with a non-profit called StopBadware.org to come up with criteria for maintaining this list, and to provide simple processes for webmasters to remove their site from the list.

We periodically update that list and released one such update to the site this morning. Unfortunately (and here's the human error), the URL of '/' was mistakenly checked in as a value to the file and '/' expands to all URLs. Fortunately, our on-call site reliability team found the problem quickly and reverted the file. Since we push these updates in a staggered and rolling fashion, the errors began appearing between 6:27 a.m. and 6:40 a.m. and began disappearing between 7:10 and 7:25 a.m., so the duration of the problem for any particular user was approximately 40 minutes.

Thanks to our team for their quick work in finding this. And again, our apologies to any of you who were inconvenienced this morning, and to site owners whose pages were incorrectly labelled. We will carefully investigate this incident and put more robust file checks in place to prevent it from happening again.

Thanks for your understanding.

Update at 10:29 am PST: This post was revised as more precise information became available (changes are in blue). Here's StopBadware's explanation.

Posted by Marissa Mayer, VP, Search Products & User Experience

Google Toolbar in Firefox: a personalized new tab page

Those of you who use Google Toolbar on Firefox are probably pretty familiar with the many features Toolbar 5 (beta) has to offer — from bookmarks to buttons and search box to "send to." We've added a couple more features to the Firefox Toolbar, so feel free to download this "second beta" to get all the latest and greatest Google Toolbar has to offer.One of the features I'm really excited about is the new tab page. Now, when you open up a new tab, instead of the blank white page you see by default in Firefox, you will instead see small thumbnails of your favorite sites (up to 9), as well as recently-closed and bookmarked pages based on your browser history. You can edit the thumbnails, and all this data remains locally on your browser, which means none of the information about your most viewed sites or recently closed pages will be sent back to Google. If for whatever reason you don't like this updated new tab page, you can always change it back to a blank page or to the website of your choice through either your Toolbar or Firefox settings. Some tab extensions may conflict with this feature, and it currently isn't compatible with Firefox 2, so make sure you check your settings or visit the Google Toolbar help center if you are having any problems.Next time you want to go to your favorite site more quickly or restore an accidentally closed tab, you don't need to type out the URL. Typing Ctrl + T or double-clicking to the right of your open tabs will open up the new tab page with all your favorite sites right at your finger tips.In addition, this release now provides the ability for Hebrew- and Arabic-speaking users to access Toolbar 5 with robust right-to-left text support. We've also fixed some of the most reported bugs to give you a faster and more stable experience with this update.We're always working on improvements and new features to Google Toolbar based on your feedback, so keep those suggestions coming. Posted by Sergey Ryazanov, Software Engineer, Toolbar Team

Google Code Jam's Ranking Library Released

Posted by Bartholomew Furrow and Sebastian Kanthak, Google Code Jam Team
At Google, we like to use our own projects for internal development. Following that philosophy, one of the first apps ever made for Google App Engine was the contest platform for Google Code Jam. The application was a good fit: a rich web interface, real-time user interaction, and a heavily parallel design. But we faced one major challenge, which was how to handle the huge scoreboard -- in the first round, we had over 11,000 contestants.
While App Engine's datastore allows you to sort entities by score, it doesn't have a built-in mechanism to answer the following two requests:

  1. Given a user, what is his or her rank on the scoreboard? In other words, for some arbitrary row in that scoreboard, what is its position amongst the 11,000 spots? This is useful for showing your own rank, as well as the ranks of your friends.
  2. Give me all users on the n-th page of the scoreboard. While it is possible to get all users in sorted order, datastore queries don't allow you to start at page n, and iterating over all of the users on the first n pages takes too much time.

To solve these issues, we came up with a library that maintains a data structure to efficiently support these use cases. We imagine that other applications will have similar problems (e.g., a high-score list in a game), and so we're happy to release our work as the "google-appengine-ranklist" library.
The library supports three different operations:

  1. Setting the score of a given user.
  2. Given a score, what's the rank of a user with this score? This is used to answer the "What's the rank of person U?" use case.
  3. Given a rank, what's the score S for this rank? This can be used to solve the paging problem, by constructing a query that returns the first few users with a score less than or equal to S, in sorted order

If you think this library could be useful for you, take a look at the documentation and at the example application. We'd love to hear from you in our Google Group.

What we did on our winter break

After our big launch of the new look and feel, the Reader team took a breather and worked on tying up loose ends (don't worry, we also took some time off). We've just pushed a new release that includes the following set of fixes and tweaks:

  • Improvements to our rich mobile interface for iPhone, Android and other browsers, including a more compact page header, visual alignment tweaks and a faster loading time.
  • Links on the home page giving you easy access to your recently read and kept unread items.
  • Fixes for some "stuck" unread count issues.
  • Better handling of feeds and items with malformed URLs.
  • Less flickering when using the "Share with note" bookmark.
  • Fixing of errors when deleting lots of tags and folders.
  • Removal of extra image and backend requests, resulting in slightly faster loading times.

One other recent change is that we're now on Twitter. We've been fans of Twitter Search as a way of gathering feedback, but now we have a way of talking back. Feel free to send an @googlereader tweet to get our attention, and follow us for updates and quips.

Finally, we were honored to be nominated for a Bloggie. We encourage you to take a look at all the nominees in all the categories, it's a collection of some really great, subscribe-worthy, blogs

Experimental: Offline Gmail available in Gmail Labs

Offline Gmail is now available as an optional, experimental feature in Gmail Labs. Gmail Labs are unsupported features that may break or behave unexpectedly, so use your discretion before enabling this feature.Editions included:Standard, Premier, and Education EditionsLanguages included:US English, UK English (where Gmail Labs is available)How to access what's new:End users can enable offline Gmail from the 'Labs' tab in 'Settings'.Premier Edition administrators must first enable Gmail Labs for their domain for users to see this option. From the administrative control panel, choose 'Email' from the 'Service settings' menu. Check the box next to 'Enable Gmail Labs for my users' and click 'Save changes'. Gmail Labs are unsupported features that may break or behave unexpectedly, so use your discretion before enabling Gmail Labs for your users.For more information:http://googleenterprise.blogspot.com/2009/01/announcing-offline-access-in-gmail-labs.htmlGet these product update alerts by emailSubscribe to the RSS feed of these updates

Post-Commit Web Hooks for Google Code Project Hosting

By Mike Parent, Google CodeWe love to create building blocks that fuel user's imaginations. Today we are announcing Post-Commit Web Hooks for Project Hosting on Google Code, to allow web services to receive repository commit notifications. Projects can use this feature to integrate with all kinds of external tools, including continuous build systems, bug trackers, project metrics, and social networks. The notification follows the Web Hooks model, and is delivered as an HTTP POST request containing a JSON commit description.To help get you started, we've created an example application that performs various tests that are useful for managing documentation. You can find the source code in the contenthooks open source project.We would love to hear feedback about new and interesting uses of Post-Commit Web Hooks!

Bringing OpenID and OAuth Together

Posted by Yariv Adan, Google Security TeamWe are happy to announce an important enhancement to our recently launched OpenID endpoint. Google now supports the "Hybrid Protocol", combining OpenID federated login together with OAuth access authorization. Websites can now ask Google to sign-in a user using their Google Account, and at the same time request access to information available via OAuth-enabled APIs such as the Google Data APIs.For example, the website www.Plaxo.com is an early adopter of the new service and has already released a beta version supporting it for some of its new users. Plaxo's UI provides both a richer sign-in offering, using the Federated Login OpenID API, and a simple and secure way to import their Google Contacts using OAuth. In the past, sign-in required multiple redirects between Plaxo and Google, and more importantly, multiple user approval pages, one for OpenID during sign-in and another for the OAuth access authorization request. No more!The Hybrid Protocol allows Plaxo to encapsulate their OAuth authorization request inside the OpenID authentication request, letting Google know that the user wants to use both APIs. Google can now display a single approval page for both requests. Here is how the new user experience looks:In their sign in page, Plaxo offers their users the option to sign in using their Google Account and import their Gmail Contacts.The user is then redirected to the Google website and asked to confirm both sign in and access authorization requests.Finally, the user is redirected back to Plaxo, where she is already signed in and her Google contacts are available. If it's the first time the user signed-in using the Federated Login API, an additional instructive window will be displayed to ensure that the next sign-in experience will be as easy and successful as the first.Not only does the protocol allows a much better user experience as shown above, it also reduces the total number of browser redirects and roundtrips, reducing overall latency.To learn more about this new API see http://groups.google.com/group/google-federated-login-api/web/oauth-support-in-googles-federated-login-api. To make it easier for you to use the new API, we created a collaborative Open Source project together with other major vendors where you can download open source implementations for your Relying Party component. You are invited to contribute your own code and suggested best practices to this website.The Hybrid Protocol is a result of the ongoing effort by the OpenID and OAuth communities to make these protocols more useful for users and websites. Google is working together with the OpenID community to standardize the new protocol as a formal OpenID extension. If you want to help further these efforts and have an impact on what the next advancements are, you are welcome to join the OpenID and OAuth mailing lists.If you're interested in looking at some code, check out our working sample using the Google Data PHP client library. The source code is available here.

"And Together We Form Voltron" (or "How to Connect the Cloud")

The 80's cartoon Lion Force Voltron featured a team of five pilots commanding five robot lions, which could be combined together to form Voltron, a powerful do-gooder whose whole was greater than the sum of his parts. At the Cloud Connect event last week a bunch of developers came together to create our own cloud version of Voltron by combining cloud services from Google, Amazon, and Salesforce.com to create applications that could do good for businesses and consumers.The event started with Rajen Sheth, Senior Product Manager for Google Apps, making the case for why businesses should move their IT systems to the cloud to a panel of CIOs. Check out his presentation and leave a YouTube comment to let us know if you're convinced. During the second day to we ran 30 or so developer-to-developer training sessions where we had Google engineers instruct on implementing Google App Engine, and GData and Google Maps APIs with the hope that they'd use our services when building their cloud-based apps. And on the evening of the third day, we didn't boil the oceans, but coding teams did create some pretty sweet apps using only cloud services. One of the cooler ones was an socially-aware job search app for the iPhone which used Google App Engine, and Facebook and Google Apps APIs. Check out developer Claes Nygren's demo: Cloud Connect did a great job of testing the ease of interoperability and data portability between cloud computing providers. We want companies to be able to use the best services for their needs across multiple providers, and ensure that we are more flexible than traditional on-premise software platforms.There are two new ways that you too can play with robot action figures – I mean Google code. Test your app ideas in real-time at the new AJAX APIs Playground. Or register for Google I/O, Google's largest developer gathering, coming to San Francisco, CA from May 28 - 29, 2009.Posted by Kevin Gough, Google Apps Team