Key Moments
Google London Test Automation Conference Opening
Google TalksKey Moments
Google's mobile testing faces immense fragmentation, with over 11,000 user agents and 100+ JSRs for Java, making automation crucial but difficult due to device and carrier inconsistencies.
Key Insights
The mobile market is projected to be the primary way users access data in the future, with over 2.2 billion mobile voice subscribers globally.
Google's approach to testing relies on building fault-tolerant software for cheap commodity hardware rather than buying expensive, reliable equipment.
Mobile testing is complicated by a highly fragmented ecosystem, including 3-6 carriers per country, hundreds of device types, and up to 85 Java specifications (JSRs) with varying levels of adoption.
The London Test Automation Conference (LTAC) was initiated by Google to address a perceived lack of technical depth in existing test automation conferences and to foster community discussion.
Google employs both dedicated testers and developers practicing test-driven development (TDD) as part of their agile methodologies.
The conference emphasizes informal discussions and networking, with initiatives like writing short essays to attend and open badges to facilitate topic-based connections.
Google's principled approach to quality and innovation
Shannon Maher opens the Google London Test Automation Conference (LTAC) by discussing Google's unique philosophy on testing and quality. Unlike traditional approaches that might involve purchasing expensive, fault-tolerant hardware, Google focuses on building software that is resilient to failures, even when running on cheap, commodity components. This principle extends to how Google handles incorrect search results: instead of solely trying to debug and fix algorithms, they leverage massive amounts of data and user feedback over time, often through machine learning, to refine answers. This iterative, data-driven approach is also reflected in their product release strategy, exemplified by products like Gmail and Google Earth, which were launched in beta and continued to evolve based on user feedback. This "fail fast and learn" mentality is core to Google's innovation, especially when introducing new products where market viability is initially uncertain.
The mobile revolution and its testing implications
The importance of mobile testing is underscored by the sheer scale of the mobile market. With over 2.2 to 2.4 billion mobile voice subscribers globally, and mobile penetration exceeding 100% in many developed markets, mobile devices are becoming the primary gateway to information. This rapid shift necessitates a robust testing strategy. Google is actively developing a wide range of mobile products, including mobile search, maps, Blogger, and personalized Gmail, highlighting the critical need to test these applications across diverse platforms and network conditions. The presentation emphasizes that mobile will likely be the dominant way users access the internet in the future, making effective mobile testing not just a technical challenge but a strategic imperative for Google's mission to organize the world's information.
The fragmented landscape of mobile testing
Testing mobile applications presents significant challenges due to extreme fragmentation. Unlike the relatively standardized internet where IP addresses lead to data centers, the mobile world involves diverse communication mechanisms such as SMS, push notifications, and various proprietary protocols. Device fragmentation is also severe, with numerous manufacturers, operating systems (Symbian, Microsoft, Palm), and device models. Furthermore, carriers (3-6 per country, with up to 50-60 in the US) introduce their own complexities and inconsistencies. Coding languages for mobile content, including HTML and multiple mobile-specific markup languages, add another layer of difficulty. Standards like SyncML, while important, are often poorly implemented or inconsistently adopted across platforms. This complexity means that even a seemingly simple task like browsing Google's vast data index on a mobile device requires navigating a labyrinth of over 11,000 tracked user agents and numerous device capabilities, making manual testing infeasible.
The necessity of automation in mobile testing
Given the overwhelming fragmentation and scale of the mobile ecosystem, automation is not just beneficial but essential for testing. The sheer number of mobile devices, user agents (over 11,000 tracked), and Java specifications (JSRs), with an estimated 85 relevant and 20 critical ones, makes manual testing impossible. Without automation, addressing even a fraction of these variables would require an impossibly large team. However, automating mobile testing is itself fraught with difficulty. Devices are often not fully programmable, lack control interfaces, and can be hard to debug or repair. This means that while automation is the only viable path forward, it requires significant investment and innovative solutions to overcome the inherent challenges of the mobile platform and its complex value chain, which involves carriers, OEMs, and content providers.
Google's approach to the mobile value chain and partnerships
Google's strategy in the mobile space involves navigating a complex value chain that differs significantly from the PC internet. Instead of direct interaction between a service provider and the end-user, the mobile ecosystem includes multiple intermediaries: carriers and Original Equipment Manufacturers (OEMs). This necessitates building strong partnerships with these players, as well as with Independent Software Vendors (ISVs). Dealing with large, established companies like carriers and OEMs, who have billions in market value and operate with hundreds of millions of subscribers, requires a disciplined approach to product integration and testing. Google must align its testing strategies with the more rigorous processes of these partners, which can be challenging but is crucial for widespread product adoption. This collaborative yet demanding environment underscores the need for flexible, comprehensive, and well-integrated testing plans.
The genesis and philosophy of the London Test Automation Conference
The LTAC conference was established because Google perceived a lack of technical depth in existing test automation conferences. By hosting their own event, Google could curate the content to their specific interests and standards. The conference was designed to be accessible, with no registration fees, and focused on fostering deep technical discussions and community engagement. A unique aspect was the requirement for attendees to submit a short essay explaining their interest in attending, which helped gauge the audience's background and focus. This essay submission also served as a valuable feedback mechanism for shaping the conference. All talks were recorded and made available online, extending the conference's reach globally to those who could not attend in person.
Encouraging interaction and future dialogue
The conference structure itself was designed to promote interaction. Ample time was allocated for Q&A after each session, and 15-minute breaks were scheduled to encourage networking. Attendees were encouraged to use their name badges to find others with shared interests or discover new topics. A 'speed dating' for ideas was facilitated through a 'lightning talk' session, where speakers had five minutes to present a quick idea or insight. Post-conference, Google planned to follow up with attendees via email surveys to gather feedback on all aspects of the event, ensuring continuous improvement. Additionally, all video recordings would be shared online, and a Google Group listserv was created to maintain ongoing discussions and build a lasting community around test automation.
Future prospects in mobile technology and testing
The future of mobile technology holds immense potential, with areas like mobile payments and location-based e-commerce being multi-billion dollar markets. While technologies like AI are beginning to proliferate, their widespread adoption is still hindered by the very testing challenges discussed. The evolution of mobile cameras also presents new avenues for innovation and testing. Google's long-term vision is to enhance mobile capabilities beyond basic communication, enabling devices to perform more complex tasks and deliver rich, immersive experiences. This involves mastering the fundamentals of mobile testing to unlock these future possibilities, transforming the mobile phone into a far more powerful and versatile tool for accessing information and services.
Mentioned in This Episode
●Software & Apps
●Companies
●Organizations
●Concepts
●People Referenced
Common Questions
Google emphasizes an innovative, principled approach, focusing on holistic quality and considering various angles. They often prioritize releasing products quickly in beta to gather user feedback, embracing flexibility and iteration.
Topics
Mentioned in this video
Where Google began as an academic project around 1997, highlighting the company's deep roots in academia.
Independent Software Vendors, mentioned as partners Google works with in the mobile space, along with hardware manufacturers and network operators.
Where Alan Hutcheon began his work in test automation in the 1990s, focusing on next-generation networking platforms.
Where Alan Hutcheon worked after Indiana University on automation platforms for several years.
Mentioned for its role in creating a unified platform for personal computers, which contrasted with the fragmented mobile platform, and for the speaker's past work in Redmond, Washington.
The company hosting the Test Automation Conference in London, with a history rooted in academic research and a focus on innovation in testing and product development.
Mentioned as a method used to tune answers and improve systems over time based on data and usage.
Mentioned as a common development methodology used within Google teams, emphasizing test-driven development.
The technology focus of Alan Hutcheon's co-founded startups in Silicon Valley.
Mentioned as a relevant markup language for mobile content, noting its continued importance despite other formats.
Mentioned as a specific technology used by the US, contributing to the diversity of communication technologies in the mobile space.
The platform where all conference lectures will be published after the event, making them accessible globally.
Discussed as a potential solution for mobile development that turned out to be highly fragmented with numerous standard specifications (JSRs).
Mentioned as a mobile platform with compatibility issues, highlighting the fragmentation in the mobile operating system landscape.
The blog where news and information about the conference and future Google research activities will be posted.
Mentioned as a communication protocol used for mobile devices, contributing to platform fragmentation.
Highlighted as a product with a humorous comparison to Gmail's beta status, illustrating Google's product release philosophy.
Mentioned as an operating system for mobile phones where compatibility issues exist, highlighting the lack of complete standardization.
Mentioned in the context of testing JavaScript, with a presentation scheduled on the topic.
Used as an analogy for a typical conference experience where one might learn but not necessarily find a community of passionate individuals.
Mentioned as a location where the first Google testing team in Europe was established, alongside London.
The location where Alan Hutcheon co-founded two startups, primarily in the Wi-Fi technology space.
More from GoogleTalksArchive
View all 92 summaries
58 minEverything is Miscellaneous
54 minStatistical Aspects of Data Mining (Stats 202) Day 7
45 minKey Phrase Indexing With Controlled Vocabularies
63 minMysteries of the Human Genome
Ask anything from this episode.
Save it, chat with it, and connect it to Claude or ChatGPT. Get cited answers from the actual content — and build your own knowledge base of every podcast and video you care about.
Get Started Free