More and more products and services are being deployed on the web, and this presents new challenges and opportunities for measurement of user experience on a large scale. There is a strong need for user-centered metrics for web applications, which can be used to measure progress towards key goals, and drive product decisions. In this note, we describe the HEART framework for user-centered metrics, as well as a process for mapping product goals to metrics. We include practical examples of how ...| research.google
Discover Google Research. We publish research papers across a wide range of domains and share our latest developments in AI and science research.| research.google
Classifying unsafe ad content has proven an enticing problem space for leveraging large language models (LLMs). The inherent complexity involved in identifying policy-violating content demands solutions capable of deep contextual and cultural understanding, areas of relative strength for LLMs over traditional machine learning systems. But fine-tuning LLMs for such complex tasks requires high-fidelity training data that is difficult and expensive to curate at the necessary quality and scale. S...| research.google
For AI models to perform well on diverse medical tasks and to meaningfully assist in clinician, researcher and patient workflows (like generating radiology reports or summarizing health information), they often require advanced reasoning and the ability to utilize specialized, up-to-date medical knowledge. In addition, strong performance requires models to move beyond short passages of text to understand complex multimodal data, including images, videos, and the extensive length and breadth o...| research.google
Wearable devices, from smartwatches to fitness trackers, have become ubiquitous, continuously capturing a rich stream of data about our lives. They record our heart rate, count our steps, track our fitness and sleep, and much more. This deluge of information holds immense potential for personalized health and wellness. However, while we can easily see what our body is doing (e.g., a heart rate of 150 bpm), the crucial context of why (say, "a brisk uphill run" vs. "a stressful public speaking ...| research.google
Generative AI and large language models (LLMs) facilitate the automated generation of assets across a variety of domains. For many use cases, several LLM agents may need to collaborate to create a joint output. A potential example is in the context of Internet ads, where advertisers might be represented by LLM agents capable of producing ads in reply to a user query. Or it could be that the LLMs represent stakeholders of a company, working together to write a joint report.| research.google
Posted by Shunyu Yao, Student Researcher, and Yuan Cao, Research Scientist, Google Research, Brain Team Recent advances have expanded the a...| research.google
Posted by Jason Wei and Denny Zhou, Research Scientists, Google Research, Brain team In recent years, scaling up the size of language models has be...| research.google
The Google PhD Fellowship Program recognizes outstanding graduate students doing exceptional work in computer science, related disciplines, or promising research areas.| research.google
Posted by Alexander B Wiltschko, Senior Research Scientist, Google Research Smell is a sense shared by an incredible range of living organisms, a...| research.google
Neural embedding models have become a cornerstone of modern information retrieval (IR). Given a query from a user (e.g., “How tall is Mt Everest?”), the goal of IR is to find information relevant to the query from a very large collection of data (e.g., the billions of documents, images, or videos on the Web). Embedding models transform each datapoint into a single-vector “embedding”, such that semantically similar datapoints are transformed into mathematically similar vectors. The emb...| research.google
Earth system models represent our best tool to predict and prepare for future changes to Earth’s environment. However, the immense computational cost of running these models at high resolution limits their ability to make regional projections at fine scales. Indeed, a typical limiting scale for these models is comparable in size to the island of Hawai’i (~100 km). Obtaining more granular projections, for instance at the city level (~10 km), is critical for planning everything from farming...| research.google
Many real-world planning tasks involve both harder “quantitative” constraints (e.g., budgets or scheduling requirements) and softer “qualitative” objectives (e.g., user preferences expressed in natural language). Consider someone planning a week-long vacation. Typically, this planning would be subject to various clearly quantifiable constraints, such as budget, travel logistics, and visiting attractions only when they are open, in addition to a number of constraints based on personal ...| research.google
The digital age offers ever growing access to vast amounts of knowledge, yet much remains locked behind complex language and specialist jargon. While complexity is often necessary in expert discourse, it can become a barrier when users need to understand information critical to their lives, such as navigating health information, understanding legal language, or grasping financial details. Tools that let users produce a simplified version of complex text that they encounter online can empower ...| research.google
Large language models (LLMs) that have been optimized through human feedback have rapidly emerged as a leading paradigm for developing intelligent conversational agents. However, despite their strong performance across many benchmarks, LLM-based agents can still lack multi-turn conversational skills such as disambiguation — when they are faced with ambiguity, they often overhedge or implicitly guess users' true intents rather than asking clarifying questions. Yet high-quality conversation s...| research.google
Each year at Google I/O, we share some of Google’s most advanced technologies. We show how they can be helpful and provide new experiences, and how developers and other communities can use them to innovate. Many of these new technologies emerged from years of work within Google Research, many in collaboration with other teams, building on multiple successive breakthroughs in AI and other areas of computer science. This year’s I/O highlights the impact of bringing research to reality. As S...| research.google
The machine learning community has consistently found that while modern machine learning (ML) models are powerful, they often need to be fine-tuned on domain-specific data to maximize performance. This can be problematic or even impossible, as informative data is often privacy-sensitive. Differential privacy (DP) allows us to train ML models while rigorously guaranteeing that the learned model respects the privacy of its training data, by injecting noise into the training process.| research.google
Differential privacy (DP) is a mathematically rigorous and widely studied privacy framework that ensures the output of a randomized algorithm remains statistically indistinguishable even if the data of a single user changes. This framework has been extensively studied in both theory and practice, with many applications in analytics and machine learning (e.g., 1, 2, 3, 4, 5, 6, 7).| research.google
Retrieval augmented generation (RAG) enhances large language models (LLMs) by providing them with relevant external context. For example, when using a RAG system for a question-answer (QA) task, the LLM receives a context that may be a combination of information from multiple sources, such as public webpages, private document corpora, or knowledge graphs. Ideally, the LLM either produces the correct answer or responds with “I don’t know” if certain key information is lacking.| research.google
Every day, billions of people shop online, hoping to replicate the best parts of in-store shopping. Seeing something that catches your eye, picking it up and inspecting it for yourself can be a key part of how we connect with products. But capturing the intuitive, hands-on nature of the store experience is nuanced and can be challenging to replicate on a screen. We know that technology can help bridge the gap, bringing key details to your fingertips with a quick scroll. But these online tools...| research.google
Generative AI models are capable of transforming aspects of life from education to innovation globally, but their reach is not matched by the breadth of their training data, which is limited in terms of languages, topics, and geographies.| research.google
Language model–based AI systems such as Articulate Medical Intelligence Explorer (AMIE, our research diagnostic conversational AI agent recently published in Nature) have shown considerable promise for conducting text-based medical diagnostic conversations but the critical aspect of how they can integrate multimodal data during these dialogues remains unexplored. Instant messaging platforms are a popular tool for communication that allow static multimodal information (e.g., images and docum...| research.google
In the 1660s, with the help of a simple, homemade light microscope that magnified samples more than 250 times, a Dutch fabric merchant named Antoine van Leeuwenhoek became the first person to document a close-up view of bacteria, red blood cells, sperm cells, and many other scientific sights. Since then, light microscopy has solidified its place as a bedrock technique in our quest to understand living organisms. Today, it is nearly ubiquitous in life science laboratories, enabling biologists ...| research.google
In the pursuit of scientific advances, researchers combine ingenuity and creativity with insight and expertise grounded in literature to generate novel and viable research directions and to guide the exploration that follows. In many fields, this presents a breadth and depth conundrum, since it is challenging to navigate the rapid growth in the rate of scientific publications while integrating insights from unfamiliar domains. Yet overcoming such challenges is critical, as evidenced by the ma...| research.google
AI models are increasingly applied in high-stakes domains like health and conservation. Data quality carries an elevated significance in high-stakes AI due to its heightened downstream impact, impacting predictions like cancer detection, wildlife poaching, and loan allocations. Paradoxically, data is the most under-valued and de-glamorised aspect of AI. In this paper, we report on data practices in high-stakes AI, from interviews with 53 AI practitioners in India, East and West African countr...| research.google
Google's Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines. | research.google
The latest research from Google| research.google
Posted by Jakob Uszkoreit, Software Engineer, Natural Language Understanding Neural networks, in particular recurrent neural networks (RNNs), are n...| research.google
While quantum processors in the noisy intermediate-scale quantum (NISQ) era demonstrate remarkable potential, they are susceptible to errors, i.e., noise, that accumulate over time and limit the number of qubits they can effectively handle. This poses a fundamental question: despite the limitations of noise in quantum computing, can these systems still provide practical value and outperform classical supercomputers in specific applications?| research.google
Quantum computers offer promising applications in many fields, ranging from chemistry and drug discovery to optimization and cryptography. Most of these applications require billions if not trillions of operations to execute reliably — not much compared to what your web browser is doing right now. But quantum information is delicate, and even state-of-the-art quantum devices will typically experience at least one failure in every thousand operations. To achieve their potential, performance ...| research.google
SQL has been extremely successful as the de facto standard language for working with data. Virtually all mainstream database-like systems use SQL as their primary query language. But SQL is an old language with significant design problems, making it difficult to learn, difficult to use, and difficult to extend. Many have observed these challenges with SQL, and proposed solutions involving new languages. New language adoption is a significant obstacle for users, and none of the potential repla...| research.google
Posted by Brendan McMahan and Daniel Ramage, Research ScientistsStandard machine learning approaches require centralizing the training data on one ...| research.google
Posted by Jacob Devlin and Ming-Wei Chang, Research Scientists, Google AI Language One of the biggest challenges in natural language processing (NL...| research.google
Multilingual Instruction Tuning With Just a Pinch of Multilinguality| research.google