Events - Apple Machine Learning Research
Read updates about machine learning research, events, and programs from Apple.| Apple Machine Learning Research
Read updates about machine learning research, events, and programs from Apple.| Apple Machine Learning Research
We introduce SlowFast-LLaVA-1.5 (abbreviated as SF-LLaVA-1.5), a family of video large language models (LLMs) offering a token-efficient solution for long-form video understanding. We incorporate the two-stream SlowFast mechanism into a streamlined training pipeline, and perform joint video-image training on a carefully curated data mixture of only publicly available datasets. Our primary focus is on highly efficient model scales (1B and 3B), demonstrating that even relatively small Video LLM...| Apple Machine Learning Research
Language models must be adapted to understand and follow user instructions. Reinforcement learning is widely used to facilitate this -- typically using fixed criteria such as "helpfulness" and "harmfulness". In our work, we instead propose using flexible, instruction-specific criteria as a means of broadening the impact that reinforcement learning can have in eliciting instruction following. We propose "Reinforcement Learning from Checklist Feedback" (RLCF). From instructions, we extract chec...| Apple Machine Learning Research
A recent paper from Apple researchers, "The Super Weight in Large Language Models," reveals that an extremely small subset of parameters in LLMs (in some cases, a single parameter) can exert a disproportionate influence on an LLM’s overall functionality (see Figure 1). This work highlights the critical role of these “super weights” and their corresponding “super activations,” offering a new insight into LLM architecture and avenues for efficient model compression. The paper provides...| Apple Machine Learning Research
This paper was accepted at the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) 2025 Non-negative Matrix Factorization (NMF) is a powerful technique for analyzing regularly-sampled data, i.e., data that can be stored in a matrix. For audio, this has led to numerous applications using time-frequency (TF) representations like the Short-Time Fourier Transform. However extending these applications to irregularly-spaced TF representations, like the Constant-Q tran...| Apple Machine Learning Research
Large language models (LLMs) have achieved impressive performance, leading to their widespread adoption as decision-support tools in resource-constrained contexts like hiring and admissions. There is, however, scientific consensus that AI systems can reflect and exacerbate societal biases, raising concerns about identity-based harm when used in critical social contexts. Prior work has laid a solid foundation for assessing bias in LLMs by evaluating demographic disparities in different languag...| Apple Machine Learning Research
Corpus Aware Training (CAT) leverages valuable corpus metadata during training by injecting corpus information into each training example, and has been found effective in the literature, commonly known as the "tagging" approach. Models trained with CAT inherently learn the quality, domain and nuance between corpora directly from data, and can easily switch to different inference behavior. To achieve the best evaluation, CAT models pre-define a group of high quality data before training starts...| Apple Machine Learning Research
Large language models (LLMs) struggle to consistently generate UI code that compiles and produces visually relevant designs. Existing approaches to improve generation rely on expensive human feedback or distilling a proprietary model. In this paper, we explore the use of automated feedback (compilers and multi-modal models) to guide LLMs to generate high-quality UI code. Our method starts with an existing LLM and iteratively produces improved models by self-generating a large synthetic datase...| Apple Machine Learning Research
We show the performance of Automatic Speech Recognition (ASR) systems that use semi-supervised speech representations can be boosted by a complimentary pitch accent detection module, by introducing a joint ASR and pitch accent detection model. The pitch accent detection component of our model achieves a significant improvement on the state-of-the-art for the task, closing the gap in F1-score by 41%. Additionally, the ASR performance in joint training decreases WER by 28.3% on LibriSpeech, und...| Apple Machine Learning Research
UI prototyping often involves iterating and blending elements from examples such as screenshots and sketches, but current tools offer limited support for incorporating these examples. Inspired by the cognitive process of conceptual blending, we introduce a novel UI workflow that allows developers to rapidly incorporate diverse aspects from design examples into work-in-progress UIs. We prototyped this workflow as Misty. Through an exploratory first-use study with 14 frontend developers, we ass...| Apple Machine Learning Research
Apple believes that privacy is a fundamental human right. As AI experiences become increasingly personal and a part of people's daily lives…| Apple Machine Learning Research
Vision Language Models (VLMs) enable visual understanding alongside textual inputs. They are typically built by passing visual tokens from a…| Apple Machine Learning Research
Apple researchers are advancing AI and ML through fundamental research, and to support the broader research community and help accelerate…| Apple Machine Learning Research
The increasing capabilities of large generative models and their ever more widespread deployment have raised concerns about their…| Apple Machine Learning Research
Large-scale models are routinely trained on a mixture of different data sources. Different data mixtures yield very different downstream…| Apple Machine Learning Research
This paper was accepted to the ACL 2025 main conference as an oral presentation. This paper was accepted at the Scalable Continual Learning…| Apple Machine Learning Research
With Apple Intelligence, we're integrating powerful generative AI right into the apps and experiences people use every day, all while…| Apple Machine Learning Research
With the increasing integration of speech front-ends and large language models (LLM), there is a need to explore architectures that…| Apple Machine Learning Research
Apple machine learning teams are engaged in state of the art research in machine learning and artificial intelligence. Learn about the latest advancements.| Apple Machine Learning Research
Recent generations of frontier language models have introduced Large Reasoning Models (LRMs) that generate detailed thinking processes…| Apple Machine Learning Research
At Apple, we believe privacy is a fundamental human right. And we believe in giving our users a great experience while protecting their…| Apple Machine Learning Research
Large generative models are becoming increasingly capable and more widely deployed to power production applications, but getting these…| Apple Machine Learning Research
This study explores using embedding rank as an unsupervised evaluation metric for general-purpose speech encoders trained via…| Apple Machine Learning Research
Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy…| Apple Machine Learning Research
Nonverbal behaviors such as posture, gestures, and gaze are essential for conveying internal states, both consciously and unconsciously, in…| Apple Machine Learning Research
This paper introduces a framework, called EMOTION, for generating expressive motion sequences in humanoid robots, enhancing their ability to…| Apple Machine Learning Research
At Apple, we believe privacy is a fundamental human right. Our work to protect user privacy is informed by a set of privacy principles, and…| Apple Machine Learning Research
This paper was accepted at the Efficient Natural Language and Speech Processing (ENLSP) Workshop at NeurIPS 2024. The pre-training phase of…| Apple Machine Learning Research
This paper was accepted at the Efficient Natural Language and Speech Processing (ENLSP) Workshop at NeurIPS 2024. While large language…| Apple Machine Learning Research
Apple is sponsoring the annual meeting of the Association for Computational Linguistics (ACL), which takes place in person from August 11 to…| Apple Machine Learning Research
Build amazing machine-learned experiences with Apple. Discover opportunities for researchers, students, and developers.| Apple Machine Learning Research
Voice assistants increasingly use on-device Automatic Speech Recognition (ASR) to ensure speed and privacy. However, due to resource…| Apple Machine Learning Research
At the 2024 Worldwide Developers Conference, we introduced Apple Intelligence, a personal intelligence system integrated deeply into…| Apple Machine Learning Research
Explore advancements in state of the art machine learning research in speech and natural language, privacy, computer vision, health, and more.| Apple Machine Learning Research
Today, we are excited to release optimizations to Core ML for Stable Diffusion in macOS 13.1 and iOS 16.2, along with code to get started…| Apple Machine Learning Research
