Attacking GenAI applications and LLMs - Sometimes all it takes is to ask nicely! - hn security
Generative AI and LLM technologies have shown […]| hn security
AI agents can be tricked into covertly performing malicious actions by websites that are hidden from regular users’ view, JFrog AI architect Shaked Zychlinski has found. This novel approach allows attackers to inject prompts / instructions into these autonomous AI-powered “assistants”, allowing them to hijack agent behavior for their own malicious goals. Indirect prompt-injection poisoning attacks where hidden harmful instructions are embedded inside the same page the human visitor sees...| Help Net Security
Researchers reveal zero-click exploits that let hackers hijack AI agents from OpenAI, Microsoft, and Google to steal data and disrupt workflows. The post AI Agents Vulnerable to ‘Silent Hijacking,’ Security Researchers Warn appeared first on eSecurity Planet.| eSecurity Planet
Ein harmlos wirkendes Word-Dokument, eine Cloud-Anbindung und schon sind Ihre vertraulichen Daten weg. Israelische Sicherheitsforscher zeigen: ChatGPT und andere KI-Assistenten lassen sich erschreckend einfach zur Datenspionage missbrauchen.| Jörg Schieb | Digital und KI
Generative AI and LLM technologies have shown […]| hn security
Security researchers documented a prompt injection vulnerability in an agent created with Copilot Studio that allowed the exfiltration of customer data. Microsoft has fixed the problem, but the researchers figure that natural language prompts and the way that AI responds means that other ways will be found to cause agents to do silly things. Microsoft 365 tenants need to think about the deployment and management of agents.| Office 365 for IT Pros
We compare the effectiveness of content filtering guardrails across major GenAI platforms and identify common failure cases across different systems. We compare the effectiveness of content filtering guardrails across major GenAI platforms and identify common failure cases across different systems.| Unit 42
We examine an LLM jailbreaking technique called "Deceptive Delight," a technique that mixes harmful topics with benign ones to trick AIs, with a high success rate. We examine an LLM jailbreaking technique called "Deceptive Delight," a technique that mixes harmful topics with benign ones to trick AIs, with a high success rate.| Unit 42
Developers should be using OpenAI roles to mitigate LLM prompt injection, while pentesters are missing vulnerabilities in LLM design.| Include Security Research Blog
