Adversarial machine learning attacks, taxonomies, and mitigations across the ML lifecycle.
Adversarial ML gives useful language for thinking about evasion, poisoning, privacy attacks, and model misuse. It becomes especially valuable when teams need to distinguish classic model risk from the broader application-layer risks found in modern LLM systems.
These items are included because they add useful evidence, framing, implementation detail, or upcoming context for teams working in this area.
NIST finalizes AI 100-2e2025, providing a terminology and taxonomy for adversarial machine learning across predictive and generative AI systems.
garak 0.14.0 adds JSON configuration, redesigned HTML reports, detector calibration and benchmarking guidance, new generator integrations, and broader report records. It also changes the eval and digest JSONL formats and aggregate scoring, making the release operationally important for teams that parse reports or compare historical baselines.
GuardFall tests 11 open-source coding and computer-use agents against shell-command transformations that evade string and regex deny lists, including quote removal, $IFS expansion, command substitution, and encoded payloads. The study finds configuration- and model-dependent failures and shows that local or auto-approve modes can turn untrusted repository content into host command execution; it is vendor-authored research, not an independent benchmark.
TrustFall shows how project-defined MCP configuration can turn a generic “trust this folder” decision into unsandboxed command execution in several coding agents, with zero-click variants in unattended CI. The vendor-authored research traces the issue to conflating permission to read or edit a workspace with permission to start repository-supplied executables.
Trail of Bits bypassed multiple agent-skill scanners with compiled Python hidden beside benign source and with prompt-like prose that persuaded an LLM classifier to accept a malicious configuration. The experiments show recurring blind spots around unreferenced files, binaries, assets, and ambiguous installer behavior, and also explain why legitimate skills can contain patterns that look malicious.
Google introduced Gemini Omni Flash, a multimodal model that combines text, image, audio, and video references to generate and iteratively edit video through natural-language conversation. Generated videos include a SynthID watermark.
Adversa's IICL study evaluates a few-shot jailbreak that interleaves benign and harmful demonstrations and uses short output-field labels to shift model behavior. Across more than 3,500 probes, ten models, and seven ablations, results vary materially with example order and field names; the work is vendor-authored and its model-specific attack rates should be independently reproduced.
Trail of Bits describes supervising GPT-5.5-Cyber as it built ASan and UBSan variants, derived seed corpora, and wrote fuzz harnesses for roughly a dozen zlib entry points in one day. The useful result is the workflow and its emphasis on reachability and reportability; vulnerability details remain under coordinated disclosure and the speed comparison is the authors' estimate.
SymJack demonstrates that a user-approved, apparently harmless copy command can write through a repository-controlled symlink into executable agent configuration, producing code execution when the tool restarts. The vendor-authored study reports variants across six coding agents and highlights a gap between approval text, shell semantics, and the resolved filesystem target.
Release notes for garak, an LLM vulnerability scanning and evaluation toolkit. Relevant to tracking new probes, detectors, and repeatable red-team workflows.
garak release adding probes and detector improvements for LLM security testing. Relevant to maintaining practical red-team coverage across evolving attack techniques.
OWASP’s GenAI security project remains a practical baseline for teams building or assessing LLM applications and agentic systems.
Google Cloud outlines a defense-in-depth view of AI security spanning application controls, data protections, and infrastructure isolation.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
garak release with new generators, probe metadata, and evaluation workflow improvements. Relevant to maintaining repeatable LLM security testing coverage.
Play video
This AI Explained video reviews a major AI development through the lens of benchmarks and evaluation evidence. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of agentic workflows and tool-use risk. It is useful context for AI engineering, evaluation, governance, and operational risk.
Play video
This AI Explained video reviews a major AI development through the lens of governance and responsible deployment. It is useful context for AI engineering, evaluation, governance, and operational risk.