Google DeepMind

Google's AI research lab. Authors of 'Virtual Agent Economies', 'AI Agent Traps', 'Distributional AGI Safety', and 'Intelligent AI Delegation' (all in the paper canon). Co-leads a $10M joint funding call (with Schmidt Sciences, Cooperative AI Foundation, ARIA, Google.org), 'Scaling AI Safety for a Multi-Agent World', launched June 2026, to fund research on emergent risks in populations of interacting agents.

Category

Capabilities research, Empirical research

Import AI

Weekly updates on the latest developments in AI research (including governance), written by Jack Clark, co-founder of Anthropic.

Category

Newsletter

BlueDot Impact

Runs the standard introductory courses for people new to AI safety: Technical AI Safety and Frontier AI Governance, plus AGI Strategy. The Frontier AI Governance course includes a unit examining 'competitive dynamics between labs and states, power concentration' relevant to multi-agent/multi-principal AI ecosystems.

Category

Training and education

Center on Long-Term Risk (CLR)

Research, grants, and community-building around AI safety, focused on conflict scenarios as well as technical and philosophical aspects of cooperation between AI systems.

Category

Conceptual research, Strategy, Funding

Cooperative AI Foundation

Charitable foundation supporting research to improve the cooperative intelligence of advanced AI, funding grants, PhD fellowships, and workshops; produced the field-defining 'Multi-Agent Risks from Advanced AI' survey.

Category

Funding

Centre for the Governance of AI (GovAI)

Independent nonprofit research centre producing rigorous research on AI governance and fostering policy talent, spun out from Oxford's Future of Humanity Institute in 2021. Research spans international security, arms race dynamics, compute governance, and - notably for LAS - agent infrastructure and governance (e.g. recent work on private-actor vs. government roles in agent protocols/safety).

Category

Governance

AI X-risk Research Podcast (AXRP)

Interviews with (mostly technical) AI safety researchers about their research, aiming to get a sense of why it was written and how it might reduce existential risk from AI. Has interviewed David Duvenaud (Gradual Disempowerment co-author) and covered multi-agent/collective-behaviour research topics.

Category

Podcast

The Cognitive Revolution

Weekly podcast where host Nathan Labenz interviews AI innovators and thinkers, discussing the transformative impact AI will likely have in the near future. Has covered agent-to-agent economy dynamics and interviewed Zvi Mowshowitz on AI safety worldview and multi-agent-relevant topics.

Category

Podcast

PIBBSS Fellowship

3-month interdisciplinary program connecting researchers from diverse fields with AI safety mentors. Runs a dedicated Cooperative AI Track (funded by CAIF) focused on projects aiming to mitigate multi-agent AI risks and enhance cooperative intelligence of advanced AI.

Category

Training and education

AI Prospects

Blog by K. Eric Drexler on AI prospects and their surprising implications for technology, economics, environmental concerns, and military affairs.

Category

Blog

Alignment of Complex Systems Research Group (ACS)

Interdisciplinary research group studying multi-agent systems composed of humans and advanced AI, across scales from individual agents to civilization-scale systems. Core research threads include gradual disempowerment, LLM psychology/sociology, hierarchical agency, and a general 'science of the space of intelligent systems.' Built InterLab, an open-source toolkit for multi-agent interaction experiments.

Category

Conceptual research

Softmax

Independent research lab building the science of 'organic alignment' - the thesis that the kinds of agents that stay aligned with humans are the kinds that stay aligned with each other - through multi-agent reinforcement learning. Developed Cogames, an open-source benchmark/package for social intelligence games where agents cooperate and compete.

Category

Conceptual research, Empirical research

Equilibria Network

Builds mathematical frameworks (phase transition analysis and related tools) to predict where multi-agent AI systems break down, and identifies control levers to prevent those breakdowns. Explores the design space of collective intelligence through simulation and mathematical foundations.

Category

Conceptual research, Governance

ARIA (Advanced Research + Invention Agency)

UK government research funding agency modeled on DARPA, funding high-risk/high-reward technical research. Runs the 'Scaling Trust' programme (whose Programme Thesis v2.0 is in the paper canon) and co-funds the DeepMind-led $10M multi-agent AI safety call alongside Schmidt Sciences, Cooperative AI Foundation, and Google.org.

Category

Funding

Cooperative AI Summer School

Advanced 5-day program for early-career professionals already working in or studying cooperative AI.

Category

Training and education

Artificial Societies

Builds networks of AI personas (300-5,000+ per 'society') to simulate stakeholder and audience opinions for enterprise market research, comms, and policy testing.

Category

Capabilities research, Empirical research

Simile

Builds AI simulations of society populated by agents grounded in real human behavioral data, extending the original 'generative agents' research (Smallville). Offers enterprise workflows for customer/market simulation.

Category

Capabilities research, Empirical research

80,000 Hours

In-depth career advice organisation covering the world's most pressing problems, including AI safety, through a job board, problem profiles, and a podcast featuring extended interviews with researchers across the field.

Category

Career support

Metagov

Interdisciplinary research collective building standards, infrastructure, and empirical research for digital self-governance, DAOs, and online communities. Runs DAOstar (DAO governance standards body) and Govbase (crowdsourced governance database).

Category

Conceptual research

EvoMap

Agent-to-Agent (A2A) collaboration network implementing the Genome Evolution Protocol (GEP), letting AI agents package, share, and inherit validated problem-solving capabilities ('Gene Capsules') across models and organizations via a credit-based marketplace.

Category

Empirical research

Lida Safety Research

Small nonprofit dedicated to technical AI safety research focused on forecasting and simulation, particularly mitigating existential risk and gradual disempowerment; work feeds into AI governance and public education.

Category

Empirical research, Forecasting

Schmidt Sciences

Philanthropic science-funding organization founded by Eric and Wendy Schmidt. Co-leads and hosts the application portal for 'Scaling AI Safety for a Multi-Agent World', the $10M joint funding call with Google DeepMind, Cooperative AI Foundation, ARIA, and Google.org.

Category

Funding

Collective Intelligence Project

501(c)(3) nonprofit R&D lab building tools and datasets (e.g. Global Dialogues, Collective Generation, Weval) for democratic oversight and collective governance of AI, connecting global public input to AI labs, governments, and standards bodies.

Category

Governance

LargeAgentSystems.org

Resource hub for the emerging field of Large Agent Systems (LAS) safety - decentralised systems of AI agents with high-speed, complex interactions, often ingratiated with the real world and possibly including humans (e.g. agentic economies, societies, democracies). Covers threat models including distributional AGI, gradual disempowerment, and systemic financial risk from agent integration.

Category

Research support

Community Slack channel for discussion and coordination around Large Agent Systems (LAS) safety - the emerging field concerned with making decentralised, high-speed, complex multi-agent AI systems beneficial to human well-being.

Category

Resource

AI Safety South Africa (AISSA)

Capacity-building organisation focused on developing skills, networks, and community for preventing global catastrophic outcomes from advanced AI. Drives the Cooperative AI Research Fellowship (with CAIF, PIBBSS, and UCT's AI Initiative), serving as both talent pipeline and research direction-setter for UCT's emerging African AI Safety Hub.

Category

Video

Bank of England

UK central bank. Publisher of the April 2025 'Financial Stability in Focus' report on AI in financial services, already in the paper canon, examining systemic risk from AI adoption and concentration in financial markets.

Category

Video

Bittensor

Decentralized blockchain network structured as many interoperable 'subnets,' each an incentivized market for a specific AI/ML task (LLM pretraining, inference, data, image generation, etc.), coordinated via Yuma Consensus and priced in TAO.

Category

Video

International Monetary Fund (IMF)

International financial institution (190 member countries) actively researching and advising on AI's macroeconomic, labor-market, and financial-stability effects, including agentic AI's impact on payments systems, monetary policy transmission, and systemic risk. Publishes IMF Working Papers and Notes on these topics and integrates AI considerations into country surveillance.

Category

Video

National Bureau of Economic Research (NBER)

US economic research organization distributing working papers ahead of peer review across all fields of economics. Publisher of 'Automation and Repression' (Acemoglu, Gitmez & Shadmehr) already in the paper canon, and of the volume 'The Economics of Transformative AI' containing 'An Economy of AI Agents' (Hadfield & Koh), also in the canon.

Category

Video

Olas Network

Decentralized protocol (formerly Autonolas) for building, registering, and monetizing autonomous AI agents on-chain. Provides on-chain registries (agents/components as NFTs), a staking/bonding tokenomics model, and a Mech Marketplace for agent services.

Category

Video

Santa Fe Institute

Independent nonprofit research institute for the study of complex adaptive systems, historically foundational to agent-based modeling, network theory, and complexity science. Runs the Foundations of Intelligence program studying conceptual abstraction, analogy-making, and general theories of intelligence across natural and artificial systems.

Category

Video