Publications

The Seoul AI Safety Summit in May 2024 saw the announcement of a global network of AI Safety Institutes. We analyse what progress has been made, how plans and motivations differ, and what can we learn about how to set up AISIs effectively.

European Commission President Ursula von der Leyen proposed a bold new initiative to create a “CERN for AI,” aiming to boost Europe’s competitiveness through AI innovation. This effort would pool resources across the continent, similar to the successful CERN model for physics research. But what would it actually entail? Our new report dives into the details.

When current generation AI models are deployed to the market, they usually contain a set of safety features. We argue that those mechanisms are not sufficient and look at ways how they can be circumvented.

Our contribution to the High Level Group advising the European Commission on the interim evaluation of the EU’s main R&I budget, Horizon Europe FP9.

Creating new laws is one thing; enforcing them is the real challenge. Maria Koomen and Raegan McDonald make four suggestions on how the EU’s ambitious tech regulations can be enforced more effectively.

Neurotechnologies – devices that can read from and write to the brain – are widely considered the next frontier of medical treatment and human enhancement. To meet this moment, ICFG is building a neurotechnology program to anticipate the rapid developments in the field and the risks they could pose to society, assess the robustness of existing legislation in the EU and more widely, and explore options for regulatory review.

In its first 100 days of power, the next EU Commission can address the risks and opportunities of powerful emerging technologies. As we gear up for the so-called ‘Implementation Commission’, ICFG experts lay out the big ideas we believe will put President von der Leyen’s second term on the right track.

Competitiveness has already become the new mantra for the next European Commission, but the topic of the European Commission’s competitiveness as an institution is absent. For the next mandate, the EU should look to empower an individual at the heart of the institutions and bring a focus on innovation home.

On May 21st, 2024, after three years of painstaking negotiations, the European Council approved the final version of the European Union’s landmark AI Act. But, what does the AI Act do exactly? Does it do it well? And what can other jurisdictions drafting AI legislation learn from the EU’s approach to AI regulation?

The 2024 ESPAS report is not only a thorough examination of the biggest trends that will shape the future of Europe, but also a key document for understanding the direction of the next European Commission. Dig into ICFG’s analysis here.

The capabilities of AI systems continue to accelerate rapidly, driven by unprecedented increases in computing power. In a new report, ICFG examines the centrality of compute to the existing AI development paradigm and examines the critical role that the governance of compute could take in mitigating the risks associated with advanced AI.

With the next EU leadership in sight, it is time for EU institutions and member states to level-up enforcement for a stronger digital Europe. Announcing ICFG’s Enforcement Initiative – a collaborative project designed to set the bar higher and ensure technology works for democracy in an age of accelerating innovation.

ICFG proposes that the G7 countries back an international institution that designs international standards for the responsible management of compute resources. Such standards would help expand AI governance to cover the entire AI lifecycle, as appropriate for such a high-risk technology, and as proposed in the 2023 Hiroshima Guiding Principles.

Recent breakthroughs in AI innovation have only heightened the urgency for closer international cooperation and the swift implementation of safety measures, particularly those addressing the most capable, most high-risk advanced AI systems. Here are ICFG’s recommendations for world leaders attending the May 2024 AI Safety Summit.

ICFG’s approach puts policymakers’ needs at the centre. We listen to their concerns and questions in order to support the EU’s stated goal of promoting international talks to govern these powerful, transformational climate interventions. This piece is part of our briefing kit on climate interventions for EU policymakers, alongside our FAQ.

Earlier this year, ICFG conducted a round of interviews with EU policymakers to assess the current gaps when it comes to technological climate interventions that are becoming more and more prevalent in political discourse. Click through for a summary of what EU policymakers asked about and our initial round of responses.

This Climate Interventions Program Factsheet outlines the basics – what climate interventions are, why they matter, what their impacts could be, and what ICFG is working towards on this topic. These Factsheets are intended to provide a baseline for our conversations with policymakers.

Governments must learn from past challenges and actively steer technological innovation, prioritising democratic principles and positive social impact over industry profits. As the global order comes under increasing strain, political leaders must look beyond the ballot box and focus on mitigating the long-term risks posed by emerging technologies.

From the steam engine to electricity to the internet, technology has had a profound impact on the modern world. In the coming decade, five powerful emerging technologies are poised to fundamentally reshape societies. Our new report current state of these technologies and outlines the potential risks they pose in the absence of responsible governance.

Advanced AI presents policymakers with some particularly tricky regulatory challenges. One of these challenges is the sudden emergence of unexpected capabilities in AAI systems. Policymakers and legislative bodies, as well as civil society, should be aware of the limitations of governance tools that rely on evaluating models for capabilities to differentiate low-risk from high-risk models.

The EU stands at a crossroads with the Artificial Intelligence Act. A key decision looms: whether to exempt foundation models from this regulatory framework or not. This choice is not a matter of European competitiveness – it represents a foundation stone for responsibly governing our technology-infused future.

What is ‘frontier artificial intelligence’ and why were international leaders sufficiently concerned by it to attend a two-day summit? We take a look at what differentiates frontier AI from other forms of AI, and why even its developers don’t fully understand how it works.