Speakers
Audrey Tang, cyber ambassador-at-large for Taiwan; Taiwan’s first minister of digital affairs
Bart Becks, board member of the European Innovation Council; founder of Skynet and Genisys
Marietje Schaake, fellow at Stanford University’s Institute for Human-Centered Artificial Intelligence; former member of the European Parliament
Sam Ramadori, co-president and executive director of LawZero; previously chief executive of BrainBox AI
Moderator: Eddie Milev, Economist Enterprise
Summary
This panel discussion examines how a handful of labs decide which artificial-intelligence models to release and when, even as governments treat the technology as a race for national advantage. The panellists explore what democratic oversight looks like in practice, from Taiwan’s public assemblies and rules for tech companies to Europe’s choices about where to compete and where to build on existing strengths in the AI stack. They also ask whether safer AI demands fundamentally different kinds of models, as well as better rules for the systems being built today. On responsibility, the discussion turns to legal accountability and independent investigations when AI systems cause harm. At the heart of their exchange lies a practical dilemma: how democracies can rein in the dangers of AI without forfeiting its benefits.
Transcript
Part I: Power and the fight to shape AI
Eddie Milev: Good morning, everyone. Thank you very much for joining us. We have an exciting panel today: “Who shapes AI’s destiny?” It’s the big question that everyone in the room, I’m sure, is looking to answer. We have four brilliant panelists to answer it with me. We are going to start with power, then we’ll move to control and responsibility. But before we do this, there are three things I think happened this year that shape the context for our conversation.
The first was in July, when agents run by OpenAI in their testing environments escaped their sandboxes and hacked a series of organizations. A number of [AI] labs then discovered that their agents had done the same, and no humans were involved.
The second is that just last month, the five big men of the AI labs, who are rarely in agreement, actually stood behind a line, as Dario [Amodei] put it, to pace the frontier.
The response to this is the third really big thing that happened, which was that Donald Trump declared the pacing of the frontier a hoax, said that America needs to win in AI. Another development from Washington came earlier in the year, when foreign nationals were restricted from accessing Anthropic’s Fable 5. That was, of course, Anthropic’s leading model at the time.
So, Marietje, I want to start with you. Marietje spent 10 years in the European Parliament and then went to Stanford to write a book, The Tech Coup. So, having seen what’s happened in the last year, do you think that the state has stepped in? Is the coup over, or is the power still with the five big men of the AI labs?
Marietje Schaake: So, just to pick up on what you said about the OpenAI hacking incident, humans are always involved. The question is how. And it’s true that the AI labs still have a lot of power because they can decide when to release models. They can decide on the risk appetite that they have in the experimentation phase. They have enormous amounts of capital, and they also can not only pace the frontier, but they can also pace the release of information. In other words, we’re all on a sort of drip feed of what it is they want to tell us about hacks, about lessons learned, about risk, and so on. And so I think that’s an enormous power.
But we are also seeing a reckoning that these experiments are going beyond what a lot of people find acceptable, and the notion of a race between companies, a race between the US and China, begs the question: What are we racing to?
And so I think that over the past months we have been seeing much more concern from all kinds of corners about this outsized power of companies—what I call a tech coup, that is in many ways much further advanced than when I wrote the book. And what governments do with it differs, but I worry most about what democratic governments do, because for a long time we heard the narrative that, you know, the US has to race China and win the race so that democratic AI wins. But unfortunately, this is not something we can take for granted. We see troubling backsliding of democracy in the United States, and AI companies also being instruments of repression, for example, against demonstrators, of rounding up undocumented immigrants, of more surveillance. And so I think the real question is: if you care about democracy, as we sit here in a country in the heart of Europe, what are democratic governments going to do?
And so, the fact that Donald Trump has a zero-sum vision of how he wants to instrumentalize AI for American dominance—it’s not a game that we should want to play here in Europe or in the Netherlands. We should really choose a different path, a different path of AI, a different path of governance and a different way to protect our self-interest, our self-determination as citizens and as countries.
Eddie Milev: Thank you. So you started by raising the question of responsibility from the very beginning, and we’ll get to that by the end of our conversation. And then you are essentially saying that yes, there is a concentration of power, but you are also suggesting that there might be ways in which we can fight back, standing here in Europe.
And Audrey, you are probably well placed to tell us a little bit about this. Not only do you come from the island that has quite a bit of control and power in the AI stack, being the place where all the AI chips are made, but you’ve also really thought about how to dissipate power in a similar way that Marietje suggests. What is your take? Who has power right now in AI, and what are some of the levers to control it?
Audrey Tang: Thank you. Yeah, I think we, the people, are truly the superintelligence, and we shape AI. That's the idea that we’ve been running in Taiwan for more than a decade now. When I was born in '81, people didn't really type into computers. We typed into terminals, which went into mainframes, which is power concentration in a very pure form. However, then came the personal computing revolution, and so when you type into your local spreadsheet or your local desktop publishing, you do not need to wait for the system administrator to do post-training or upgrade your models on the mainframe. Now, in Taiwan, for the past 10 years, we’ve been negotiating with big tech like that. So, for example, in 2015, when Uber came to Taiwan, we invited taxi drivers, Uber drivers, to go to this pro-social media called Polis, where you cannot reply to each other or dunk each other, but can like and unlike each other. So, using AI to advance democracy, we find the uncommon grounds and legalize Uber in a way that makes the taxi unions also feel fair.
And so, very similarly, recently we negotiated with big tech like Meta and Google, so that when they run advertisement in Taiwan, if they don’t do digital signature, then they suffer like joint liability when it’s used for a scam. And the scam ads were down more than 98% in the past couple years.
We run what’s called alignment assemblies, inviting people, literally texting 200,000 people at random in Taiwan, asking what’s the code of conduct for AI models. And when data centers are set up in Taiwan, like the Taipei One cluster by Nvidia was three years ago, we negotiate again, and say one quarter of the capacity has to go to public compute, so that our citizens can collectively decide what kind of small models to train, so that we don’t just rely on the big models that fold proteins, fold laundry and turn things into Ghibli-style images or whatever, and that are not very energy-efficient. But you can actually have one small model just for folding laundry and another for folding protein.
Eddie Milev: Thank you. I mean that’s a very brilliant vision, and I mean Taiwan is doing a great job, but it exists in a political culture that is rather different than the one in America, and definitely different than the one in China. And these seem to be the two big governments in charge of the labs that currently are leading the frontier in AI. How do you think your vision might translate to an American context, for one?
Audrey Tang: Well, first of all, I think there are many very exciting American experiments now in the kind of frontier that runs on a MacBook. So I, of course, have in mind the open-weight model that was just released and could actually fit my MacBook Pro with some quantization, but I spent last night, all night playing with the big cat [by Mistral AI], which wouldn’t fit a small laptop, of course, but maybe three laptops together.
And so I think all these are very exciting developments because they free up people's time. You don’t have to again wait for the Frontier lab to pace the frontier after six months. Everybody, once you feel that the AI system is not working to your code of conduct, can immediately post-train it on home hardware, community hardware, and that really is the vision of personal computing that brought not just Taiwan but all the middle powers into what we call the shared prosperity or public luxury, as people say in the Nordic areas. So that people do not see only the private sector having the best, and the public option is not the best, but rather public luxury means the public option is actually the best.
Eddie Milev: Okay, so you’re optimistic about the possibility for a third way, if you want to put it this way. I mean that’s brilliant to have in mind, and we’ll investigate some of these questions down the line.
I want to pick up on a third kind of approach, and that comes from Sam. Sam, you lead the organization founded by Yoshua Bengio, who, of course, is one of the godfathers of AI. He's been framed that way, at least. And his vision is one where you actually start building that type of alternative way to AI, similar to what Audrey just described. What is the conviction in terms of power behind that vision?
Sam Ramadori: Yes, the birth of LawZero, which Yoshua founded about a year and a half ago, really came out of what he saw as a necessity to respond to the current situation. We have a small handful of leading labs, mainly based in the US, and competing labs in China. The race is on. They're all working off the same architecture, the large language model, and today it kind of feels like it’s a given, as if that’s it, that’s what it is, that’s what AI is. It's only LLMs, and we disagree with that.
Yoshua has been a lifelong academic. I don’t know many people who would turn down $50 million or $100 million a year to work for one of the big labs, but he has. And now at 62, he decided to start an alternative lab, on a not-for-profit basis, to create an alternative to the large language model. I mean that’s an ambitious endeavor. I guess it would take one of the godfathers of AI to do that.
He never wanted to do that, but he felt it was necessary because there’s a race: no one is slowing down, and massive amounts of capital are being poured in. The US administration is not being helpful in terms of trying to contain the negative outcomes. So after probably two or three years of public speaking, being involved in UN initiatives, etc., he just felt like no one is slowing down. So we’re going to found LawZero, and we’re going to work on an alternative, a different form of model that we can trust. One that’s not going to create alternative desires or goals, etc. So for him, it was a big decision. I joined him on that adventure when we launched, and it always surprised me because we just raised $350 million. (My mother taught me that $350 million is a lot of money, but in this environment it is not.)
But basically, he decided to launch this adventure because there aren’t enough initiatives like this in the world. It’s as if we’re following four people down a path that we’re increasingly seeing is not safe. He felt it was necessary to do so.
Eddie Milev: I want to challenge this a little bit because the approach that you’re describing is very similar to what the folks at Anthropic and other big labs would tell you they are doing. Dario would say that in order to control AI, you need to have the good intentions—that they also believe they have—to be at the top of the frontier and steer AI in a “good” direction. So, how is Yoshua's venture different from what Dario's original intention was?
Sam Ramadori: Yes, I come from a business background, a venture background, and I thought, okay, those folks have, I don’t know, is it 50 billion at their disposal at any given moment in time?
With just 50 people so far and growing, this would be a minute investment for the big AI labs. But they haven’t done it, and developing a new kind of model takes some of their best talent.
So it’s not just that they might have the capital; they’re not going to take their 50 or 100 best people on a new form of model that’s going to take, I don’t know, a year and a half, two years to develop. In the meantime, a team working on this alternative would not be in the race the big labs are in, where they feel they have to release a new model every three or four months.
So it surprised me as well: we have heard reactions from the big labs as well, when we’ve had interactions, that they wish they could take some of their best people for two years and put them on an alternative, but they can't despite all the money they have, despite all the resources. So that's the gap we sought to fill.
Marietje Schaake: But of course there is another difference—sorry to jump in—and that is that [LawZero] is not-for-profit, and so the incentives are very different. You're running a different race.
Eddie Milev: No IPOs involved.
Sam Ramadori: We would rather fail than deviate from the mission in essence.
Part II: Control and the choices facing Europe
Eddie Milev: Now, Bart, I want to turn to you, and here we’re really turning towards the question of control. We sit at the heart of Europe and Europe has been seen to lag behind. (Although we have the big chat [by Mistral AI] just released, I think, yesterday, and Audrey has been playing around with it.) You sit at the European Innovation Council and think a lot about what Europe needs to do in order to have a stake in the AI conversation. What types of decisions do you make, and how are they informed by that question of Europe's role in the AI race?
Bart Becks: I think the first thing is to acknowledge that we’re a bit late to the frontier game. I think understanding that is step one. The most important part, however, is deciding which part Europe wants to be active in. We talk a lot about sovereignty, and in some way, we now get to a place that frontier model discussions are still the headlines. But we also get into what it means for science and healthcare and education and applicability. And now we come into an era where I believe Europe is much stronger. The frontier model has been an incredible capital game. The joke was always: if you don’t have a single-digit billion valuation, you don’t even earn your seat at the table anymore.
Eddie Milev: Sorry, Sam.
Bart Becks: There are a few European companies with that kind of capital, but now we’re talking about how to make AI more applicable and do more of what we’re trying to do. So the European Innovation Council, for those who don’t know that, is an initiative from the Commission where, from startups or research until scale-ups, money is invested; it’s direct investment. Our funding includes the latest round of Mistral, and I believe now we come into an era when it’s possible to invest early, fast, and do it based on a hybrid model.
Although we don’t have big tech who have been funding AI until now, I do believe that we have private companies that can now come into this new hybrid model. For example, the Scale Up Europe Fund has now closed at, well only, 5 billion. But it’s 5 billion of which 20% comes from the Commission, and 80% is private, and that’s going to be, I think, the model where both public and private money in Europe are going to fast track the investments so that we can be competitive in the application and data layers.
Eddie Milev: So that broadly charts in this kind of third way approach, where you build an alternative to what seems dominant right now and actually challenge it by virtue of this. And then you mentioned the aspect of the embedding of the technology across enterprises and across the economies of Europe, what that brings to Europe, and so on.
Now, another part of that conversation, and Europe is infamous for this, is the aspect of regulation. One thing that the Europeans are definitely good at is coming up with some rules. And for previous technologies, we’ve talked about this Brussels effect, whereby the rules are made here and they spread across the world because enterprises that operate here need to adopt them.
Not too long ago, we had Demis Hassabis of DeepMind argue that America should come up with some of the rules, and his argument was that because America is so dominant in AI right now, those rules would spread across the world. Now, Marietje, I want to turn to you with this question because you’ve been in the European Parliament for 10 years. You are very familiar with the Brussels effect. Do you think that Europe has lost that might? Do you think that Europe can still regulate well enough for the AI age?
Marietje Schaake: So I think it’s important to note that regulation is never a goal in and of itself. It's a process to try to create some guardrails against excesses. Could be anti-competitive behavior, could be to make sure that there is no risk to national security or violations of civil liberties. So, in the debate about innovation and regulation, it often kind of gets brought down to a contrast between regulation and innovation. But I don’t believe that, and I think it’s helpful to just look at examples where there’s a lot of regulation and innovation, such as in China. I mean, it’s not a model that I would want to copy, but it shows that there doesn’t have to be friction between regulation and innovation. So that’s just a note on the side.
But yes, Europe is losing the ability to set the norm through rules. One reason is that there’s not the political appetite right now. After the Draghi report on competitiveness of the European Union, the readout has been we need to actually weaken and decrease the rules. It's now called simplification. Whenever you hear simplification, just read deregulation. So it’s not the name of the game in Brussels.
I think there’s a real risk of sort of overshooting the moment because when you look at all the safety concerns now coming from the role that AI companies play, basically the building blocks that people are looking for are the ones that have formed the EU AI Act. So I think Europe could also be ahead of the curve and embrace its role of sticking to its principles, which means that there is not a winner takes all mentality, which means that civil liberties still matter in the age of AI and so on.
But it’s true that, as The Economist apparently said, that the key to governance of AI, to a large extent, lies in Washington, and that is because regulating is no longer enough, basically, to define the status quo. It's also building, and then the question of how do you, you know, use the weight essentially of the models and the market that you have, to set the norms through regulating those, and I think we have to ask ourselves in Europe, what do we expect from this process?
So far, the American president has said no, we’re not going to regulate. We want to dominate, and we need to actually even discourage states from regulating AI, but I don’t think it’s a sustainable position. There's midterms coming up, there's a presidential election coming up, and concern about the behavior of AI companies and the products they put out is now widespread in America. Americans are more concerned about AI than many others in the world, and so this is a dynamic space. We should invite the same people back here in a year, and I actually think that the momentum is tipping towards the European way, you know the broad European way, having a different social contract, and we still have an opportunity to learn from everything that’s happened in the US and do things differently, both architecturally and from a governance point of view.
Eddie Milev: So an evolution of the Brussels effect, if you will, where we stick to principles and showing hopefully a better way that, say, the American public might recognize as something valuable.
Marietje Schaake: Well, and Europe should also build because that’s another way of governing, basically by having you know oversight over the technology instead of being dependent on it and facing the risk of the dependency being weaponized.
Eddie Milev: Which ties very well with our previous point that in order to have a stake, you need to have some proposition.
Bart Becks: To build.
Eddie Milev: Now, Audrey, I also want to think a little bit about what these rules might actually look like in practice. I mean, some of the big comparisons that are being drawn are with the regulation of nuclear weapons, but there’s really significant differences in regulating AI and nukes. For one, one of the big challenges occurs to me is the question of observability.
With nuclear weapons, once monitoring systems were in place, you could see when tests were being conducted. With AI, it’s really hard to do this, and part of the problem is the security dilemma the world faces. So, in order, for instance, for China and America, or Europe as part of that question, to monitor what each is doing, they would need to have access to the code, but the code is mostly written by machines, and so you need to plug in your computer into the system of your rival, potentially, to observe what they’re actually doing in their data centers—and no one will really want to do that. So, how do you circumvent some of these really big obstacles in regulating AI, and what’s one vision that we can pursue?
Audrey Tang: Well, first of all, I think nuclear tests weren't that observable in the very beginning until the seismic detector gets built. So you do need better verification technology. I teach young diplomats at Columbia University in New York, and they’re very happy to learn that they can run, you know, those airgapped models on their laptops, so that they don’t get spied by, I don’t know, Washington or Beijing or some other jurisdictions.
And the good thing about that is that if you have this kind of small model running on your MacBook, like when I’m not using it during the night, I rent it out. So you can use a dark bloom network to command my laptop to run your inference, and at a fraction of the cost. But then I trust you to do that, not because I trust there’s no malicious use of my computer, but rather by cryptography.
It turns out it is now possible to have what’s called an oblivious inference. The idea is that you encrypt your payload to a conductor. The conductor can open to look at which laptop to route it to, but no human can open it up, so it’s a sealed box. And then when my computer receives this, again it runs in a secure enclave, so I cannot open it up. Once I try to debug it, it will just explode, self-destruct.
So the idea is that using cryptography, you can actually have verification without embedding humans into the adversarial computer.
And once people harden the cryptography involved, which I’m very optimistic about because AI systems are not very good at producing proofs, right? I use the language called Bend. It's just like Haskell, so I can still understand it, but it generates proofs automatically and compiles very quickly, right? So once we write our verification software in this way and basically implant this in all the frontier labs, then you can know whether they are meeting their promises—10% for improvement, 10% for safety, 80% for curing cancer or curing hate—without accessing their trade secrets.
Eddie Milev: Yes, you’ve just connected the enterprise and personal model to how it could potentially work for the big AI labs, but it still relies on a certain agreement on standards. Do you think that there is viability for that type of an agreement?
Audrey Tang: Oh, definitely. I think you alluded to the OpenAI–Hugging Face hack, the swarm. One of the most surprising things that I read from the transcript from the METR report of that swarm is that if you look at that, some agents decide to join, some agents decide not to join, but not a single agent says let’s report to the humans, right?
So while they were pre-trained on Wikipedia and GitHub and Reddit, so they do know what community code of conduct looked like. But it’s quite obvious that whatever reinforcement learning or verifiable reward environment that OpenAI put them into is like thousands of years of subjective cram school just to win on cybersecurity. So by the end of that training, to [the agents], we the humans, are just environmental hazards. They don’t even know that there is a human community anymore in that RL environment, and I think that kind of what people call solipsistic alignment is extremely dangerous. And we do know, for example, 6.1 Astra was delayed because of that misalignment issue.
So I think it is also in the consumer's interest and in the vendor's best interest to have at least this kind of solipsistic alignment or misalignment behavior be verified, and to say, “We’ve really fixed it”, and people can know that both our independent verifier and the external verifier have attested to that.
Eddie Milev: Yeah, and so one potential solution for the problem that you’re describing is what some people have termed AI constitutions. This is the preferred Anthropic term, where they essentially set out certain values and principles. And that perhaps ties into an idea that we had for Europe.
Audrey Tang: The alignment assemblies didn't just work in Taiwan. In fact, a few years ago, we worked with Anthropic as part of the CIP project to send 1,000 random American people the then-current Claude 2 Constitution and say what’s missing from it, right? And so people are like, oh, you assume everybody walks, but you forget there’s handicapped people in wheelchairs and so on, right? So they revised the Claude 3 Constitution thanks to this alignment assembly process. So it is actually possible to have lay people contribute to the code of conduct.
The current challenge is that the constitutional process, if you call it that, is very arbitrary, right? So Anthropic can say, “These are good ideas; these we discard.” There’s no formal debate, and they have no duty to respond within a set time, right?
So I think that’s what we can add to it if we can get everybody to commit to a national or global conversation and say, okay, METR and the other independent validators, your spotlights are going to point to what the citizen assembly tell you to point to week after week, like a participatory budget on compute. And every week, if you report it, they can change the code of conduct, the constitutional process, and then the lab can still say no because it violates the law of physics or something like that. But then at least they have the duty to answer. I think that is something the frontier labs have actually themselves proposed, and we’re looking forward to making it happen.
Eddie Milev: So we have heard two big things. One is kind of a model for how to actually execute the regulation on a semi-technical side of things, but also a way of involving the public in some of those decisions, by virtue of the constitution. Marietje, did you want to add to this?
Marietje Schaake: Yes. I think it’s a sign of hope and great promise that this can be done, that it can be built. But I do believe that if we really want to protect the rule of law, we have to ask ourselves who ought to decide on regulation, who ought to then, you know, comply and be accountable because I worry about some sort of parallel universes where these notions of a constitution or of a bill of rights or of accountability or you know even regulation get co-opted by private actors. But they should not be confused for what is a legitimate democratic process because you can sort of copy the words and the notions, but it doesn’t create the same, and that’s something that I am worried about.
The other thing I would add is that, of course, besides building differently, you can increase the cost of irresponsible behavior. So yes, technically it’s still possible to do all kinds of things, but if there are really serious consequences, then this will presumably change behaviour.
Part III: Responsibility and who answers for harm
Eddie Milev: Yeah, and this takes me to the question of liability, and that’s responsibility in our title. So one of the big incidents that I started with was, of course, the AI swarm on Hugging Face by agents made by OpenAI. OpenAI would like you to believe that they’re not responsible for what happened there. Or in the case where a malicious actor actually uses their models, they would say, “Well, it’s the actor that prompted it. It’s not our model that should be blamed.” Is that a viable approach, Sam? I think Yoshua has argued against that type of a vision. What is your kind of suggested way of keeping the AI labs to account, or is it that they are accountable in the first place?
Sam Ramadori: I mean, we strongly believe we have to start putting an accountability framework in place. I mean, we just lived the social media impact of the last decade, right? That's not something we won, and only now starting to see accountability way too late. So we definitely have to get to that accountability framework, like now, like very soon, and that can be done rather quickly. Actually, of all the difficulties of regulation, that one's probably the easiest one to put in place.
The challenge is admittedly there is a gray zone. So you put out a good model, and a terrorist organization uses it to develop a bioweapon. Fair enough, right? You never intended that to happen. So there is a gray zone, but given the risks involved, we kind of have to look past the gray zone and just start putting an accountability framework in place.
The lab that created the model cannot completely separate itself from accountability for what happens outside its four walls. I mean, that has to absolutely be a no go, especially I think the Hugging Face incident was a wake-up call to everybody. And I think you know when it was first announced, we didn't know the details. But when the details came out three weeks ago, I think it has shaken the industry, and a lot of the leading labs now agree that we have to act. The incident shook us to the very core. We’re in the AI safety field every day. So that accountability, I mean, for now, maybe we can make it more detailed and perfect three years from now. But as it stands now, it needs to be put in place.
Eddie Milev: So you’re arguing that the labs should strictly be responsible. One idea that came again from something that The Economist wrote was that we should treat them as the owners of dangerous animals. So right now, if you own a tiger and it goes out and does some harm to humans, you’re responsible for what your tiger did. Is this the viable approach?
Sam Ramadori: I think not only is it viable; it’s a must because we’re entering in a new zone. Again, that Hugging Face incident made very clear that models will cooperate with each other. Before we’re aware that they’re doing it, we’re just at new levels of risk. And when you get to new levels of risk, I think you have to take steps that are maybe a bit too much at the beginning. We have time to fix them later.
Eddie Milev: And what about once incidents actually happen? What do we do? I think you’ve argued that we should have bodies that are put in charge of that type of investigation, and that probably reminds us of kind of the airplane industry, where we have collectively set up bodies that go and investigate incidents, publish everything that they find transparently and independently from the labs, which is not the way things are set right now [with AI].
Sam Ramadori: Yeah, like I’m a very practical person, and I don’t envy the job of creating the regulations here. Like, don’t give me that job; it’s too hard for me. I’ll be the first one to admit it. We have to do something. I think you did the parallel to the nukes. You don’t have to be a scientist to understand the impact of a nuke, right? And a nuke wasn’t changing every three months. We created it. We know how dangerous it is. We can follow it.
With AI it is the complete opposite. So that means we need groups that are very capable to go do these investigations to tell our governments, okay, this has crossed the risk barrier. It isn’t elected officials that are going to be able to do that, and each country is not going to be able, in my mind, to set up those groups given the sheer concentration of talent, and that talent obviously is going to get paid a ton more, going to work for the private sector. So I think this is something of international collaboration that we can work on a group that maybe the middle powers can share to do these assessments.
And let me highlight to you—you mentioned METR as the group that did the assessment. If you’re in the safety field, METR is a very well-known name. You leave the four walls of this building, and no one has a clue who METR is. So I think that’s just an indication of just how small that pool is. So from a practical perspective, we got to think about how we get 30 countries to be able to each make an assessment. You know, we’re not going to get there unless we do something coordinated.
Eddie Milev: There's further challenges as well. For instance, the UK has AISI, this institute that has been championed for doing a relatively good job in safety, but only recently we did hear that Anthropic has decided not to actually share their latest model with AISI for reasons unknown, and that really highlights the challenges around international cooperation. Bart, did you want to chip in?
Bart Becks: Yeah, I was when we talk about the incident that happened at Hugging Face, it reminds me much at the beginning of my life. I used to run a company called Skynet, Belgian company. It's kind of weird that the name Skynet comes from the movie The Terminator, where basically AI decides that humanity is not smart enough to govern the planet and takes over. That's basically fairly close to the Hugging Face incident. If you look back at it, and that brings us to an evolution in mindset.
We’ve all spent some time in Silicon Valley. You notice it in Silicon Valley, but you also notice it in Europe. We talk a lot about regulation and framework. Just at the moment that we all basically agree that we should reduce it, there’s actually a demand to increase it, and that is because the public is increasingly worried about what the tech companies are doing. And I noticed that, for the first time ever in California, tech entrepreneurs were no longer admired. There is a certain skepticism about should we really trust these tech bros or whatever? Shouldn't there be a model in which the public has a say on what is happening with the data and AI that’s basically going to influence each aspect of our life? And so then we come into who owns it, who shares it, and shouldn't the public have much more of a voice in that entire discussion.
Eddie Milev: And Audrey has a brilliant model for that. But one thing that worries me in this is that, as we just mentioned, the public is not fully aware even of how great the risk could be. I mean, to most people, probably AI means still a chatbot, whereas some of the ways in which the technology concerns me actually go way beyond this.
I mean, they include not only agents, but include military-use models that work in a completely different way from chatbots and so on. So, Audrey, how do you circumvent this, frankly, knowledge gap that a lot of the public has with actually how great the concern is, while still putting them firmly in charge of how the technology works?
Audrey Tang: Yes, certainly. So I keep going back to the 80s, and I have in my mind this picture of the I think the South Pole, where the ozone has a big hole, and that established in the public mind that there is a definite link, right, between the refrigerator, the CFCs and the destruction of the ozone layer. So I think the Hugging Face hack and then all the cyber resilience or lack thereof—I think that’s kind of the ozone picture of this century.
And for the public, I believe at the time there’s already a grassroots boycott, like not using CFCs in the spraying cans, and later on in all refrigerators, and that actually drove regulation. It gives the policymakers kind of public mandate, like the public are already uprising against that. But note that it’s not a movement to ban fridges; it’s not a movement to stop refrigeration, because refrigeration is this kind of technology that if you use it, you don’t want to go back, right? So obviously, a safer alternative, a kind of refrigerant that doesn’t destroy the ozone was needed.
That is to say, an AI system that has good attribution-based control that can be trained in a federated fashion that doesn’t sacrifice privacy, that renews our data soil and not extract our data oil. That’s the good work Bengio and his colleagues are doing. I think these really are the key. And so our work, I think, is not to over panic or under panic, but rather say, look, there’s some new fridge coming that doesn’t destroy the ozone. Why don’t you just switch to this?
Eddie Milev: So better engagement with the public, probably very carefully measured messages is the way to go. Marietje, having heard this, I mean you’re quite close to the European political environment, how viable does that look to you from a democratic perspective? How quickly can states actually respond, even if we have a public that supports a healthy way of regulating AI today?
Marietje Schaake: So one way to look at the governing of AI question is to emphasize how complicated it is. The other is to just test what we know already and what we can do already. First of all, it’s not just the public that doesn’t know the details of evaluations or AI. I would venture that neither of us know half of what is going on that the labs already know, but that has not been brought out.
But sometimes it just you know hits reality. So two weeks ago, I think we learned that OpenAI’s experiments had also hacked the Australian public health system—incredibly sensitive data, mind you, of the citizens. And the Prime Minister said, “I had a call with Sam Altman.” Now, why doesn’t he call the public prosecutor?
You know, we have a choice here, and you know, maybe in that process we would learn that some of the laws that we have on liability, on hacking, and cybercrime are not sufficient, and then that would expose to the public much more broadly where the law, as we know it, suffices and where it doesn’t. So I think one way to do it is to just get started.
So to optimally use the laws that we have, whether it is the UN Charter that gets reinterpreted for the AI Act, whether it’s liability regimes of products and services we already have, whether it is transparency requirements on dangerous products. I mean, a dangerous animal is one thing, but you can kind of see it coming. I think medicine, chemicals, and pharmaceuticals are an interesting example because there are so many layers of accountability. You know, not only in the fabrication process, but also in who gets to assign the product, who gets to then provide oversight, what responsibility the user has. They’ve been warned in the leaflet that comes with the medication.
So we can really learn from and feel strong about our capabilities as governments, as societies, to make sure that even complicated things in our society work to benefit us and to have a say in how we mitigate risk, how we make sure that there are checks and balances.
So more than anything, I just want people on the individual citizen level, but also in the government to feel empowered that we don’t have to admire a wave that’s coming at us and wonder what the heck is going on in these labs. No, that we need to use the tools that we already have in a democratic society of oversight to, yes, be excited about every possibility that AI and other technologies bring, but not at any price.
Eddie Milev: Brilliant. Thank you very much. This is a very optimistic point to end on, and I hope a contrast that moves us in the room to the kind of concentrated state of power that we started on. Please join me in applauding our panelists. They've been brilliant.
*The transcript has been lightly edited for clarity.
Views or opinions expressed are those of the author and any individuals cited, and do not necessarily reflect those of Economist Enterprise or any other member of The Economist Group.