In the Loop AI

ChatGPT just discovered and arrested a spy in OpenAI's own HQ.

And that's the least craziest thing I'm going to say on this entire episode,

because this whole story is crazy. It starts off with an unknown individual

planting an audio listening spyware device in the back of a television wrapped in toilet paper.

Why it's wrapped in toilet paper, I have no idea. But the important part is

the conference room. Why do we care?

Well, a bunch of really important executives and business partners at OpenAI

meet to discuss proprietary information, secret product plans.

So you can imagine if that information got into the wrong hands of a person,

a company, or maybe even another country, that could probably become a bit of a problem.

So OpenAI handed over the case to the FBI and their own internal model,

who ended up ratting on the individual who had planted that device.

Kash Patel, the head of the FBI, said this is the Bureau's utmost priority at

the moment, that AI has finally become an importance or asset of national security and risk.

Imagine if James Bond, 007, had access to Fable 5.1 back in the day.

He'd be a bit of a beast. But also imagine if his enemies also had access to this model.

That's what we're going to uncover in today's episode of In The Loop,

brought to you by Qualcomm.

So the story starts off two weeks ago, September 30th, when an OpenAI employee

discovered a device plugged into the back of a television in an executive conference room.

And it was wrapped in toilet paper for God knows what reason.

But upon recovering this device, it was handed over to the security team at OpenAI.

They started looking at it and they were thinking, hmm, this looks like a USB

stick, but it has this weird circuitry that's kind of attached to it.

So it seems like a bit of a hack job. What on earth is this device?

And so they started to examine it and they asked their own internal AI model

to uncover what this device might be used to.

And they discovered three particular things. One, it had very heavy cryptography

designed around its entire thing.

So accessing the information that this device had was near impossible.

Number two, the device was designed to be continuously recording audio,

that it would also be triggered to start recording audio when someone would start speaking.

So it seems very intentional that this device was to try and spy on conversations

that were happening in that particular room.

And then number three, any kind of data that got extracted from this device

that could be transferred to, you know, your computer to view it or review it

or listen to it was under heavy encryption as well, which meant that the data

transfer layer had been very meticulously designed.

And so OpenAI kind of like looked at this and thought, this seems like an attempt to spy on us.

So we should probably hand this case over to the FBI, give them access to this

internal model or work with them at least to try and figure out what the hell is going on.

So opening i continued their internal investigations whilst they gave it to the fbi

and very soon using their own internal model of i don't know gpt 6.5 gpt 7 call

it whatever that latest model is the one that solved all the math stuff definitely

go check that episode i made that very same model named someone.

Traced back the entire circuitry design of this device and who was most likely

to have planted the device and uncovered a name.

So Open Air had an idea of who it might have been, but let the FBI continue

on their investigation. But while they were doing this, they also decided to

plant a camera in almost the exact same position where the USB stick was.

So in the event that someone who planted it came back to look at it,

they would be able to capture the suspect red-handed.

And that's exactly what happened a few days later when the suspect returned had a look behind the tv.

And the camera captured exactly who the culprit was and on the same day that

this culprit was captured on this camera security team discovered a bag left

behind by the culprit by accident

in the printing room they went through it and they discovered a second device

that looked very familiar to the one that they found behind the tv

so they just took the bag put it in lost and found but they took the device.

They were trying to capture this individual wet-handed. They also wanted to

figure out what on earth was going on, what this individual might be using this

information for. So they, again, were keeping hush and quiet.

And they handed over the device and the information that they had to the FBI.

Now, this individual discovered that his bag had gone missing and went to Lostom

Property to try and find it. And he found the bag. So he felt pretty good about

it, except that he noticed that the second device was missing.

And he also noticed that the device behind the TV when he went to go check it

was also missing. So he reported that there had been a theft of his own property.

But by that time, the FBI and the OpenAI team was already on him and issued

an arrest or warrant for his arrest the exact next day.

Upon arresting this guy, they discovered that the suspect, his wife,

and another family member of the suspect had a one-way ticket to Saudi.

So he was going to fly out with his family and likely never return because.

Presumably he thought he'd been busted so the guy got arrested and he has been

internally investigated and being put through court or whatever all that stuff

is by the fbi and cash patel

who's the leader of the fbi and his team has been handling it since then but

open ai has come up with an official statement kind of giving a high level idea

of what's been going on but never named any names

who has been naming names is pretty much every other media publication out there

which gets to the next section, who the hell did this and why did they do this?

The man's name is Abdulrahman Al-Fozan.

He's a 35-year-old Saudi national who's lived in the US since 2020.

But he's also worked at OpenAI for over a year on the product team.

So he's like a product slash engineer. He's worked on multiple product releases

at OpenAI, likely a core employee, been there for a while as the valuations

have increased, so likely his equity is worth quite a bit.

This is a person that people actually know at the company.

If you look at his LinkedIn history, he's worked on things like AI agents,

which is important for the next bit of this story.

What was he doing? How did he create this device? And why was he trying to use

this device in that secret room? Let me take you back six weeks prior to this

device being discovered.

This individual, Abdul Rahman, queried an internal personal agent at OpenAI.

Guess what he asked that agent?

Hey, where do high-powered execs at OpenAI?

Have meetings? Also, business partners. Where do these execs have meetings with

those business partners?

Can you tell me the exact rooms? This guy was literally asking an internal version

of ChatGPT, which the security team and any OpenAI employee or anyone with a

brain cell could get access to and see who was querying it.

That's the device that he, or rather, that's the agent that he used to try and

target and locate an executive room. So he finds this room. He finds the perfect

room. It seats six people and more.

They only have discussions about proprietary research, designs,

product releases, things that you don't want getting out to the public.

But there's a problem. He can't book the room. He can't get access to the room.

So that's only bookable by execs and their secretaries. So he tries to then

devise a plan as to how he can be a listen-in on the conversations there.

And he comes up with a bright idea.

I need to create an audio device that can spy and listen in to the conversations

that are happening in there, but I don't know how to build that device.

You know what might be useful? Let me ask AI.

So he goes to his wife, who has a chatbot account, and he goes on to ask it

a series of very specific prompts, which is,

I want to create an audio device that continuously listens to the surrounding

conversation around it.

But also when it stores this data, I want it to be encrypted such that only

I have access to this data.

And in the event that someone gets a handle on this device and is able to extract

the data, I still want it to encrypt the data so that no one can get access to this thing except me.

But I also need to design this device so that it plugs into a television in a room.

But I don't want the television to detect the device because that would give it away.

So could you create the device such that it only appears to the TV as a power

device, which means that it's charging the device, but it doesn't know what

the device is being used for.

So a series of back and forth prompts over the next like four weeks from that

initial six week kind of like starter, ask my agent, which rooms should I plant this device in?

He starts to plan this device and presumably he ends up building it.

This is the thing that is wrapped around toilet paper.

Investigators looked into what it took to make this device.

It cost all but $24. So a $24 device cost this guy presumably a large chunk

of his life. I looked into the prison sentence. It's like up to five years for

the particular case. But before I get into that.

The idea here is a $24 device basically caused this guy and the entire company

and this entire kind of spy mission, a whole world of pain.

Okay, so then we get to September 28th, two days before the device is discovered.

Abdul Rahman enters the OpenAI Office HQ building. He walks up to his desk.

He chills. He's working.

He's going about his day, seemingly. Keeps glancing over to the executive conference

room, where he keeps wandering around, scouting the area.

He enters the building at 2 18 p.m and he's there till about 6 30 p.m so whatever

a regular working day in his eyes but he's caught on cctv because of course

investigators are now onto him and they can see him in hindsight

scouting the entire area where he then pops into the room

when presumably someone exits and he's able to hold the door open this is something

straight out of a spy movie this is ridiculous also why the toilet paper i don't

understand okay so then fast forward two days Zeta September 30th and it's the story I just told you.

Someone discovers the device at the back of the television, presumably because

the guy was caught on CCTV being shady around the office or the conference room in the first place.

Everything comes down around him. He's caught, he's arrested,

and they're currently working on what his sentence might be for his particular perjury.

Now, before I get into what the

ramifications of this is, I think it's important to spend a short moment

how OpenAI's internal model discovered the guy because it got access to the

device and it basically reverse engineered the entire thing.

And it was able to uncover two things.

Number one, how it was particularly designed and in accordance with the CCTV

footage, which OpenAI also fed it.

They could then figure out who the person likely was.

And then when they had a suspect, which was this Abdul Rahman guy,

they started to look at his activity secretly without warning him.

And they discovered his conversation with the internal personal agent that he

had six weeks prior, basically asking.

Hey, where do all the execs have super important information and high priority

product release information so that I can plant a listening device?

I'm not a spy, but Spy 101, one.

Don't talk to the tools that your companies own if you're trying to pull off this kind of a heist.

It's crazy. But then they took it an extra step further.

And this is the part where I'm still trying to figure out how they did it.

But they were able to discover that he was having a conversation on another personal chatbot.

And he then used this to prompt engineer what that $24 device would look like.

He built it himself or he got someone to build it for him.

And then it led to him planting it in the back of the room. So the guy used

AI to plan and build and execute the entire thing.

And then AI that he used ended up becoming the exact point of reason for his

downfall, discovery and arrest.

It's just this insane loop where you could have used the AI to do the bad thing,

but you can use AI to do the good thing as well.

So it's kind of like, it's not a wholesome story, but it does give me more confidence

when, you know, we talk about fearmongering around AI cyber attacks and AI is

going to wreak a lot of havoc, AI can be used for a lot of good as well.

So it's just this insane story. And I have a lot of takes on this.

So let's get into them on today's episode of In The Loop brought to you by Qualcomm.

But the analogy that I would use here is he effectively walked into the building

and what would be the analogy here?

He asked the office receptionist where the most secretive conversations were

happening amongst execs, found out the information from this receptionist,

and then went to a colleague and asked them,

hey, how do I build a listing device that I could plant in this room?

He basically gave the cards over to people or employees or agents of the building.

And he's honestly stupid to have not thought that people wouldn't investigate

the tools that he would use, that he wouldn't leave a digital trace.

And I guess that's like one of the larger points about this whole thing.

Putting the politics and the crazy espionage theories aside,

everyone's digital footprint when they use AI has become massively enlarged.

Remember, when you're talking to your chatbot, it may seem like it's quite a

personal thing, but all your data is being tracked by OpenAI Anthropic or whoever

the model lab is, unless you're using open models hosted at home on your own.

That's not necessarily a bad thing. It's a good thing in this case that they

were able to trace it back.

If this was an open model, probably not. The second thing is around the sentence.

So I think right now, rumors or reports are saying at least that they have him

on attempt to overhear or record something.

I forget the term that they were I think it's like oral conversations,

something very generic like that.

Nothing that would indicate espionage, spying, or a breach of national security,

which would presumably result in a much larger sentence.

But right now, the max sentence that he can have for recording of oral conversations

is five years, which seems honestly surprising to me.

But he should get the charge for maybe like spy stuff. But again,

details are unraveling. FBI is on it.

That Kash Patel, head of the FBI, is fully committing resources to try and figure

out what the hell this is.

Which brings me to my third take. AI has officially become.

An issue of national security. Now, I've said this many times on previous episodes

on this channel, and it's crazy to think that this is happening now,

and there's real-life examples of this happening.

I think of a conversation that Jacob Coxon, who is the crazy safety and alignment

researcher, remember the one that we spoke about on an episode previously where

he left OpenAI, joined Anthropic, then left after three weeks because he did

not have any faith that either the company were taking model alignment seriously.

He went on a podcast last week and he said, there's this clip of him that went viral.

He said that there is a 100% or sorry, 90% chance that there are spies at the frontier labs.

What are the odds of him having that conversation, going live,

and then literally two weeks later, opening out uncovers this spy,

Abdul Rahman, who had been planted there for presumably a year.

It's becoming all too real, if that makes sense.

And it's one thing kind of talking about it to you guys, talking about it on

a camera, hypothesizing what might happen. It's another thing,

it actually happening. And then it's a third thing, us being aware of it,

it being all revealed to us.

But can you imagine how many other attempts are currently underway?

If you're Anthropic and you're seeing this news, if you're Google,

if you're SpaceX SI or whatever Frontier Lab.

You're going to be now on high alert that someone is trying to access or listen to your conversations.

Now, with all intents and purposes, this could have been a really bad situation for OpenAI.

This guy could have attempted to do

something even worse, not listen to conversations, but try and exfiltrate

model weights or model blueprints that they could then use to build a competitor

model or use it for another model for more nefarious purposes at the state level

or at the company or entity or even the individual level.

Like there's myriad different ways that you could use this thing.

Thankfully, OpenAI's investigation has revealed that there's likely not a chance

that he's been able to pull this off.

And personally, if the guy's wrapping his device in toilet paper.

It doesn't sound as sophisticated.

But you know, you can imagine that they are high level organizations probably

trying to attempt this right now, maybe even still at OpenAI or at Anthropic

or any of these other labs.

The fourth take is AI is officially a national security threat.

And how we protect those models, how we keep them secure, needs to become a national priority.

That's what Cash Patella said. But right now, it's under lock and key for small,

well, small private companies such as OpenAI and Anthropic who employ their

own security team and who kind of take matters into their own hands.

And they're not necessarily a public company. They don't have public investor

interests. They're only beholden to themselves and their current shareholders.

So I can only imagine that the government wants to get involved,

they are probably placed on high alert now.

And Kash Patel's probably thinking, I wonder how I can plant an FBI agent or

several FBI agents just in the day-to-day nine to five at OpenAI's office.

I'm almost certain that they're having this conversation.

And then it kind of got me thinking about Dario's suggestion of pacing the frontier,

about a month ago, where he suggests that we want external parties to get involved

and kind of evaluate and monitor the alignment side of AI.

I'm noticing a trend, essentially, I guess is what I'm trying to say,

where external parties need to start getting involved or will likely start getting

involved with these private companies.

Traditionally, it's been at the whim or direction of Sam Altman and Dario Murray,

but now they may not have a choice.

And I wouldn't be surprised if in the next few weeks, Trump,

the administration or whoever announces that there'll be some form of light

nationalization of these companies.

Because as these models get more powerful. And Anthropic and OpenAI are both

rumored to be housing much, much more powerful models than we currently have

access to internally, which they can't release because.

The ramifications of that could be quite dangerous. As a government,

as a nation state, if that got into the wrong hands of your adversaries,

such as China, whoever that might be, you could end up having a massive problem

just from a pure security standpoint. So the labs are national security sites now.

And then the other thing I was thinking about is Apple.

Apple is currently suing the hell out of OpenAI, accusing them of stealing or

reaching out to hundreds of Apple employees trying to exfiltrate their own blueprints and,

device architecture. That's it for the AI hardware device.

Seems like OpenAI has got their own problems. And if the sophistication levels

of their own attacks that they're dealing with is a toilet paper wrap device

plugged into a television,

it seems highly improbable that OpenAI has a sophistication to be able to pull

off a similar attack on Apple, although I hope they're not doing that.

So that's really the episode that is an insane story that i read late last night

i woke up this morning i thought i have to make an episode about this if you're

wondering where i still am i am still in kyoto japan and yes

it is amazing yes if you ever come here get a kimono but i still need to keep

putting these episodes out guys it's it's fun and it sounds like all the crazy

stuff is happening whilst i'm on this crazy time zone right now i am,

currently 13 hours ahead of the world or New York, as I call it, and.

News breaks overnight as I'm asleep.

So thank you so much for listening to this episode. I hope you enjoyed it.

I hope there was some valuable and useful takes.

I would say if you had to leave this episode with one top takeaway,

it's that objectively, the DNA or surface area that you have online on digital,

surfaces, whether it's your phone, whether it's your computer,

whether it's any kind of agent or software package that you use.

Is now being tracked, especially if you use ChatGPT or Claw.

And that's not necessarily a bad thing, as we've seen in this instance.

And so I'm happy to finally talk about a crazy, dangerous thing that actually

had a really good ending.

Because at the end of the day, I do think optimistically that when it comes

to cyber defense or cybersecurity, using AI can actually end up putting us in

a much better world than we exist in today.

And all the doom it takes about AI agents running rogue will definitely happen,

but we'll have a way to figure it out. So if you enjoyed this episode or if

you disagree with any of my takes, let me know. Leave us a comment.

Please share these episodes with a friend, whether you listen to it on Spotify,

Apple, or wherever you listen to it. We are anywhere and everywhere.

Give us a rating. Leave us a comment. I read all of them. I try and reply to

all of them. Give me feedback.

I know this is a solo show now, but we're trying to figure it out, me and producer Luke.

So if you have any feedback, it's been great. i've been you know really enjoying

the feedback that i've received so far and also follow me on x i kind of post

a lot of my thoughts even when i'm you know in japan with my family,

um and so you know you can hear more news there more current up to date and

some of my crazy takes i guess you can also dm me there as well,

but yeah thank you so much for listening and i will see you on the next episode

of in the loop brought to you by qualcomm thank you.