Limitless: An AI Podcast

Ejaaz:
The King is officially back. Anthropix flagship model Fable 5 is finally available

Ejaaz:
for everyone after being banned for almost two weeks by the US government.

Ejaaz:
It is still the world's best model. I've been using it in the 12 to 24 hours

Ejaaz:
since it's been launched. I'm super excited about it.

Ejaaz:
There's a bunch of demos that we want to get into almost immediately,

Ejaaz:
starting off with my favorite kinship as a child, Hogwarts and Harry Potter.

Ejaaz:
What you're seeing in front of you is a completely simulated fabrication of

Ejaaz:
the entire Harry Potter universe starting off with Hogwarts and you'll notice

Ejaaz:
that I can click into this universe immediately and start looking around.

Ejaaz:
It's important to mention that this was built from a single prompt.

Ejaaz:
I'm not entirely sure how long it took but Fable 5 did this in one complete

Ejaaz:
shot and if I press B you'll notice something pretty spectacular.

Ejaaz:
I'm on a broom and I'm flying around in probably my favorite childhood universe.

Ejaaz:
Josh, I don't know if you're a Harry Potter fan, but this is pretty freaking awesome.

Josh:
It's unbelievable. And I'm not sure if it was one shot. This is pretty impressive

Josh:
for one shot, but it was absolutely generated with Fable 5.

Josh:
And it's a testament to how impressive the model is and how happy I am that it's back.

Josh:
Thank God. Holy shit. It's been a long couple of weeks without you.

Josh:
I mean, what's most amazing about this, I find, is that not only is it able

Josh:
to generate the map of Hogwarts, but it's able to generate the unique pieces.

Josh:
Like you just went into the dining room. This is the weeping willow tree.

Josh:
It has sound design built into it.

Josh:
It understands the physics and the scaling of the way things should be built.

Josh:
And look, and like, now you're going over to play Quidditch.

Josh:
It's this like unbelievable awareness of the world and this unbelievable ability

Josh:
to generate these visual elements. Like this is so amazing. The fact that you're

Josh:
playing this in your browser

Josh:
just by someone who prompted the Fable a couple of times and then deployed this.

Josh:
And you could see it's like pretty accurate. You have Hogwarts,

Josh:
you have the dining hall. Everyone has Fable until July 7th,

Josh:
I believe the day is. 7th. In which the limits get...

Ejaaz:
Unpaid subscriptions.

Josh:
And then the limits change a little bit. And this is subject to change as always.

Josh:
But currently right now, you have about a four to five day window to really

Josh:
play around with this as freely as it's likely ever going to be.

Josh:
The intention of showing this demo is to showcase that the possibilities of

Josh:
what you can make with this model really are pretty much endless.

Josh:
And I think the challenge to everyone now is now that there is this model that's

Josh:
out that's truly unbelievable in capability relative to anything we've seen,

Josh:
it's to go out and actually try unique and fun things.

Josh:
Like I would have never thought to go off and make a Hogwarts game that I can go and play.

Josh:
I mean, traditionally you buy this game. I have a Hogwarts game that I bought

Josh:
for 75, $80 on Xbox to play around with. So what types of difficult challenges

Josh:
do you have that you'd like to try and that you never thought were possible?

Josh:
And like, those are the things that I would encourage trying out with Paypal 5.

Josh:
Really amazing model very cool you can see here the second demo this is all

Josh:
about physics and the physics understanding and it shows it on a relative basis

Josh:
compared to the other models and it's just so far superior in its understanding

Josh:
of the worlds of physics so

Josh:
a lot of these are visual examples it does really well on basically all other types of work so

Josh:
That's the thing is just try it out, see what you can do. I love this intro

Josh:
post that they shared Fable is back and it's now showing and the whole sign

Josh:
lights up and it's like, man, we are so back. So really exciting day.

Josh:
If you were listening to this, go track Fable, go play around with it.

Josh:
It is an incredible model.

Ejaaz:
I want to talk about what has changed because obviously like we can't ignore

Ejaaz:
the elephant in the room, which is this thing was banned for about two weeks.

Ejaaz:
The US government specifically banned it because it presented itself as a potential

Ejaaz:
cybersecurity threat across all national defense systems, which of course is very important.

Ejaaz:
The TLDR of their blog post is essentially they have added further safeguards

Ejaaz:
and more monitoring as to how people use the specific tool.

Ejaaz:
They'll be looking at every single prompt in terms of whether it could potentially

Ejaaz:
be an attack vector on the US or anyone in general.

Ejaaz:
And they've been working closely with partners from Glasswing,

Ejaaz:
their collective, which they set up once they created Mythos to make sure that

Ejaaz:
their defense systems are hardened appropriately against this model.

Ejaaz:
That being said, there was a lot of fear mongering around the re-release of

Ejaaz:
this model. There was talks of potential KYC verification or potentially only

Ejaaz:
allowing people in the US or American-born people to get access to this.

Ejaaz:
I'm happy to say that all of that has been dismissed.

Ejaaz:
The government and Anthropic, as well as OpenAI and a bunch of other Frontier

Ejaaz:
AI labs, have been working pretty closely together to form some form of a framework

Ejaaz:
for any future model release such that

Ejaaz:
Everyone can be banned against it, but it's released in a more secure way that

Ejaaz:
people can still benefit from the access of it and not be used in a malicious

Ejaaz:
way. So I'm happy about this. This is Fable 5 back in all of its glory.

Ejaaz:
I'm using it across all of my chats right now. Opus 4.8, you have been a pleasure

Ejaaz:
to work with, but sorry, I need to work with your older brother at this point.

Ejaaz:
But I'm excited about it. You have until July 7th, as Josh mentioned.

Ejaaz:
So use it as much as you can.

Ejaaz:
It switches to usage based pricing depending on what task you have.

Ejaaz:
But it's great to have it back um josh what have you been using it for personally i'm

Josh:
Curious i've been using it for all of my hardest challenges and all of the things

Josh:
that i do mostly day to day a lot of um just like research a lot of production,

Josh:
pretty much any challenge that i have i have switched over to fable and i noticed

Josh:
that it does it much better,

Josh:
so in the case that we're researching a topic for a show for example it will

Josh:
go much deeper it's more comprehensive you can kind of read through its reasoning

Josh:
and understand how it comes to conclusions and it's a really good thought partner

Josh:
at least for me in a lot of cases,

Josh:
a big problem that i have again is figuring out where the edges of this thing

Josh:
are and what i actually can use it for to be a little bit more creative it's

Josh:
like okay i shouldn't really be using this as an extension of opus what new use cases can i try

Josh:
and i think that's what i'm going to be spending a lot of time this weekend trying to figure out is

Josh:
what are the harry potter like demos that i could use that i'm actually really

Josh:
interested in building myself.

Ejaaz:
Yeah so one thing i've noticed is um i've never had a personal website before.

Ejaaz:
And I've thought like, hey, why don't I just give it a go and try and figure it out?

Ejaaz:
Now, the one bit of critique that I've always had with any AI model up until Fable 5 is,

Ejaaz:
the taste is just terrible. I'm sure you've tried this before,

Ejaaz:
but you're like, hey, like, build me a website that looks something like the

Ejaaz:
Airbnb website, you can share screenshot demos.

Ejaaz:
And it just kind of does a terrible job or a janky job.

Ejaaz:
Fable 5 is the first model that I've worked with that

Ejaaz:
has this visual intelligence, which is super weird, because I can't currently

Ejaaz:
use Claude right now to generate me an image, I can use ChatGPT to do that,

Ejaaz:
I can use Google Gemini to do that, Nano Banana, but I can't do that with

Ejaaz:
Claude specifically right now as an LLM.

Ejaaz:
But yet it understands what I see so well.

Ejaaz:
So what I've realized is if I'm using it to code up some form of a visual artifact,

Ejaaz:
and we use visual artifacts a lot on our show as props or materials,

Ejaaz:
it does an amazing job and it understands taste.

Ejaaz:
It doesn't look like generic AI slop anymore. So I definitely recommend folks

Ejaaz:
go and check that out. Now, one thing that we have to mention on the show that we noticed pop up was

Ejaaz:
There was a discussion once Fable 5 was re-released because people started coding

Ejaaz:
with this thing and they updated their recent version.

Ejaaz:
And they noticed that there was some form of like, they described it as a spyware-like

Ejaaz:
code in CloudCode that would basically track certain users' kind of activity on what they were using.

Ejaaz:
And obviously there's a big reason for this because a lot of foreign state-based

Ejaaz:
labs, such as in China specifically, were distilling Cloud models to build their

Ejaaz:
own models, which is strictly against Anthropic policy.

Ejaaz:
And what we got was a very quick dismissal from Darek himself at Anthropic saying

Ejaaz:
that this was an experiment that we were launched to basically prevent distillation

Ejaaz:
attacks from foreign adversaries.

Ejaaz:
Now, if you're Anthropic and if you've been following the news,

Ejaaz:
especially on Limitless, you'll know that a bunch of the Chinese open source

Ejaaz:
models have effectively attained their status because they've distilled frontier

Ejaaz:
models from American labs such as OpenAI and Anthropic.

Ejaaz:
There was recent reports, I think this week of Alibaba's Quen doing the exact

Ejaaz:
same thing. There was an official announcement from Anthropic.

Ejaaz:
So it's good to see us taking these steps and measures to prevent such attacks.

Ejaaz:
This is part of the whole safeguard approach, which is new with Fable 5.

Josh:
Yes, big week for Anthropic. In addition to that, they also released Sonnet

Josh:
5 and Cloud Science, two major releases.

Josh:
Sonnet 5, as everyone knows, is the kind of smaller model. It's the follow-up to Sonnet 4.6.

Josh:
In fact, it's priced a little bit lower than 4.6, at least for the intermediary up until August 31st.

Josh:
And it really is just like the entry-level model and the model you'll find that

Josh:
Fable probably best wraps to.

Josh:
So when you're using Fable, sometimes it gets expensive. You're using it for

Josh:
very long-running tasks. Sometimes not everything needs frontier intelligence.

Josh:
Sonnet 5 is a pretty good substitute for it to defer and start to offload tokens and requests to.

Josh:
And it seems like it's just like, you know, a solid upgrade.

Josh:
There's nothing exceptionally incredible about it, but it is there.

Josh:
It is cheaply priced and it is very strong in the parts that it needs to be.

Ejaaz:
I actually really like the look of Sonnet 5 for one particular reason.

Ejaaz:
Over the last week, people have been praising GLM 5.2, which is a Chinese model from the lab Jipu.

Ejaaz:
Way and we did an episode on this and it's a very impressive model um but

Ejaaz:
but my take on this is i think american frontier labs aren't just going to sit

Ejaaz:
around and let chinese ai labs outcompete them on cost specifically

Ejaaz:
so they have the best cards to play which is if you have the frontier ai model

Ejaaz:
such as claude fable 5 or chat gpt 5.6 sol

Ejaaz:
you can just distill your flagship model into a much cheaper model and that's

Ejaaz:
what we're seeing with sonnet 5 it does opus 4.8's capability but at a fraction of the cost.

Ejaaz:
And so I think we're going to see this trend continue until we find that perfect

Ejaaz:
medium in between how cheap GLM 5.2 is and how expensive Fable 5 is,

Ejaaz:
and people will just stay within the Claude ecosystem.

Ejaaz:
And I think it's a smart approach, and I'm happy to see it. But you mentioned

Ejaaz:
Claude Science as well, Josh.

Josh:
Super cool, yeah.

Ejaaz:
As a former bio nerd that did a four-year biology degree and spent a lot of

Ejaaz:
time studying genetics, I wished I had this tool.

Ejaaz:
This is basically the co-pilot for any kind of scientific research that you

Ejaaz:
do. It's a model that basically is fine-tuned and built to evaluate scientific

Ejaaz:
primary research, help you do all your data analysis.

Ejaaz:
And it's been used across a ton of different things. I mean,

Ejaaz:
if you look at this guy called Andre, who has been playing around with cloud science.

Ejaaz:
He installed it with Ligand AI, which is basically an MCP tool,

Ejaaz:
which basically allows him to analyze different molecules.

Ejaaz:
One of the big things when you're kind of figuring out whether an antibody or

Ejaaz:
a molecule is good enough to fight a disease is you need to know whether it

Ejaaz:
can attack a specific active site, which is like basically the vulnerable site of a

Ejaaz:
bacterial virus or whatever that might be.

Ejaaz:
And this system with cloud science

Ejaaz:
basically helps you identify it um de novo which is like super cool

Josh:
Yeah i think this is probably the most interesting and exciting release of the

Josh:
week granted fable 5 already released so we're just doing it again um but cloud

Josh:
science you could think of it kind of like.

Josh:
The way that cloud code is intended to support engineering cloud science is intended

Josh:
to support science and that's all of what this is about it integrates a very

Josh:
specific set of tools it allows you to audit the outputs and anthropic is actually

Josh:
using it they announced that they're using it to pursue their own drug discovery

Josh:
programs for some rare neglected diseases. And people have been using it.

Josh:
The Allen Institute now completes 100 plus page literature reviews that would

Josh:
previously take up to two years.

Josh:
And then some analysis now run in a 10th of the time of what they previously

Josh:
did, thanks to this new software harness that they've introduced for science based purposes.

Josh:
So when you combine a model like Fable into a harness like Claude Science,

Josh:
you get some pretty amazing technological.

Josh:
Innovation or at least the opportunity to do so if you are feeling so inclined

Josh:
to through people who just like are involved in research in general so i think

Josh:
this is super exciting for the science world i think we're going to see a lot

Josh:
of really interesting things coming from this and that moves us on to our next

Josh:
topic which is science related from

Josh:
meta believe it or not the uh the metaverse company that was supposed to make

Josh:
metaverse they were all supposed to make ai well it turns out the thing they're

Josh:
releasing this week is is biological based and it is a brain-to-text decoder.

Josh:
What is a brain-to-text decoder? Well, basically, topologically,

Josh:
they insert some, or they put some electrodes on your head, and correct me if

Josh:
I'm wrong, you guys, but they have actually managed to hit 61% word accuracy

Josh:
versus 8% for previous non-invasive methods.

Josh:
So, they're basically able to go into your brain, detect which neurons are firing,

Josh:
and then derive a series of words that you are meaning to say without actually

Josh:
saying those words. It's essentially reading your mind. Is that right?

Ejaaz:
The best part about this is they only did this with nine volunteers,

Ejaaz:
nine people, no, not even double digits.

Ejaaz:
And they trained this model on basically 22,000 sentences. So comparably,

Ejaaz:
if you look at like an LLM, which is honestly the frontier LLMs like Fable 5

Ejaaz:
are upwards of like 15 trillion parameters, and they use a ton of compute.

Ejaaz:
This is a small model in comparison, but it's equally so, so powerful.

Ejaaz:
They basically are able to interpret different brain signals and convert that into text or words.

Ejaaz:
The reason why this is so important is, well, firstly, there's the medical side

Ejaaz:
of things. We've spoken about Neuralink quite a bit.

Ejaaz:
This is kind of Meta's answer to that, at least in the initial types of research that they're doing.

Ejaaz:
Now, application wise, I am curious what Meta is going to use this model for.

Ejaaz:
Obviously, there's the medical side of things, but we know and we've spoken about this previously.

Ejaaz:
They are working on a series of different brain models, as well as ECG-related

Ejaaz:
devices that you can wear on your wrist and on your heart that can basically

Ejaaz:
interpret everything that you're thinking and figure out what the best type

Ejaaz:
of content is to produce for you.

Ejaaz:
So it's an interesting thing where Facebook is a social media site.

Ejaaz:
And one thing that they've spent a lot of time figuring out is how to get all

Ejaaz:
the wonderful imaginative thoughts in your head out into the open world.

Ejaaz:
They've done that with Instagram through you picking up a phone and taking pictures of stuff.

Ejaaz:
They've done that with Facebook, translating your thoughts into cool ideas that

Ejaaz:
you can share with the world.

Ejaaz:
This seems to be a step in a direction where for those folks who aren't really

Ejaaz:
good at, you know, using those skills to translate your thoughts into some kind

Ejaaz:
of a vivid pitch or whatever that might be, you can now use this potential tool

Ejaaz:
in the future for something like that.

Ejaaz:
And listen, I have a more optimistic take for all these kinds of tools,

Ejaaz:
which is, you know, they're going to use it for the advancement of humanity and good.

Ejaaz:
But I don't know what meta specific purpose for that might be.

Ejaaz:
But it's interesting to see nonetheless. And I like that this kind of research

Ejaaz:
is being focused on despite LLM is being focused on being chatbots or generating

Ejaaz:
images this is like advancing real science and you know between OpenAI's gene

Ejaaz:
benchmark between Claude Science and between this this is great to see

Ejaaz:
uh it's a great week for science in AI for sure

Josh:
Yeah it's been really exciting it's cool to see progress and then I'm also just

Josh:
like very confused at what Meta is doing because we're talking about science

Josh:
now but then there was another big pivot that just happened where Meta is shifting

Josh:
to cloud infra they're they're like pivoting to the SpaceX plan so it seems like

Josh:
Meta over the last decade has just been this series of unfortunate pivots and

Josh:
every time we pivot somewhere new and we try this new hard thing it doesn't

Josh:
quite work out and they pivot again and it feels like a startup kind of trying to find some traction

Josh:
this is the newest iteration of that in addition to the science breakthrough

Josh:
that they're sharing in terms of like the topical neuron receiver stuff they're

Josh:
also just going to start selling compute

Josh:
so i don't know if this is bullish or bearish for their internal ai

Josh:
program i have to imagine it's not that exciting because if it was going so

Josh:
well they wouldn't have extra compute sell

Josh:
but the idea here is at least that they will begin to sell their compute to

Josh:
third-party companies similar to what spacex has been doing with their deals with anthropic and

Josh:
who else did they sign a deal with google and i think it's just another instance

Josh:
of them trying things throwing stuff against the wall and seeing what sticks

Josh:
i don't know you just do you have any takes when you read this news.

Ejaaz:
Yeah, I have an alternative take, which is, I think it's a smart move for Zuck.

Ejaaz:
It is... Market loved it. Yeah, it's a potential massive revenue earner.

Ejaaz:
The stock went up, I believe, like 6% to 8% on the day.

Ejaaz:
And the reason is pretty simple. Meta has aggregated one of the biggest fleets

Ejaaz:
of the most important metals needed for AI right now. And that is the GPUs.

Ejaaz:
We said the same thing about SpaceX. So SpaceX has Colossus 1, 2, and 3 data centers.

Ejaaz:
And that is roughly over, I think it's a million of NVIDIA's top GPUs.

Ejaaz:
And so that fleet can be very attractive for competing labs,

Ejaaz:
such as Anthropic and Cursor, who they are selling their GPUs to.

Ejaaz:
Meta saw this and thought, hmm, that did pretty well for the stock price.

Ejaaz:
Let me do the same over here and bring some money in to justify additional AI

Ejaaz:
capex spend for future years. So I think this is a strategic move from Zuck to do this.

Ejaaz:
And I have another bit of evidence which proves that this isn't necessarily

Ejaaz:
Meta running for the hills, which is Meta signed a compute deal with Google.

Ejaaz:
And Google this week announced that they have to restrict Meta's use of their

Ejaaz:
Google Cloud because they're using too much of it.

Ejaaz:
So my suspicion, and we'll find out in the Q2 earnings report from Meta in a

Ejaaz:
few weeks' time, is Meta's allowing all their old GPUs to be sold for inference.

Ejaaz:
And they're going to make a ton of money in the same way that SpaceX has charged

Ejaaz:
a huge premium to Anthropic and Cursor for selling inference through their GPUs.

Ejaaz:
Meta's going to do the same, make a bunch of money from that,

Ejaaz:
and save all the Vera Rubins and the newer GPUs to train future Meta models.

Ejaaz:
And we reported, I think, two weeks ago that Meta is working on a potential

Ejaaz:
Mythos level model that's going to be available nine months from now.

Ejaaz:
I know that's kind of late, but I still think that in this race,

Ejaaz:
they're just very, very, very far behind.

Josh:
Well, hey, all the power to them. I hope they figure it out.

Josh:
Now we should probably pivot over to someone who has figured it out mostly, which is OpenAI.

Josh:
And we've recorded an entire episode on this. They released 5.6 this week.

Josh:
If you haven't listened, I would highly advise going to check out that episode you recorded.

Josh:
The update now is that it's still not live. And by the time of reading this,

Josh:
that might have changed. By the time you're listening to this,

Josh:
hopefully that's changed. I think everyone's really excited to get their hands on it.

Josh:
But they have not met the same fate that Fable has just yet.

Josh:
So they have Sol, Terra, and Luna, all associated with GPT 5.6. they are

Josh:
probably being used internally you have to imagine they are being used in an

Josh:
internal research project

Josh:
that is with select companies but they are not broadly available so we're still

Josh:
at a standstill in that point but the real news that we got this week was this

Josh:
five percent number that's been floating around and none of this is confirmed

Josh:
by any means but there's a lot of reporting going on

Josh:
in that open ai is proposing to hand the,

Josh:
united states government five percent of the company just for free right they're

Josh:
just gonna give it away yeah.

Ejaaz:
Yeah um i was going to bed last night, and I thought we had our agenda all prepped

Ejaaz:
and ready, Josh, for today's roundup.

Ejaaz:
And then I saw this breaking news from Financial Times, which basically says

Ejaaz:
that Sam Altman is reportedly shopping 5% of OpenAI to the US government to take ownership.

Ejaaz:
And the craziest part about this is, I don't think he's even requesting an investment.

Ejaaz:
He's just kind of like handing this over for the good of the world.

Ejaaz:
And if you want to know some context as to why he might be doing this,

Ejaaz:
I think there's two perspectives of this. Perspective number one is

Ejaaz:
Sam believes deeply that governance and national government should be involved

Ejaaz:
in how these AI models are shaped.

Ejaaz:
And we know this because in an earlier blog post from Sam, I think about a year

Ejaaz:
and a half ago, he said this. He said it might actually make a lot of sense

Ejaaz:
for the government to take stakes in major AI labs.

Ejaaz:
The reason for this is, as we've seen with Claude Mythos and Fable 5 being banned,

Ejaaz:
it is incredibly important that the government gets access to these models pre-release

Ejaaz:
so that they make sure that it's not a cataclysmic threat to government security systems and whatnot.

Ejaaz:
So it seems to be a good perspective from that side of things.

Ejaaz:
But on the other side of things, this could be Sam using a chess piece to basically

Ejaaz:
get a political advantage when it comes to writing policy frameworks.

Ejaaz:
As we know, Sam's been heavily involved in lobbying certain governments and

Ejaaz:
laws in terms of making frameworks more favorable for OpenAI's types of models.

Ejaaz:
We see other AI labs doing the similar types of things. If you were a critical

Ejaaz:
person of Sam, you might think that this is the move that he's making.

Ejaaz:
It's kind of weird. I don't know how I feel about the whole nationalization

Ejaaz:
of the AI movement, but it certainly seems to be the trend that we're moving towards currently.

Josh:
Well, I like the fact that it was at least voluntary. This was presumably not

Josh:
forced. They're offering this up.

Josh:
You have to assume, I mean, again, these are all reports. Nobody knows what's

Josh:
actually going on, but the idea is that they would just make a generous donation

Josh:
to the united states government and this makes sense when you look at it in

Josh:
terms of alignment it's like what is the

Josh:
most pressing issue that a lot of people are very upset about right now it's

Josh:
ai and the downstream effects that they think it's going to cause on their life

Josh:
they're scared they don't understand um they have no vested interest in this

Josh:
and i think giving everyone a

Josh:
even if it's a small amount of vested interest in the success of these companies is probably net good

Josh:
If it could even change the sentiment a little bit, if it can get people aligned,

Josh:
if they could feel good about these companies winning, because it means not

Josh:
only are they winning, but the country is winning and everyone is kind of aligned

Josh:
directionally on the success of these companies.

Josh:
I think it creates a really compelling case for handing some over to the government

Josh:
and giving it back to the people that are allowing you to do this.

Josh:
They might just be feeling exceptionally confident this week because we have

Josh:
another report, another rumor, again, unconfirmed.

Josh:
All of this is unconfirmed. We are waving our hands around. We are putting our tinfoil hats on.

Josh:
But allegedly there has been some sort of optimization achieved internally that

Josh:
allows them to cut inference costs in half for models that was applied to that,

Josh:
that's crazy because half of the costs on hundreds of billions to trillions

Josh:
of dollars is not marginal that is a pretty serious innovation so what what

Josh:
is going on here you just is this even possible.

Ejaaz:
Yeah. So if this report is true, the analogy looks like the following.

Ejaaz:
Right now, if you're a free user, ChatGPT, there's roughly around,

Ejaaz:
I'm going to say 100,000 to 200,000 GPUs that are out there that are processing

Ejaaz:
your prompts, your inference, your requests, giving you answers.

Ejaaz:
Those GPUs, you know, I hate to mention it, are super, super expensive.

Ejaaz:
Now, if this report is true, it's cut down that inference cost that cost to

Ejaaz:
50 percent which means that this is now a couple

Ejaaz:
hundred of gpus and this is not a number that i'm making up this is a number

Ejaaz:
that's in the informations report

Ejaaz:
specifically a couple hundred gpus which means that if you can cut down costs

Ejaaz:
that massively you now have free capital to spend on compute in other ways in

Ejaaz:
open ai's case it's probably to train more models

Ejaaz:
or to use inference for other types of usage such as inference at test time

Ejaaz:
scaling, or whatever that might be.

Ejaaz:
So this is a major jump for a few different reasons.

Ejaaz:
Inference roughly makes up, I think, 60 to 70% of revenue earnings across top

Ejaaz:
frontier labs right now. So

Ejaaz:
if you're a frontier lab, a lot of your cost typically goes into training.

Ejaaz:
A lot of the money that you make comes in from subscriptions.

Ejaaz:
Now it comes in from inference specifically for people using it to service agents

Ejaaz:
that they're running on Autoloop for hours and hours on end.

Ejaaz:
Agents have become basically the new trend. Enterprises are using it 24-7 at this point.

Ejaaz:
So it's become its own type of economy. And inference is basically the bedrock

Ejaaz:
that underpins all of that. Anthropic this quarter is reportedly meant to be

Ejaaz:
the first AI lab that goes profitable in Q2.

Ejaaz:
A large part of this is achieved through the cost or the revenue margins that

Ejaaz:
they earn off of inference specifically. It's roughly around something crazy

Ejaaz:
like 80%. I remember Christian Rao mentioned it on a podcast previously.

Ejaaz:
So the point is inference is very important. And any way that you can cut costs

Ejaaz:
down on that and extend your profit margin is a huge deal.

Ejaaz:
And if OpenAI has found some kind of a breakthrough, and I don't know how they've

Ejaaz:
done this. I'm looking forward to actually an official announcement.

Ejaaz:
This would be a huge leap for them in terms of getting more capital to acquire

Ejaaz:
more compute to build better models.

Josh:
We do have a confirmed breakthrough this week. There is actually something that

Josh:
has been publicly announced and it comes in the form of this company that just

Josh:
exited stealth named Etched.

Josh:
And Etched, I saw the post and I was like, oh, all right, cool.

Josh:
It's like another one of those things. It's like they think they're going to

Josh:
change the world and they're going to solve the GPU problem and tokens and price

Josh:
per watt and they're going to solve all the efficiency gains.

Josh:
And I was like, absolutely not. These guys are like in their early to mid 20s.

Josh:
They have no experience doing this.

Josh:
Surely this is just another one of those things that is a headline and then

Josh:
it devolves into nothing but i started to read about what they were doing and

Josh:
i started to listen to a few of the podcast episodes that these founders were on.

Josh:
And i was shocked at how competent and incredible the product is and how talented

Josh:
the density of this team is and everything about this

Josh:
story turned into this like unbelievable breakthrough i was like wait holy shit

Josh:
this is this is like pretty amazing and pretty magical and i think so much so

Josh:
that we're probably going to record an entire episode about this company and

Josh:
this general state of inference as it really sees breakthroughs probably next week.

Josh:
But just a TLDR on Etch is they've basically figured out how to create...

Josh:
A new chipset from the ground up first principles vertically integrated meaning

Josh:
they're building all these things from scratch they're integrating them all

Josh:
into a single unit their product is the way in which they manufacture

Josh:
this server rack and the server rack is the delivered good but within that requires

Josh:
cooling and chip architecture and all of the plumbing as it connects to everything

Josh:
and they've created this series of breakthroughs using

Josh:
low voltage inference which is one so they're able to squeeze out a lot more

Josh:
performance per watt generally on a gpu like a blackwell chip you are getting

Josh:
maybe 50% of its theoretical peak because if you go to 100%, the chip melts.

Josh:
So being able to squeeze more tokens per watt is a really impressive thing.

Josh:
They figured that out. They figured out a memory problem where normally you're

Josh:
constrained to the memory on a chip.

Josh:
Well, they've constrained it to the memory of the cluster. So the cluster becomes

Josh:
one single coherent piece of memory.

Josh:
And there's a series of these breakthroughs that they've kind of discovered

Josh:
that were really impressive.

Josh:
And then you look at the team and you're like, oh, wow, everyone on this team

Josh:
is like shockingly competent and have been working in the industry for many,

Josh:
many decades. In fact, I think the youngest people are the founders.

Josh:
So Etch seems like a grand slam.

Josh:
Stay tuned. We're going to have a lot more to say about this company.

Josh:
It was shocking. It was really cool to see this week.

Ejaaz:
Well, just to kind of like sing their praises a bit more, these guys have raised,

Ejaaz:
I think, $800 million, but they already have a billion dollars worth of orders

Ejaaz:
for a chip that currently doesn't

Ejaaz:
exist yet and is currently going to go live by the end of the year.

Ejaaz:
The backers for this team are pretty insane. You've got Peter Thiel,

Ejaaz:
you've got Jane Street, but then you've got TSMC themselves.

Ejaaz:
As far as I'm concerned, TSMC is the company that is required for NVIDIA to

Ejaaz:
stay alive at the moment, they do all the manufacturing processes around all their GPUs.

Ejaaz:
And TSMC taking an equity stake in a competitor to NVIDIA in the first instance,

Ejaaz:
as far as I'm aware, is a huge deal.

Ejaaz:
Now, you mentioned what the breakthrough is specifically, I think,

Ejaaz:
When we talk about other companies that have made inference specific chips,

Ejaaz:
because I'm sure you're thinking about this, listening to this,

Ejaaz:
we've spoken about Cerebrus that went public in a huge IPO about a month ago,

Ejaaz:
and they created an inference check.

Ejaaz:
And it was the size of like a couple of plates or something crazy like that.

Ejaaz:
Their breakthrough was effectively designing a chip that would allow your AI

Ejaaz:
models to work incredibly faster, but it would use a lot of power.

Ejaaz:
It was quite expensive to use.

Ejaaz:
The difference with this chip that Etch supposedly is going to create and release

Ejaaz:
at the end of this year is it uses low voltage, which means that it uses 75%

Ejaaz:
less power for the same amount of inference output. That's the breakthrough that they've made.

Ejaaz:
And if this company can pull it off. And like you said, Josh,

Ejaaz:
it's like a bunch of Harvard dropouts.

Ejaaz:
This would be an incredibly competitive threat to NVIDIA. Because as far as

Ejaaz:
I'm concerned, NVIDIA is focused on general purpose GPUs, which are used for

Ejaaz:
training specifically.

Ejaaz:
But the inference market is pretty open to anyone and everyone.

Ejaaz:
That's why you see Cerebrus, Grok, and now Etched come to light.

Ejaaz:
So it's pretty cool to see.

Josh:
It's so exciting. It's rare that a company comes around that's actually new

Josh:
and novel and making real breakthroughs. So I'm stoked about this company.

Josh:
I could not be more excited. I would love to participate. I wish they had the

Josh:
ability to buy some shares because my God, what a monster of a company that's going to be.

Josh:
For the people who do own shares in public markets, chances are they might not

Josh:
have done too hot this week.

Josh:
The memory stocks have been getting hit. People haven't really liked this.

Josh:
And we have this post on screen that's at 2 million views.

Josh:
And there's no way that this post moved memory markets, right?

Ejaaz:
I think it did. That's insane. I was racking my head to try and figure out why

Ejaaz:
memory stocks were down 10% yesterday.

Ejaaz:
It was one of the most aggressive dumps we've seen of recent,

Ejaaz:
especially after a meteoric rise in memory stocks over the last year,

Ejaaz:
pretty much is up on average between three to 500%, depending on which memory provider you pick.

Ejaaz:
Andrew Curran is one of my favorite accounts on X. If you don't follow him,

Ejaaz:
definitely give him a follow.

Ejaaz:
He posts this very cryptic tweet. He goes, I'm posting this prediction now so I can quote it later.

Ejaaz:
There has been a significant breakthrough in architecture, specifically around

Ejaaz:
memory efficiency, not by one of the big labs, but a team that spun out of OpenAI.

Ejaaz:
This will probably be announced soon.

Ejaaz:
And this led on everyone on a spiral in social media.

Ejaaz:
Now, I have a bit of tea because I have some sources which reported a few things to me.

Ejaaz:
Now, apparently the company that he was referencing is this company called Core

Ejaaz:
Automation. What this breakthrough is,

Ejaaz:
He has no idea. We have no idea. So right now, it's just word of mouth.

Ejaaz:
And if core automation wants to make an announcement, they can.

Ejaaz:
But if this announcement is true, it would mean that AI models largely require

Ejaaz:
much less memory. Now, why is that a big deal?

Ejaaz:
The most expensive material that is required to build GPUs and train models

Ejaaz:
and inference models right now is memory. It roughly makes up around 50 to 60% of the cost.

Ejaaz:
And the price per unit of memory has skyrocketed over the last year specifically.

Ejaaz:
Margins on Micron's recent earnings report, for example, has hit 80%.

Ejaaz:
Traditionally, this is around 30%.

Ejaaz:
So the fact that it is over double this is absolutely insane.

Ejaaz:
And the margins keep going up, the prices keep going up. In fact,

Ejaaz:
Apple announced recently that they are lobbying the Trump administration to

Ejaaz:
buy memory from Chinese companies, which is a big no-no in terms of import-export

Ejaaz:
controls that Trump and previous US governments have placed.

Ejaaz:
We don't know if this is going to get passed, but the point is companies like

Ejaaz:
Apple need to now increase the price of their products because they can't get

Ejaaz:
their hands on enough memory.

Ejaaz:
So they need to end up paying more, which means they need to charge you as a

Ejaaz:
consumer more. So memory is in very scarce supply.

Ejaaz:
My take on all of this is it's being blown out of proportion.

Ejaaz:
Memory is an incredibly constrained commodity, and there's no way to scale it.

Ejaaz:
You need a bunch of very complex machinery and chip fabs to be able to do this

Ejaaz:
and we are nowhere near i don't see the supply scarcity being alleviated until

Ejaaz:
at least 2028 by the end of that year

Josh:
Well, yeah, we know this. The markets are very short-sighted always.

Josh:
When there's headline news, they're going to overreact. There's going to be

Josh:
this knee-jerk reaction. That's what we've seen. In fact, there isn't even really

Josh:
headline news. This is just kind of all speculation.

Josh:
It's funny, I'm looking at Core Automation and their URL is coreauto.com.

Josh:
It looks like a freaking car manufacturing company. but it's like okay sure

Josh:
like they have these breakthroughs but how even in the case that they do have

Josh:
a breakthrough open ai or anyone what is the

Josh:
the lag time between breakthrough to actual implementation where it is felt

Josh:
in the market probably fairly long there's probably

Josh:
i mean certainly a tremendous amount of demand for memory in everything that's

Josh:
being built not only on the software side but also on the hardware side if you're

Josh:
building robots if you're building automated hardware manufacturing,

Josh:
all of this requires requires a tremendous amount of memory so is Is there a memory bubble popping?

Josh:
Like, probably not. But again, not financial advice.

Josh:
Who the hell knows? Talking about robots, though, we do have a robot topic because,

Josh:
we're getting some consumer robots like robots are becoming a thing.

Ejaaz:
Look at this cutie pie, Josh. You want this in your apartment?

Ejaaz:
You want this in your home?

Josh:
It's pretty adorable. Like, do I want that in my apartment? Honestly,

Josh:
no. My apartment isn't big enough for the two of us.

Josh:
I feel like we'd probably bump into each other and it probably wouldn't be.

Ejaaz:
This thing is massive.

Josh:
It's massive. And, you know, it's still moving a little slow.

Josh:
It's like not really doing everything that i need for my liking maybe if you

Josh:
live in a big house it's fun to have this guy running around but i do think

Josh:
it's cool and exciting that more companies are trying this in fact this one's

Josh:
actually dare i say kind of cute

Josh:
it looks like this fun little robot that you can have walking around the house

Josh:
you can get it in different colors and you can pre-order it for eight thousand

Josh:
dollars which hey for a humanoid robot that can roll around your.

Ejaaz:
House a month or 500 bucks a month so like you know an expensive new york gym

Ejaaz:
membership you can have uh you know this robot walking around and cleaning your

Ejaaz:
apartment with these freaky looking pincers

Josh:
Did you see the demos did it look compelling was this an interesting product.

Ejaaz:
Honestly no it was too slow for me and i hate the the fact that i'm pointing

Ejaaz:
this with my cursor right now that it's effectively one of those old in high

Ejaaz:
school josh did you have those like projectors where like sometimes it

Josh:
Rolls around the.

Ejaaz:
Cart rolls around the cart this looks exactly like that my number one question

Ejaaz:
is like when you come across like a carpet or something like how are you getting

Ejaaz:
over there what like i don't want to have to like be uh

Ejaaz:
the topple monitor where i have to like pick you up if you fall around like

Ejaaz:
how do you know where to place my clothes i I don't know, there's just a number

Ejaaz:
of different questions that doesn't justify $8,000 in my mind, or even $500.

Ejaaz:
But the good news is, this isn't the only company that's releasing robots like this.

Ejaaz:
We had Nori L2 demoed there. Less attractive robot, I have to say,

Ejaaz:
more bare bones, but this is like a demo.

Ejaaz:
It seems to move a little bit more agile and quicker than the previous Weave product or robot.

Ejaaz:
The fact is, a lot of American robotic startups are releasing their products

Ejaaz:
now. And if you compare that to six months ago, it was all theory.

Ejaaz:
It was all hearsay. It was all demos from the likes of FIGURE.

Ejaaz:
You had their robots in the BMW manufacturing plant, if you remember this,

Ejaaz:
and it was supposedly operating all the controls and building cars.

Ejaaz:
Now we see these robots, one, coming at a more affordable rate,

Ejaaz:
and two, being applied to your home.

Ejaaz:
Now the big question is, do you want to allow any of these robots in your home right now?

Ejaaz:
And can this be scaled, mass-produced can this be in my hands right now the

Ejaaz:
answer is still unfortunately no but the target date is a lot closer you can

Ejaaz:
get these robots in a couple of months or by the end of the year at least so you know uh

Ejaaz:
fingers crossed i hope we're entering our robotics era our chat gpt era for

Ejaaz:
robots but uh right now i'm not utterly convinced uh the pinces still throw

Ejaaz:
me off i need some hands i need something like dude i'll

Josh:
Believe it when i see it man ship the product let me see working demos in real

Josh:
people's houses like give me something to really get me invest.

Ejaaz:
In the company you know a few other things like that but only

Josh:
The good ones it's like a lot of most of these companies are going to fail and

Josh:
they're going to just create fun products and they're going to create these

Josh:
little like flashes in a pan and then they're going to dissolve into nothing

Josh:
because turns out it's really difficult to manufacture these things at scale

Josh:
and now they're competing,

Josh:
with the supply chain that the entire planet is also competing on they need

Josh:
the same memory that these other ai companies need and there's only so much

Josh:
of it to go around and the economies of scale are just not really practical for these companies.

Josh:
So in terms of research and development, I think it's amazing.

Josh:
It's fun to kind of hone in on a form factor of what these robots could look like.

Josh:
It's fun to figure out demos and use cases. But in terms of actual practicality,

Josh:
I don't think I'm going to have a humanoid robot in my house this year.

Josh:
I don't think I'm going to have one in my house next year. Maybe 2028,

Josh:
perhaps Optimus will be around by then. Perhaps Figure will have some cool robots by then.

Josh:
But it's going to take a little while to ramp up these production lines and

Josh:
get something that's actually viable via these economies of scale.

Ejaaz:
But here's a question for you, Josh. What would be the one...

Ejaaz:
Pain in the ass that you would definitely allow a robot to solve that would

Ejaaz:
be that would tell you like you know what i'll pay the money i'll pay 300 bucks

Ejaaz:
a month to get this robot in my house to solve this one problem do you have one

Josh:
You know what's funny is like i don't think there is a problem

Josh:
so large that it requires a robot it's like one of the things that's kind of

Josh:
annoying is groceries i have doordash and uber eats they will go and shop for

Josh:
me and they will deliver it or even with whole foods they have their grocery

Josh:
delivery you can get someone to come and clean in your place once a week.

Josh:
And that's like a very good service. And they're going to do a significantly

Josh:
better job than a robot will.

Josh:
Okay i i do laundry but it takes what like 10 minutes to fold the clothes and

Josh:
it's not that big of a deal uh make my bed in the morning when i wake up i like

Josh:
to cook my own food there's a dishwasher to handle the cleaning of it

Josh:
it's like there isn't really anything very pressing in terms of i mean initially

Josh:
my head goes to chores that i think a human or robot would would require so

Josh:
i think that's kind of why i'm waiting just to see,

Josh:
the unique use cases they could figure out or if they just get it down yeah

Josh:
i think it's really down to use cases if there's a killer use case that I see, I'm like, I need that.

Josh:
Then sure, there's no amount that I wouldn't spend on getting this because it's

Josh:
such a cool piece of technology. But there has not been anything even remotely

Josh:
close to that use case. It turns out we've actually automated a lot of things

Josh:
pretty effectively in our life.

Ejaaz:
You see, I've come to the same conclusion, and that's not necessarily bearish

Ejaaz:
robotics. I just think it's bearish robotics in your home.

Ejaaz:
I think robots apply to more industrial use cases, such as manufacturing,

Ejaaz:
or maybe even food delivery makes a lot more sense.

Ejaaz:
I think like fixing that last mile problem, last mile delivery problem where

Ejaaz:
like, you know, you can deliver everything to another country by getting it

Ejaaz:
to that specific address is the most annoying part is a valid use case for robots.

Ejaaz:
I think the manufacturing side of things makes a lot of sense in terms of like

Ejaaz:
car automation or building whatever in the future, maybe even building robots

Ejaaz:
in themselves in the future, which is what Tesla's focused on.

Ejaaz:
This is what Tesla and this is what Prometheus, which is Jeff Bezos's new robotics companies focus on.

Ejaaz:
This is what Atoms, Travis Kalanick's ex-Uber CEO is also focused on as well, right?

Ejaaz:
They're focused on the manufacturing and industrial applications of robots.

Ejaaz:
And I think if you were an investing type of person that wants to focus and

Ejaaz:
figure out where the robotics trends is heading, I think it's in these particular

Ejaaz:
sectors. And that's what I'm

Ejaaz:
I don't know, a robotics specific episode going forwards. If that's something

Ejaaz:
that you guys are listening to and would like to hear more of,

Ejaaz:
I think Josh and I can whip that out. That would be a pretty fun one to do.

Josh:
Yeah, let us know. Would love some feedback. Oh, I wish there was a way to participate

Josh:
in these companies, man. It's driving me crazy. Of all the cool companies we

Josh:
talk about and every single one of them is private.

Josh:
It's like, oh, Etched is awesome. Can't invest. Adams is the most incredible company. Nothing.

Josh:
So I don't know. Hopefully through all of this, we'll figure out a way to get

Josh:
access to the upside of these amazing companies. But I think that's everything.

Josh:
This is a long episode. If you're still rocking with us here,

Josh:
congratulations and thank you. You are a real one.

Josh:
We appreciate everyone making it to the end. The outreach for sponsors has actually

Josh:
been so nice and so sweet.

Josh:
So thank you for everyone who's reached out in terms of getting us help with

Josh:
sponsors, becoming a sponsor yourself.

Josh:
We are looking for people to help support the show so we can keep the lights

Josh:
on and continue to do this for longer. Yeah. So please, if there's anyone you

Josh:
know, please feel free to reach out. We will be happy to converse either through

Josh:
X or in the email in the description below.

Josh:
If you enjoyed this episode don't forget to share it with your friends and if

Josh:
you've made it this far you're all caught up go enjoy the weekend it's fourth

Josh:
of july baby have a great weekend happy birthday america happy semi-quincentennial

Josh:
yeah semi-quincentennial year 250.

Ejaaz:
Years old let's go that's

Josh:
Huge that's huge for the gang like what a country look how far we've come in

Josh:
a quarter of a century or no two,

Josh:
Two and a half centuries?

Ejaaz:
Two and a half centuries. What do you call a thousand?

Josh:
Quarter of a...

Ejaaz:
I actually don't know.

Josh:
An eon?

Ejaaz:
That's a question for Fable. For Fable 5.

Josh:
So when we end this, use your cutting edge AI model to go ask,

Josh:
oh, what is a thousand years? How do we say that?

Josh:
Oh my God, we're so cooked. Happy birthday to us. We are so cooked.

Josh:
At least our brains that we offload to are getting smarter. So even though we're

Josh:
not thinking for ourselves.

Ejaaz:
I officially got smarter this week and I'll be smarter until July 7th until

Ejaaz:
we get to usage-based credits at least at this point.

Josh:
And then I'll probably still wind up using the usage-based credits. But yeah, that's it.

Josh:
Have an amazing 4th of July weekend. Hope to celebrate an exciting milestone

Josh:
for us, EJAS. I'm not sure if you are aware, but we are very close to episode

Josh:
number 200 in Limitless.

Ejaaz:
Which is unbelievable.

Josh:
This right now, if you're listening to this, and also this is news to EJAS too,

Josh:
this is episode 199 of Limitless.

Josh:
So we are one episode away from 200. We're going to record that probably next Monday.

Josh:
And uh that'll be a celebration so for the people who have been here since the

Josh:
beginning like seriously thank you so much it's been an amazing journey thank

Josh:
you um 200 in a little over a year is crazy we're recording yeah four episodes a week and.

Ejaaz:
It's been what 65 000 subscribers on youtube and yeah many many more listeners

Ejaaz:
all over the world across spotify apple music wherever you are waiting commenting

Ejaaz:
letting us know your thoughts uh to go from scratch to this has been insane

Ejaaz:
josh i don't know if you have any any

Ejaaz:
thoughts or highlights about this maybe we should talk about this on the 200th episode we

Josh:
Might have to reflect on episode 200 so stay tuned for that one and i have a

Josh:
feeling it might be about etched because i have a lot of cool things i want to talk about

Josh:
when it comes to that company and inference so as always stay tuned don't forget

Josh:
to share this with a friend if you enjoyed if you find someone else that might

Josh:
enjoy we have a newsletter dropping at the same time you are listening to this

Josh:
as well and that's everything so happy birthday america

Josh:
thank you all for watching and we'll see you next week.

Ejaaz:
See you guys