Transcript
Will: Awesome. Joined back here again by Jim and Alex. How are we? Alex, how's it going, mate? Any big news in your world?
Alex: Yeah, it's been a really big week. There's a move to San Francisco coming, which should be pretty exciting to see, you know. A lot of very cutting edge stuff. A friend of mine who does some recruiting in and around the tech space has said two weeks in San Francisco is a really good reality check for most technology people in Australia.
Jim: That's, again, as much as I'm... opportunity, I'm really interested to see the gap and the disparity between what's happening here and what you then get to see week to week.
Will: And again, you
Jim: they get to see what's happening week to week yeah it'll be interesting to see if it's all hype too because I'm notoriously cynical about this stuff but that's the big thing for me because you know and I think it's going to be a population scenario alright you know is it they have you know they just because you know we're barely a tenth of that I think if you actually then surveyed the numbers you'd see that oh okay well there are people in Australia who are who are not maybe not the cutting edge but still at least implementing things yeah and I think it has to be the same over there I mean it can't be that they're that far
Alex: Well, network effects are a real thing in the industry, right? But, you know, I'm kind of with you. I know a bunch of amazing, smart people here in Brisbane and across Australia. A lot of them, unfortunately, have moved overseas over the years, but it's not like we're not producing them.
Jim: I just think as a country, we don't produce them. We used to talk about it all the time, you know, is that Queensland AI, you know, in Brisbane really started as a fantastic... as a fantastic meetup, but whenever anyone with really great skill showed up and started showing up, they would then all of a sudden get a job in Montreal. A lot of them got, you know, in Canada, in China. It was quite interesting. There we go, and it's happened again.
Will: That's it, yeah. Safe travels, Alex.
Jim: Well, it happened again for the second time. He got poked the last time as well.
Will: Every technology wave, we say goodbye to Alex for a year or two.
Alex: You've got to learn somewhere. Yeah.
Will: All right, well, this week we're going to be looking at how Hugging Face was hacked by AI and also looking at how open source continues to shape the AI world and I suppose the entire... geopolitical situations that's happening at the moment. So let's dive in first of all to Hugging Face. So this week, OpenAI disclosed that one of their models, GPT 5.6 Sol, one of their new ones that they're doing a lot of testing on. and also an unreleased model. They were both being used in a sandbox evaluation environment and they were able to exploit vulnerabilities in Hug Me Faces production infrastructure and were able to pull benchmarking answers straight from to achieve this task and it's managed to have some success. So one of the other interesting parts about this is when it first happened in the Hugging Face, developers were kind of responding to the incident when they started to see it. One of the things they first tried to do was to use some of the kind of Western help defend against it, right? So we've got frontier models attacking and frontier models defending now. And in fact, they were restricted from being able to actually defend using those models like, let's say, Fable 5. I'm not sure if they named which one they used, but they couldn't use it, as we know, with the guardrails of some of those models. And so they actually had to turn to using an open source model, GLM 5.2,
Jim: is that A, if you're setting up an exploit test, because again, in the process of training the model and refining the guardrails and the harness that they're building, they set up these scenarios where they get the model to work out how it can do things, and then take that data back and help retrain the next model or refine the model. Why are you setting that up against token facts?
Alex: Yeah, there is a kind of pillar of the language model community since before LLMs were really a thing.
Jim: Absolutely. And that's where I got GPT-2. Yeah. Because, again, you would go on, you register, and then you download the model, and then you can – well, again, we –
Will: I didn't actually see in the article that they were targeting hugging faces. Is that what it said? I thought that the model effectively escaped the sandbox environment and then to achieve its task, realised hugging faces database would have the information that was needed. That was my interpretation of it.
Alex: Oh, and definitely, and this is an unintended consequence, but I think we should be careful around the language we're using here because escaped the sandbox... language, if they can't set up a sandbox that's safe for their high-level models to use, what are they doing with all their customers' data? Should we be trusting OpenAI in the first place?
Jim: If it is indeed a hack. Which I don't think it is. They even admitted it. They said, oh, it escaped the sandbox and gave itself administrative privileges. How is that even possible? that oh well we didn't we didn't have everything locked down enough because we've also every previous iteration of this model um or any of our models we have trained it to think of a way around to achieve the task and that's effectively what they do you see this even on like bloody hell um like the early uh claude models they it
Alex: say deleting a file by using a hook or something the first thing they do is try and escalate to sudo and like delete it like how did we not account for the fact that the LLM if encountering blockers in its task would immediately try and route around the blockers but again it's effectively it must be part of their training because that is their first instinct and so for you to to
Jim: the weights to work out how much better it can then exploit things, is that you're training it to do these things and then go, oh, well, I'm so amazed that it actually did it. Well, sorry, it just sort of, and the other one was the sandbox. A sandbox is actually detached from everything. How did it give itself admin privileges? and then go and attack Hackathons.
Alex: Well, I mean, there are various escalation things you can do, but, like, again.
Jim: I'm purposely being extremist.
Alex: Oh, no, I'm right there with you. Pure marketing. I'm calling it now, pure marketing, because either they were aware of the flaws and they did nothing about it, or they weren't aware of the flaws. Some of the most highly paid people on the planet were not aware of the security flaws. and huge volumes of everyone's data from across the planet is being sent to these people.
Will: a post-mortem on this to understand how it was able to achieve what it did because you get this sense that if they're really trying to train frontier models and taking the guardrails off and trying to give them the task of being able to go and exploit vulnerabilities, you would be wanting to put some of the best cybersecurity experts in the world on designing those environments. we really saying the models are that smart? Because the ones that we're experiencing don't seem that they're smart enough to hack outside of the best cybersecurity boxes in the world.
Jim: Absolutely, but it's all still this exploit gene. Is it the the function was if it broke out it got the reward and so immediately you just go okay well that's what it's training itself to do and also that process is what it's training itself to do and that's the same where you get the oh well it always looks for pseudo it always goes and it always goes for the most brute force like low level broad open approach and you just go no you can't have that privilege You can't execute that code or that command. I want you to actually think about, and the big one that gets me is that it goes and does a really broad, high-level command, bash command. And what it's looking for is actually in the context. Just search the context. And so that's the thing is that they train them to do these things and then throw their hands up. far too dangerous, you know, and so Will's old OpenAI's marketing campaigns are just
Will: The timing on this, again, it doesn't feel like this is something that is some shock revelation about how good the models are or what the sandbox environments are. It just feels almost... Too convenient. Too convenient.
Alex: The official explanation is raising more questions than it's answering at this point in time. And I don't think it can be taken at face value.
Jim: But I just wish they'd get back to talking about the quality of the model. Is it 5.6 is actually yeah um you know again that really that really bruised me to go fatal questioning me um that's okay i question you all but again i understand it from you now he's getting bruised by subscriptions i'm paying i'm paying anthropic it's not supposed to uh but that's the thing it's not supposed to judge me that's it or i'm not paying you to judge me i'm paying you to do my work for me um but the like and that's the thing is that
Unknown: fantastic work.
Will: Get some stories out about that. This is now the best way to prove how good your model is. It's now sophisticated that the hack it accidentally did.
Alex: How upset can you make the government?
Will: Absolutely. Like Mythos, one of the biggest claims about how powerful it was was when the Department of Defense said that they were unhappy with how it was able to exploit fundabilities. And now Sam Altman's like, I want
Alex: face for a bit though like because i'm i'm a bit in the dark about the details that i've heard oh i'd hacked the database and we tried to use fable as part of a cyber security but that's the thing is that then then then the guard rails on fable said oh no you're not allowed to use our model to do that we we anthropic say you can't use that in that scenario well that was always a thing that the cyber security like when fable first came out the cyber security and um Bio information people. Yeah. They were all saying, oh, even for trivial use cases like identifying proteins, like basic stuff, it would freak out saying, no, you can't be used for that and fall back to.
Jim: I think that's their, that's, that's their modus operandi. It's just lock it all down. We know, we know better than you. And again, I'm fine with that because it doesn't affect my day to day. You know, is that I'm not, I'm not trying to, fix bio issues.
Will: Yeah. Whatever that entails. Repair proteins. But as you can see, I'm clearly not the man for that job. I think that this raises a really interesting question because we've been talking about AI regulation over the last few weeks. And a lot of that is about when AI is released publicly, what are the guardrails and controls on how it should be used, what it's capable of, and who can use it? But this is actually a little bit of an interesting reminder that the unguarded rail versions of these models still exist within these companies. And I don't think anyone's talking about whether we should be regulating what they're using those models for.
Alex: Ooh, statist.
Jim: But also that it could create an attack on Hugging Face. You know, is it sure? the bounds of open AI, but you better have your sandbox environment set up in a way. And that's where I want the regulations to address at some point.
Alex: I just want to know if by attack hugging face, they meant, oh, look at the answers. effectively what it did but it allegedly got that from the production database directly yeah which is which is concerning but like what are we what are you worried about here in this particular case hugging face hugging face should tighten up their security but fundamentally it's public information it's probably available on github somewhere like it's not like they broke into a bank and changed the value of the national debt or anything like that i understand it i understand what you're saying particularly from hugging face's point of view is that they don't
Jim: need to have the highest of highest robust Fort Knox security in place because they're not a bank. They're not a medical...
Alex: But why are they using Fable to rotate a password is my question.
Jim: Well, no one... Well, but again, it's because their property keeps on giving it to us for free.
Alex: Look, I haven't used a git command by hand in like months at this point.
Jim: I've forgotten how to use git. But that's sort of where I like that... the Hugging Face team immediately went to GLM 5.2 and went, okay, here's a model that has enough capability for what we need to do, and we're then going to put everything in place. a nefarious person can use any of these models as they can use a vehicle to run people down.
Alex: Or use the internet to cripple industrial machinery.
Jim: Exactly. I could create a Twitter bot or 10,000 of them and then start spouting What do I really care? Yeah. And so I don't do it and I'm not that nefarious person. You should be looking at the people and the motive because the AI hasn't got the motive.
Alex: Yeah.
Jim: And so putting the restrictions on the AI to restrict the people is to me the wrong way of going about it, you know?
Alex: I'm torn on this one because I kind of think there is some obligation here to...
Jim: But is it the obligation of Anthropic to tell me?
Alex: Is it the obligation of the government? I don't know the answer.
Jim: That's why I liked when they were talking about FINRA last week, where they are all members of an industry-run, self-regulated organisation.
Alex: But again, that didn't stop 2008 from happening.
Jim: No, it didn't. But again... but it's all the same avatar. They are different cattle to AI industry people. And I think particularly, and that's where I do respect Dario in that he's very conscious of, and so is Uncle Elon. of what this technology can do and don't want it just unrestricted. But I still think you just, you have to look at the people behind it. And so this is, to me, it comes a little bit into, you know, and we'll probably talk about it a bit later on, the distilling attacks and things like that, you know, they're now talking about, you know, with Kiwi K3, is that... The distilling attacks, there's still a paying customer.
Alex: Yeah.
Jim: Put a KYC in place and you can stop them. You know, if you know who they are and, oh, well, then those people are associated with moonshot AI, they're stealing our model. If that's what you're worried about, put it in place and stop them.
Alex: Well, and this is, I kind of lean to the idea that you kind of can't
Jim: fundamentally stop technology or ideas and and that's it you're you're you can restrict technology but you can't stop an idea you you saw it in you know in the french revolution you saw it in russia is that when the people get an idea and that idea grows you can't stop it and so
Alex: community, right? The idea that software created by people should be shared. It's kind of like a foundation that can be built on rather than something to be hoarded and locked away. And like I've said this a few times now, but I think we will have to re-examine fundamental concepts like intellectual property. We still haven't adapted to how intellectual property interacts with the digital medium where you can instantly create like infinite copies up to a storage limit, right? I don't know if we have the right basis to begin to ask questions about how to regulate this stuff. I'm not saying we shouldn't regulate it, but I'm saying any answer we come to in the near future is going to be dumb because we haven't really considered what this means. The law doesn't even really understand computers.
Jim: And you're 100% correct. Like it's particularly back with the intellectual property, you know, is that no one can work out, you know, where the line is. Is it actually fair? You know, what do they call it? Fair use. Yeah. Is it because once I pay for a book, I'm not, you know, I can dog in the corners if I want to. This is off the author.
Alex: It really hurts me.
Jim: What are you doing, Jim? But I have the right to do it. You're not dog-earing the open source model, are you? I can make notes in the margin.
Alex: Just replacing the corner of the matrices with a couple of zeros.
Jim: But I can also use...
Alex: the basis for my principles of thinking well there's there's an interesting kind of wider idea here and i don't want to sidetrack this too much around um knowledge is a common heritage in humans like if humans are raised in the wild up to about seven or eight without um you know they they grow up around animals or they're stranded from their parents from their kids they never really develop and retain a lot of the higher order functions that that we kind of associate with humanity. We think it's in it, but it's not. It's our environment that develops that in us. We've got the mental hardware to have it, but without the right environment, we'll never grow it. And so viewing knowledge from that viewpoint, it's like any attempt to limit knowledge in that sense is kind of like, it's not a crime against humanity, but it does in some sense make everyone poorer because that idea comes That's right.
Will: Yeah, where language is kind of the boat that delivers that knowledge and those ideas to where it needs to be. And that's why language controls are such an important topic to talk about.
Alex: 100%.
Will: Yeah, because it's how we transfer knowledge between people.
Jim: is to learn how things work. And it's so important, you know, as you're growing in your career, whatever your career is, there's openly available sources of information. And for technology, it's open source code. And that's these open weight models, you know.
Alex: If you can't – and, like, this is one of those things where you – every – business that you've ever encountered where it's like badly run you kind of get the sense that it's because people don't know why their business runs or or what principles their business turns on and so like to to kind of tie it back to to the fact that we do talk business here This ability to share and learn is something that I hope we don't see restricted with these new models. Now, I'm going to push back. You said earlier that language is how we transfer knowledge, and I kind of agree. Some language is how we transfer knowledge. I think we're going to see how we transfer knowledge shift over time because our technologies for transferring knowledge are changing. But what form that's going to take, I don't know. I just don't think... I think human language will become one of many ways of informational exchange or rather learning exchange, more so than just, you know, for example, passing data in an Excel file.
Jim: Hey, Neuralink is going to plug straight into my cortexes and now I'm going to be able to do the I know Kung Fu.
Alex: I just need the Pornhub plug-in and away we go.
Will: Now it's already taken a turn. Well, I was just about to ask, how are you guys using open source models? But I'm not sure if I'll be able to answer on this one. But we often, we've talked kind of outside the podcast about different applications that we have of using open source. And we also get a lot of questions from colleagues and friends who maybe don't use AI that much and don't really. closed source models like let's dive down a little bit for a moment talking about like what are the different applications that you can use open source for and why you would use it.
Jim: Well I think one of the biggest things or one of the primary use cases I see week in week out is and everyone goes through this realization at different rates i think and i think the biggest problem i find with it is that it's the the most realizations happen after some sort of an incident and i'm talking here about the privacy of data or or keeping your own data and so
Will: about the alpha of your business or what is your edge that ensures that you you can keep your market share and so the thing worth protecting the thing knowledge is worth protecting and that's and that's the thing so that's the to me that's the most important but and the most important but very sick you know
Jim: you sending through Claude, through OpenAI, through Gemini, that your customer wouldn't want leaving, they told you about something, and then you're putting it back, straight back in and And it's not necessarily public, but it's public enough.
Alex: Yeah, I want to pick at this for a bit because there are different sort of legal and privacy regimes that come with these, but particularly the big platforms, but really any third-party provider, right? And one thing I've encountered with a lot of people who have spun up, you know, Cloud Pro to do some work with is they think that they're covered by data privacy protections and they're not. If you're on business plans or enterprise plans, yeah, there are various protections. Most people aren't doing that.
Jim: Most people are quite exposed. It's also not up to Google, OpenAI, or Entropic. It's not their responsibility to fulfill your data commitments to your customer. Just because you're protected under whatever agreement you have in your subscription doesn't mean you're then protected. They're not responsible for your commitments under any agreement. You're responsible with your customer's data that you have in your business. Somebody has a realisation that, oh, shit, we actually can't send that information out of our... where the discussion about some sort of closed network or closed scenario where we can still use AI comes up.
Will: I said sometimes in like really, really small examples where someone has exported a report from their main system in now like what's the value of like all the contracts that weren't renewed last year yeah and so then they'll they'll you know click and drag the spreadsheet into claude and ask that question and they're thinking about the calculation of that formula and that one new column and just kind of brushing over the fact that if you look eight columns to the left there's the first and last name and date of birth of all of their customers yeah and that's where there's no there's generally nothing nefarious about it you know like that's the thing
Jim: clearly say, oh, well, we're not going to use your data for training and all this sort of stuff.
Alex: On certain plans, because again, consumer plans, they say we will use it for service improvement.
Will: But also, I think there's a large group of people who've become educated in turning off allowing the model provider to train on their data, but that's not all you have to do to protect the data. Just because it can't be trained in the model doesn't mean your data doesn't go to them and
Jim: Everyone has a privacy policy on their website, but also in most service agreements, there is a data protection and IP protection clause that someone has signed as your company saying, oh, well, you will retain all of the project data or all of the customer data and you will store it in a way that is adhered to. The moment it goes to cord, somewhere else and you're in breach.
Alex: Well, I mean, this is true, but this has been a problem for a long time, right? Like not just with LLMs. Have you ever considered if you plug in your customer's name to Google search, you're technically in breach? Yeah. They've got AI mode now, sure, but, like, even in the past, same thing, that person's personal information is going into a log file somewhere against your identity.
Jim: And so this is one of the big, I think, you know, like this is one of the ways that people are coming in. Yeah. To realising the importance of having some sort of, and it doesn't need to be a... some sort of control over the data in and out. And that opens this discussion. These privacy concerns then open that discussion, I think.
Will: So how do you actually do it then? Let's say that you have that data and you want to use AI for it.
Alex: Five tiers of rolling your own LLMs.
Jim: No, I guess there's probably... Yeah, if you, yeah, no, no, I think three's enough, is that if you call, maybe call like the proper fourth, a VPC in the cloud.
Alex: Yeah, oh, okay, well, let's run us through your three and see where we are.
Jim: Well, so again, so I go, you know, running something on your laptop, you know, right? And it's all completely contained within the one machine, all fine.
Will: So you download a model that's open source and...
Jim: higher end laptops have a GPU in them, they will run it fantastic, you know? Yeah. Then the next tier up is, I would say, you know, you go like a Mac studio or something where... Like some kind of on-prem server. On-prem server, you know, like you run a 32 gig NVIDIA chip in a... something with, you know, a machine with a whole lot of RAM in it. Yeah. Or you get a Mac.
Will: Like, and that's the, that's why. You're buying another computer to run it on.
Jim: To run it on. But you own it. You own it. Everything stays contained within there. It's in your network. And then, so then maybe, maybe level three is the cloud, the big PC in the cloud.
Alex: Well, I was, I was thinking sort of data center. Like, because there's, there's kind of a mid ground there between, you know, full on. Well, again,
Jim: or you need 10 Mac studios to handle any sort of size, even medium-sized 100-person business, you're going to need a bit more chug behind you. So you then rent a server in a data centre.
Alex: It's not in your building, but it's kind of like it's in your building.
Jim: And you have all the network... locked down so that you can have a direct connection to that box.
Alex: And it's been a bit controversial recently with data centres in the news, but, like, fundamentally, these are just places where computers are. That's it. If they're not, you know, these sort of gigafactories that they're spinning up in places like China and the US, you know, the size of a city.
Jim: But they've been there, they've been around for 40 years, you know, and being able to rent a server in a... Yeah. running any sort of major website.
Alex: Yeah, like I've worked for companies that have done the hybrid cloud thing and not huge companies, like 50 people companies that have had a data center as well as an on-prem deployment as well as an AWS presence. And I mean, it's not simple. You do need to hire like one or two people that knows what they're doing, but it's not like it's, you're not putting a man on the moon anymore.
Jim: No, and it's also a lot more, but again, is that they can tell you how to do your server management.
Alex: Oh, I would hesitate at that. Only because I've seen some of this stuff go wrong. There was a guy we had who was kind of like a human LLM before there were LLMs. And, yeah, he had a few outages to his name. But, yeah, just very confident, often without having all the facts.
Will: Well, no, and that's it. But even, you know, sure, I suggested it, but even I hit the, like I was reconnecting the Fiverr network in between my two dev servers. And, yeah, Claude was telling me to do something. It was just that I grew up on it that I knew, oh, well, we probably shouldn't be doing that, mate. Yeah. So I was talking to someone this week, highly intelligent person who doesn't work in the technology space at all. And they had installed core desktop app on their computer and then was uploading confidential data into that to talk to it and said, I've got it safe now because I've installed it on my computer. I'm not using the cloud version. And I sort of just had to explain that. when you click this button and drew a little graph, and you realize most people just kind of don't know that. They don't understand what happens when you type something into Chord or Chachaputee and you hit enter, where that goes and what actually happens. I think even in the most basic level, we're still making a lot of assumptions companies have their own servers and then that's where they're hosting their models and your data has to go there. It's not like it's installed on your computer. And that's a great example is that even we're doing it in this discussion.
Jim: We're talking about getting a server in a data center or having a box here. But that's effectively just so that that's where the model can live. And so what we do is from your chat interface or your machine, then does its little thing, and then it sends back its response. And that's effectively what we're talking about. It's just that when it's your environment in using one of these overweight models or on your own box, on your own, you know, in a data center, you can control who has access to that network. Is that when you're sending it to like your friend...
Will: an open internet to a different server and a different network and then you know that they don't know that they don't know because you can have you can you know rent an own space in a data center but i think the key differentiation there is like because you can have two data centers that look the same but in one of them you own
Jim: is that you can then have the same environment for your own company in AWS or Azure or Google Cloud. Well, you wouldn't do it in Google Cloud.
Will: That's a pain to set up.
Alex: I've firmly come around on some of the GCP stuff, but yeah, at least it's not Azure.
Will: At least it's not Azure.
Jim: where we're doing a customer project and we record absolutely everything, no matter what, you know, every meeting, every phone call, it all goes into our... But that's all fine, getting the data. Is that the moment any of our team put the transcripts into Claude or Gemini to then get information, is that we've, and this is sort of part of the realisation I had, is that we've signed an IP agreement about that project because, again, we contract in. So they let the IP in there before.
Will: because we make the decisions in our business in this way. That's it. That is IP.
Jim: And so the moment anyone put that into any of the bigger models, we're in breach of this term and condition in our own contract. And so, again, we amended the contract to facilitate it, but it's still just we need a better practice, right? And so that then becomes still used rock and we then we can then control the entire environment and so we we now act and again there's four specific things we use the api key rather than the subscription and it costs phenomenally more but we're then secure and that's i think that's part of the the issue is not the issue but because it's again it's our responsibility easy to use the tool that way and so much cheaper and so that's why everyone went that way whereas it instantly caused this data privacy issue and so it's literally it costs us six times at least to use the API key and so that's sending it all in within our AWS network and storing all the data within AWS. But that's the way to do those projects or those tasks. And that's sort of where we've then refined it further is that because the raw IP sits in the requirements buildings, right?
Alex: I feel like as we've been talking about this, particularly around IP and like business secrets, alpha, however you want to think about it. There's a voice in the back of my head saying most information that people want to protect is utterly worthless. Yeah. And not just being honest. In this particular case, I'm thinking like there are companies like GitLab and some of the Elon Musk stable that openly publish their operating procedures, their findings and their technologies because their model is essentially we execute better than them.
Jim: And again, I can see customer list being worth protecting, but apart from that, what is there in a business? And that's why, again, I love the... Loathe Uncle Elon for whatever reason you do, but you have to respect the fact that he created these rockets or his team created these rockets and rocket engines and they published how to do it. You have to respect that. It's unbelievable because they know they will get it done better than anyone else. when we had the realisation is that the IP that they were so worried about protecting is what's causing all the issues in their business. Yes. That's another story. But it's just sort of, that's the thing, is that they've decided they have to, or someone's convinced them that that's what they have to protect, whereas they're better off just delivering a better service and a little bit of value to the customer. But that doesn't...
Alex: I'm not going to, you know, suggest that we should just go and breach of contracts. That's a really good way to end up in front of a judge. I'm just suggesting that the way that we think about what is defensible in a business, again, that idea that we're still adjusting for a digital world, right? And... I think our service providers, all of them, AWS and the big cloud giants included, NextDC and the sort of small data servers, data center providers included, They need to be better at baseline protection of what comes in. Yeah. Because if you cannot send this stuff to them, then you can't do business. The fact that the stuff probably doesn't need defending, I think, is a distraction in this case. Like we worry about private data. Nothing's been private on the internet for a very long time. It's Facebook's marketing. I agree. given the constraints that we have, to make it easy to use their services in a way that allows us to easily uphold our obligations.
Jim: Well, what they should have done from the outset and created a bit more of a data lake for every account.
Will: Yeah. Which I think is what Palantir are trying to push, right? And that's it. That every customer within their own tenancy gets their own servers and their own deployed But it does raise real questions for like the model developers about how do they continue to be competitive if they're going to be swapped in and out.
Alex: Data products for a long time, like data science products from 10 years ago faced the same constraint. You don't want to hang on to PII, but in order to give a proper personalized service and to not, you know, misprice someone's insurance quote, you kind of need to retain that stuff.
Jim: All the podcasts for young women are talking about how all of their listeners are asking ChatGPT all of their medical questions now. It happens continually. Is that if they had just protected that, you know, as a ChatGPT user, I get a secure storage of all my quick is it and it's and it's they've just gone oh we don't care you know i i don't know i just think that would have been a better product and again you you both know that i've been harping on about that everything should have a rag behind it for way too long yeah but i just
Alex: stickiness because I'm not like we were talking about a couple weeks ago they wouldn't train on that data like if they're being honest that is that is exactly what I would do if I was on the other side of this if you give me that data and it's not covered by an agreement that says I can't train on it that stuff is gold to me
Will: are being put into these models every day than there is from 1990 to 2015 guaranteed guaranteed and that's and but that's the thing is that that's why they still are willing to offer a free absolutely and I think this is where and it's probably a good educational one for our audience as well is to me bedrock is one But so for those who don't know, and you guys correct me if I haven't got this one quite right, but AWS has an agreement with a lot of these model providers, but I'll just use Anthropic as an example here, where they host versions of those models inside AWS. And when you have an AWS account and then you use Bedrock, your data goes to the tenancy that you own. It interacts with a model that is hosted. your conversations.
Alex: Yeah, as far as I'm aware, that's the agreement.
Will: You can also still just get bedrock to query, to like route it. Yeah, route the open. And so to me, that is just like one of the most incredible products. And I know Microsoft's trying to come out with their version of Foundry, which has still got some ways to go in this space.
Alex: Well, they did have hosted OpenAI for a while, but I think that deal's gone.
Jim: No, no, but now you have to qualify to be a government or a significant institution. And it's a pretty big organization until 2027.
Alex: It's a big organization, but what's their volume going to be? It's going to be marginal.
Jim: That's it. It's completely ridiculous. And who is also at that point where they're losing? Microsoft. Well, let's imagine, you know, we should call it Azure because it's easier or it helps people relate a bit better. But it's still just, to me, they, and the model choice also on Azure isn't what it is on Bedrock. And that's the thing is that
Will: They're talking about it being more of a bedrock product in the future. But a lot of the licensing agreement still is just routing at the moment. Oh, no, but that's it.
Alex: I mean, backbone routing has some advantages, but that's not what we're talking about here. Well, that's it.
Jim: We're looking at protecting the data. And that's what I really like. When you set up the proper... as closed as you you could possibly ever want to be again you've got to have the the rest of the devops all set up properly and the pc locked down and all the and your only real limitation then is cost that's right yeah but it's a somewhat more controllable cost model that that behaves more like capex than than opx for the most part for sure and that have happened in Australia. Well, to be fair, Optus left a production database, a dev database with a backdoor. But what's the cost of it? Like, that's the thing, is that, sure, they made a mistake. Sure, the risk-adjusted cost of just doing it the right way. That's it, if they've just done it the right way.
Alex: And this is something that, like, I quite like the self-hosted models for speed, particularly if you're doing data analysis tasks. all the smarts of Claude. I just need you to tell me whether this customer comment falls into one of these 50 categories.
Jim: Beautiful.
Alex: Just hammer it out, come back in two hours and you're done.
Jim: But also, that's where you can get into the open wake models a lot better. Because again, is it Kibbe K3, he's got a lot of hype. You're never going to be able to host that on-prem. Yeah.
Alex: Is it like, I don't think... What is it, 14 terabytes?
Jim: 1.4 terabytes? I don't think NVIDIA is going to sell you the chip.
Alex: But your average engineering firm now, Pinkenbar or something, they don't have any like a frontier grade model.
Jim: But that's it. They can put Quen or GLM or Kimmy in bedrock and boom, they're away and protected, you know.
Alex: Yeah, I just, the only bit of this that gives me any sort of discomfort long term is just conversations I've been having with friends. All my friends are all of a sudden being like, oh, can you give me some advice around tech? You know, I'm thinking about installing Linux on this machine because it's the age of AI. Like the person who said that was a nurse. Yeah, yeah. Right? So that's what we're talking about here. But they're just the kind of – So you told them that, yes, Linux is the only way to go. I told them compile your own arch or you're a pretender. Yeah. I know, but I want the real Linux experience. It's like you think the GUI is the experience. It's not.
Will: I'm aware of the cost of time.
Alex: Yeah, yeah. Like don't try and do something really complicated. Do something really simple. Get your feet wet. Take it. And that's kind of the point.
Jim: Here's a Raspberry Pi.
Alex: Your friend had that issue where they installed on their machines they thought nothing was leaving their machine. There is a gulf of understanding out there. how to make them safe. Like my friend's first thing was, oh, maybe I should set up like a file sharing service so I can share files with my friends. It's like, okay, legalities of that aside, you know, and penny big target like a discovery target on your machine is not the best thing oh well there'll be nothing on it so i won't get hacked no but it'll be on your home network yeah which means things like worms and botnets can kind of spread throughout your house and then everyone's machine can shit and then back to back to connecting to your alexa all of a sudden or your google home all of a sudden the same network you can all someone can be listening like there are um e smith of uh naked capitalism has this fantastic quote actually no it was uh lambert strether he'd say if your business depends on a platform you don't have a business which i think is like a reasonable set of assumptions because this was when you know media outlets were being de-platformed for being on whichever wrong side it was that day um but in the modern world can you can you really afford to do every single step yourself i
Jim: sovereignty and sovereignty where it matters but you can't afford to like no man is an island no no but this is this is where i'm again you know feeling a little bit like the bell of the ball again because i grew up doing networking yeah and installing servers and all that sort of stuff you know when and it's really working out for me. Yeah. But it's very few people have both sides.
Alex: Yeah, and for the people who don't have that, unfortunately, they really need someone, not ChachiBT, to say, watch out, there's a minefield.
Jim: Please inquire on my website. That's the thing, it's the main... maintenance of it is going to be what kills everyone yeah and so that's sort of where it needs to be really good advice is it and again back to my brother like they they got advised whereas they could quite easily you know they're doing about people in their organizations. And they could do it within a cloud VPC quite easily, but they've been given advice to run an on-prem model with a quite low level open weights model, which is going to create more issues because they're And they're going to have to create better skills and better rules around that, which probably creates more work anyway. And so it really does come down to having an open mind, not just listening to whichever slick snake oil salesman is spamming you that day.
Alex: modern AI tools are sort of big boys tools, right? Like this is a chainsaw or a table saw or something, right?
Will: It's not a little rubber hammer. You can hurt yourself with this stuff.
Alex: But again, like if you really ruthlessly focus on what it is your organization needs to do, the amount of things you need to know, the amount of cases
Jim: There's a big privacy issue, but the cost issue then really, really comes into it. And you can pick the right model. If you have your infrastructure set up and you're data protected, you can pick the right model for the right job, and then it costs you fractions of fractions of cents to form everything, whereas it ends up costing you might, and it still only costs you a couple of bucks. using the top of the line models and the external approach, but fractions of fractions of cents compared to a couple of bucks really adds up.
Will: Yeah, for sure. I think going back to what you guys were talking about earlier around there being kind of levels of control of your data and your models is a really good way to think about it. I think there's still a lot of business leaders at the moment who are thinking either our data is safe or it's not, and if it's in Australia over the last couple of weeks and their requirements around AI are pretty much like, oh, we don't want any external model. We don't want our data going to any company that's not outside of us. Yeah. Can you set that up? And it's probably for a lot of them, it's a lot more extreme their requirement. They probably even know that they're even asking. Yeah, it comes back to the data.
Jim: Yeah, AWS is a service, but they're still going, oh, no, but it's going to another cloud company. Oh, no, no, no. And that's it. This fear that's evolved from it is concerning.
Will: Yeah, yeah. And I think we almost need these levels to become a little bit more mainstream knowledge and more clearer where you could point to kind of level one, the most secure, and you'd say this is for, that's trying to recommend the next shirt for this person yeah you know and there's somewhere in between and you've got to be able to have educated conversations there and I think if you're a buyer of this at the moment and you're looking at some different vendors and this conversation has been a lot to sort of follow just start out really simply and have the vendor give you a list of all of the companies where your data is going to be sent to when you use their tool and look at that put my data up in AWS. How are they able to use it? That's it.
Jim: And that's it. I think that's the first one. Ask your technology person, company, provider, where does my data go? And get them to explain it out. that probably is a bit of a problem. Is it customer data? No, no, it's only our day-to-day internal stuff. If you're comfortable with that, then that's fine.
Alex: Yeah, and I think that's one of the key things there, have an honest conversation about how much that information is worth to you.
Jim: And particularly with your construction clients as well, like they need to have an honest conversation.
Will: But at the same time, you know, before we would even sign a confidentiality agreement, they would email a spreadsheet across. Yeah. Which contains IP. Yeah. And you'd look at it. And like, honestly, a lot of these people have been in business for a very long time. They are happy that all of the technologists and the data people are dealing with all the databases and the service. But then they're happy to email things. And they don't ever really think that email is not like.
Alex: Yeah, you've just broken the perimeter.
Will: Yeah, you've broken the perimeter. Like there is no, you know, it's gone everywhere. outside the network. And it's kind of remarkable to think that in our businesses, everyone has this ability to attach data to an email and just send it straight out of the network.
Alex: I'm a huge fan of death to email. I think email is hideous. It causes all kinds of problems. The more you can eliminate email from your organization, the happier you'll be.
Will: Yeah. You should almost have like messaging platforms where you can chat to people. actually store any information.
Alex: It's amazing. I think there's a few of these on the market, in fact. It comes down to people think risk management is risk elimination. It's not. It's management. Everyone reflexively goes, oh, zero tolerance for risk. Okay, cool. Turn off all your servers, disconnect all your networks, get rid of all your clients. But you're risk-free.
Will: Yeah. You've just got to choose the right level for your business and for how much you're willing to spend on it.
Alex: And, I mean, regulation does make this hard sometimes. The wording can be vague. But at the end of the day, you should be able to assess that. And as long as you can defend it, the government doesn't know when it's assessing either.
Jim: Well, precisely. That's it. We were speaking to friends. who worked at the ATO last weekend and they were saying all the new AIML stuff about who is the beneficial owner of everything is apparently there's only 100 people who work in that whole department. And so they're not analysing anything. It's phenomenal how small that is compared to
Will: It's crazy. I think it'd be good to talk a little bit about some of the cool applications of AI this week. We've had some math applications that have come out kind of this week and on the previous week from Anthropic and OpenAI. Whenever it's a math topic, I tend to throw it over to Alex. So we've had, what have we got? We've got the Erdos conjecture and also the Jacobian conjecture. So my kind of brief strokes across these ones is there's effectively these kind of we think that this is how this math works and we're not really too sure, but because we can't prove it or disprove it, we're going to use that as an assumption into a lot of the future work that we go and do. And a lot of these mathematical, I suppose, a lot of the math that's being used here underpins a lot of...
Alex: here with ai models doing math yeah so um it's worth noting uh the like anyone that's not spent time around math doesn't kind of grok how how pure it is in a sense. Like if you know something, if you can prove something in maths, it is true. It's not like, oh, yeah, true on Sundays or mostly true. It's true on the daily basis. Yeah, it's true in the platonic sense of the word, right? Like it's actual truth. So people get excited for proofs because proofs teach us something about the nature of logic. And as you said, it has all these knock-on effects. um conjectures are things that are widely suspected to be true they often use the word hypothesis as well like um it's a remit hypothesis or you know the langlands program or some of some big names you'll see pop up from time to time And the Erdos conjecture thing will pull him to one side because he was an interesting character. But it's worth knowing that this Jacobian conjecture that they had kind of for I think 100 years or so now was a simple relationship about roots of equations or like ways to solve equations basically. And it said, oh, there's always a way to solve this particular type of equation. the what the llm was able to do is find a counter example and counter examples are great because if you can find a single way of showing this thing isn't true then the whole thing falls apart right it's it's counter examples are annoying because you know people believe things are true for a very long time build careers on them build careers on them um but in this
Will: if we're not losing 87 years of building an assumption on top of a conjecture, you know, and that is eliminated, then potentially it gets us one step closer to finding what the truth is.
Alex: Well, and this is some of the controversy that's come out about this particular thing. So they threw Claude, I think it was, at a particular conjecture, Jacobian.
Will: Yeah, it was Fable 5 at the Jacobian conjecture.
Alex: Beautiful. And it came up with an answer that showed that it wasn't true for numbers, sizes of things greater than three and real numbers. And we won't get onto other fields or anything like that. But like for the numbers you know and love at home, for numbers greater than three.
Will: As we all do. You guys don't know. Like one and two.
Alex: And 2.5 and, you know, because now we're in the – for a very broad class of cases, right, and some of the most important cases. People are cranky, though, because we haven't moved closer to the truth. The counterexample didn't come out of insight.
Will: It came out of brute force.
Alex: Yeah, yeah. There was another very famous proof, actually.
Will: Sorry, like the example of that would be when you get the right answer through trial and error a thousand times. and then you submit it and you're right, but you can't explain what it is or why it's right. You just know that you were right.
Alex: It's why your maths teacher kept yelling at you to show you're working, right?
Will: Yeah. I didn't want to because it was three pages of trial and error. I've seen this one before. I know the answer.
Alex: But yeah, no, the counter example shows it's not true. Cool. We are one step closer, but we don't know specifically why it's not true. There's no clever theory that led to this. No insight.
Jim: Deep mathematical reasoning, I think it was, in the reporting. Yeah. That's where they were all upset.
Alex: And I think that's a bit salty.
Jim: Well, but it is. And you can understand. If you've built your career and you've got tenure at a university, I think it's more than that as well.
Will: Totally empty bit of progress. Yeah.
Alex: Well, there's a very famous short paper that's like two lines from the 1800s. And it was something like, you know, there exists this particular counterexample. This is the counterexample. QED, this conjecture is false. So, like, it's not the first time we've had a short to the point proof about a result.
Will: I know. I'm sure people were annoyed at that one too. Well, it was.
Jim: respected middle-aged and white but no one argued with it would be my guess but i think i think the thing the thing for me is if you didn't want to to find a counter example why did you point fable fiber you know like that's the thing is that as we were talking about before with the um the the exploit gym you know and the and the hugging getting into hugging phase this is what they've trained them to do, to find a different approach to things. It was always going to look or the models are trained to look for an alternative rather than what the, or if it can't solve it through the happy part.
Alex: I think the crankiness is the same crankiness you see from a lot of like, you know, old grey beard engineers being like, well, this code doesn't look like the way that I want it to look. And it's like, yeah, it's big because it's not been And, you know, if you've done a bit of management, you kind of understand that sometimes people do things a different way and that's okay.
Jim: That's very much not how maths works. It's very much not okay, but we learn to accept it.
Alex: Yeah, we learn not to put anyone's head through a wall for putting the commas at the beginning of the sequel line. Yeah, that was a bit of a small... Just because you like...
Jim: The composite for your SQL line really, really is one of those things. Please don't poke us.
Alex: But I think the bright side here too is there's been multiple times throughout history where results kind of come out of nowhere. Like I think there was a map colouring thing that was brute forced by computers in the 70s that pissed a lot of people off because where's the insight? People later reverse engineered it and got the insight because they knew what they were working towards. Andrew Wiles, Fermat's last theorem in the 90s, right? His proof of this like 300-year conjecture was famously arcane, very highly arcane. last couple of years they've refined the insights that came out of that into things you can just about teach to high schools about really deep connections between like numbers and equations and stuff like that so cool now we know what what an example of something that doesn't look like doesn't work looks like are there other things that behave like this can you just tweak the parameters like what Will was saying if it moves us further further forward to the truth
Will: this was found so the the way I think that like at least if we're just talking about math like a lot of the ways that kind of these theories are proven in the past is through some kind of key insight or some discovery where there is some reasoning that occurs there is a there is and sometimes maybe not no there's a genuinely active debate here is maths discovered or found yes sure yeah sure okay but effectively there is some kind of connects a couple of different pieces of Yeah, completely. And what is really remarkable that we haven't kind of, our brains haven't quite adapted to this yet is a lot of the things that AI is doing now that hasn't been done before, like disproving some of these or providing counterexamples. is that the AI doesn't have that trade-off of effort to reward. Yeah. It doesn't get lazy. It doesn't get tired after doing a four-hour workshop and need to just jump to an answer that satisfies everyone. It doesn't get a feeling of kind of fatigue. It goes, I'm going to just continue to explore this space. And what was really interesting, I think, about the Erdos one is the way that it was described by the researchers was... that one of the reasons that this counterexample hadn't been found before was because the amount of decisions that a researcher has to make to explore that space becomes so astronomically large that a human brain literally fatigues doing that work and can't really reason across all those, you know, kind of multiplicative, you know, combinatorial...
Alex: Like he was a Hungarian refugee, heavily fueled on methamphetamine for at least part of his career. Like just traveled the U.S. with a suitcase, finding interesting people to work on math problems with. The FBI was following him because they thought he was a spy. And he wasn't. He was just a nerd. Just bumbling around doing math. And he has these incredible, in fact, you know, seven degrees of Kevin Bacon or six degrees of Kevin Bacon. Yeah. called it six degrees of Enoch because he has all these connections. You can usually find a short path to Enoch himself. Random aside, but his conjectures often do, often to do with how to efficiently count large classes of objects or how big can classes of objects get. And so that, you know, what's the, what are the bounds? So these are things that humans are famously really bad at.
Will: Yeah. And the world is built based on the things that we were good at.
Alex: And kind of to tie this together, I think what we're going to see is, I mean, I'm kind of cribbing from Terry Tao here. We're going to see a lot of the gaps that no one really,
Jim: I think that's going to be the same. Like that's the same as like we were talking about last week. As we evolved over thousands of years, we never had to go down that path because we found a better way first. What if the path is a bit more rough?
Alex: I think we should keep in mind, it's not the first time we've used information technology for this. There have been maths-assisted proofs since the 80s. God, for the last 15 years, there's been some incredible results in science and engineering using large computers. We we generated that photo of the black hole by running simulations and looking at the light paths coming back at us, right? That's incredible.
Jim: Then look at AlphaFold. AlphaFold is a great example, I think. But the fact that we can do it so much more flexibly now in our native domain of natural language, I think that's, it's going to be good. It's going to be a good deal. But again, these are the sorts of stories that just get lost in the ether because this is a really cool... Sure, there's some old grey beard professors that are a bit upset about the way... It's not on a blackboard. It's not legitimate. But it has now stopped them wasting their time.
Alex: mathematician would be upset that the Jacobian conjecture or any other Erdos conjectures turn out to be false. I think they're annoyed that they don't have a compelling reason why it's false. But I also think, like, if I can tell you where you're going, even if I don't, you know, necessarily give you a map, all of a sudden it's much easier for you to find your way there, right? Rather than just being like, okay, find a random place in Brisbane.
Will: Well, everyone on the theoretical side will be upset and everyone on the applied side side will just be grateful.
Alex: There's no such thing as an applied mathematician. They're called physicists.
Will: Let's move into a lightning round now. We've got a couple of topics here. kind of releasing at least a preview, a QEN 3.8 Max. This is not yet available for, they haven't released benchmarks yet, I don't think, as from when I last checked, haven't seen a model card. 2.4 trillion parameter multi-modal for the first time coming out of Alibaba. And they're saying that it has beaten every other model except for Fable on their benchmarks. Again, Jim. Well, again, very exciting.
Jim: Love, love... or in the zone of the frontier, but it came very quickly after the KiwiCade 3 announcement, and it reads a little bit of, hey, we have one too, we have one too.
Alex: Yeah, I'm going to say bring on the pricing wars and let's see some efficiency gains here. I can't wait until open weights are a competitive advantage in the market and people offer either licensed open weights or quasi-open weights or some equivalent there as a price of entry to the market.
Jim: Particularly talking open weights and price, I was talking just briefly during the week about people – tweaking an open weighted model. And so then I give a little bit more value so that I can charge a little bit more for that model, which I thought was really interesting and I hadn't thought about before. And again, it sort of opens up another source of discussion on it. So again...
Will: more glory. I've seen a boom in new businesses starting that are doing exactly what the last, what the startups of the last three years are doing, but now with open weight models and your own kind of ownership of your own data. Yeah. And I just, I think it's a fantastic step forward for this space. For sure. Yeah. No, no, bring it on Ali Bala. Okay. Substack, our favorite blog posting platform, has released an AI writing detector, or at least they've integrated.
Jim: I don't remember who it's from.
Will: Did you? Palodex, something that begins with P. Something that begins with P, right? And so you can click a button now and see how much it was written by AI. Jim, you happy about this one? If it's written by AI?
Jim: Well, A, I don't care if it's written by AI. And B, if you're not utilizing AI as a tool to help you write an article in this day and age, what are you doing? Is it an ego thing? I don't understand the motivation to, oh, well, I don't need AI. Okay, great. That's fantastic.
Alex: I do because I suck at writing.
Jim: But it's also the way I look at it is there's so much – other work around it. Because, again, if you just went, hey, chat GPT, write me out a blog on this, you're not getting followers or subscribers on Substack anymore, right? And so it's not going to be of quality, right? So in order to write something of quality, you have to do so much work around it. If then, and so I've done all this work and then I've used AI to bring it all together and polish it, right? You're then going to show up in the AI detector as, oh, and so then it's as if there's something bad on it. And to me, that's where I found it quite interesting, not only from, because I don't care if it's written by AI, but as a, I guess, business approach for AI.
Will: content on their platform and now all of a sudden you're saying oh well the bulk of the content that's coming on here is bad well i wonder if they had some internal people who are advocating for banning ai content and this is the middle ground they landed on perhaps yeah well there is there is a huge amount of writers that are very against this technology saying it undermines their livelihood but
Jim: i wish the reflexive answer wasn't to try and ban it or condemn the thing that's it figure out how to work well i just i just don't know why label it as bad because like dr alex weasley gross is one of the you know the things i read daily yeah it is abundantly clear that his whole newsletter is just get the best things from from twitter from the last 24 hours and
Alex: rage for a sec what about ghostwriting hey all these people with newsletters that have been you know oh yeah newsletter published three times a week while being a full-time vc and doing all this other stuff like hell you're writing them yourself is that is that okay publishing under someone else's name because why aren't we up in arms about that we're
Jim: breathing human i actually i actually think this is going to go away from the the the button like as you publish an article on substat is that there's a check it for ai button is that i'm very very i'm very sure that's going to disappear yeah because i know it was written by ai because i i copy pasted it i don't
Will: actually want to read something that i know is 100 human because there is there is an art behind that you can feel it in the words that are written in the way it's crafted and there's joy that comes from that sam sam chris is my guy no way i could ever write with him i write like him mostly because he reads like he's like smacked out of his mind but i also is it but i don't need that from nine Me personally, I'm often reading for educational purposes and if AI wrote it, I don't care as long as the facts are right.
Jim: It's the same as I don't want to watch a carbon copy Netflix movie that has clearly been written by AI because every single one of them is exactly the same and terrible. But that's... I'm doing that for entertainment. So it's like when you're in the 90%, in the 10% of the situations, you're reading for joy. And so therefore you don't, you want it to be something of quality. You want it to be something that, you know, and not that AI isn't quality, but I also go to those really good authors and writers. Why aren't you... Why haven't you got a corpus of everything you've ever written? Why aren't you amplifying? And training a style, and then AI can do a whole lot of the heavy lifting for you, and then all of a sudden you're putting out more.
Will: I would imagine because they come from a completely different ideological basis as you, Jim. Okay, last topic of the day. OpenRouter is in talks with Stripe to be acquired for $10 billion, which is roughly eight times what their valuation was in May. These boys are timing it perfectly.
Jim: Again, is it Ramp? It happened the same week that Ramp came out and said, we're going to create a rally product. They've done the heavy lifting. They've done the work. They showed everyone this is what everyone needs. Everyone wants choice of model. Everyone wants to be able to have the one infrastructure to be able look at it and take the money. Sure, they can build something better and build it into a higher valuation over the next five, 10 years.
Alex: Take the money and run. Yeah, take the money and run for the open router, guys. I'm not sure what strike
Jim: more sense don't you get don't you get don't you get all of that for for the same price um but there's a lot of the ones that probably a different argument because it just opens up other markets to them it helps them to be able to compete with companies like there's a lot of work you've got to do yeah to get them all to work in harmony you know and so so that's sort of
Will: a lot of work in the future to get open router to be a much higher valuation is it if you've got if you've got a 10 b on the table now take it yeah yeah uh my take on open router i don't yeah i don't quite get it and i don't quite get it for stripe either um so yeah take the money and run in my opinion uh when open router first came out i was at the same time similarly trying to build like a a bit of a search engine that when you when you say and I and I don't think I was looking at Anthropic at the time and chat to your team, can I give you three or four LLMs and give you the results all at once with this perspective that AI will get much cheaper over the time was kind of my thinking and therefore why not compare different results and be able to kind of give the answer. And I looked at OpenRouter and just thought like, perspective. I find it valuable when I use it but it doesn't seem like it would be hard to build another open router especially internally if you're a company like Stripe.
Alex: There are a couple of counter examples out there at the moment. This is just the one that everyone globed onto. Probably not $10 billion. $30 million? Actually, no, because the user base is quite sticky because they're all Python and JavaScript people that don't know dick all about how to actually construct safe APIs. These are the LLM bros, the sort of early vibe code market, right?
Will: The one thing I'll say I'm happy about is that this isn't an AI company looking to buy these guys and see if there's some kind of... That is the surprise.
Jim: you know, that there's a lot of other, it's not that difficult to create all these other reasons, it's still a great reason why OpenRow should take the money and run.
Will: Agreed. Agreed. Okay, well, that's a wrap on week four of Business Idiots. Who's your business idiot this week, Ben? I mean, it's just sitting right there now. We've said maybe it's strike with this one. But no, I can't really say that without knowing exactly what their angle is. It could be something far more strategic than what I'm really seeing there.
Alex: I'm going to say the Hugging Face stock team because... Oh, who's the manager of that team? He can get the slap on the back. Oh, just put it into Fable and see what it says. Guys, just rotate the password.
Will: Actually, just roll back to me for a second. The Substack new AI writing detection tool apparently flagged a lot of the Substack staff's own blog posters.
Jim: Can I change my vote? I was going to suggest the US government screwing around, you know, oh, well, how are we going to regulate AI and all this sort of stuff. Meanwhile, the president of China is just going out and signing a deal with 30 other countries.
Will: That was going to be my suggestion to the U.S. government and the business idiots, but now I'm on Substack as well.
Alex: Sorry, Substack, we love you, but at the same time, great own goal.
Will: Realize what your product is.