Nice rundown. One additional point in the "Google advantages" column - Sutskever's SSI is apparently using Google TPU's, which implies they have some narrower and higher variance strategy that they're doubling down on to try to push the SOTA frontier - and of course, if they find it and demonstrate it, Google will know it exists, and could immediately throw 10x TPU's at it when they figure out what it is.
The Windsurf price is obviously crazy - but it seems to me what's happening is that OpenAI is pivoting to a more traditional products focus and are counting on that to be their moat. Windsurf probably fit one of the product niches they've mapped out, and they overpaid for it in a context where there's lots of capital chasing AI wrapper companies right now.
What a "product pivot" means overall for the overall OpenAI AGI goal is interesting to me - I never really bought any talk of slowdowns or hitting the wall that was going around a few months ago, and indeed soon after everyone was worried about that, 2.5 came out and cooked, and now o3 is out and impressive, and you know every single company has a model at least one gen advanced on what's public internally. But why worry about a product focus and customer retention now, when you have the biggest DAU's and the most name recognition by far? GPT is literally the "kleenex" of AI for normies. Were they spooked by 2.5? Is it a play to get more user data for further training?
> What a "product pivot" means overall for the overall OpenAI AGI goal is interesting to me - I never really bought any talk of slowdowns or hitting the wall
Interesting, I also hadn't thought of this angle. Personally I think it's unlikely that it's this though, Google really does seem to just be heads down improving things like context windows and reasoning, and DeepMind is the OG RL group. There were a few other answers that were interesting to me from the HackerNews post:
- telemetry data on how people use IDEs to get better RL training data
- distribution, as mentioned in the article
- competition with other AI IDEs like firebase studio
- aquihire the talent (this felt the weakest to me)
Still...OpenAI really just needs data and compute or its going to have a really tough time. I guess maybe they feel the compute stuff is handled as best it can be with Stargate, and maybe I'm just wrong on the data side?
Related: apparently OAI had gone to Cursor to try to buy them, _twice_, and cursor turned them down. So Windsurf was the second option. Which clarifies at least why they didn't just go after cursor instead of windsurf.
> Still...OpenAI really just needs data and compute or its going to have a really tough time. I guess maybe they feel the compute stuff is handled as best it can be with Stargate, and maybe I'm just wrong on the data side?
In terms of data, Gwern has made a pretty good argument that now we can ladder upwards on data - essentially that each extant model can create good enough synthetic reasoning patterns for the next generation.
"Every problem that an o1 solves is now a training data point for an o3 (eg. any o1 session which finally stumbles into the right answer can be refined to drop the dead ends and produce a clean transcript to train a more refined intuition). As Noam Brown likes to point out, the scaling laws imply that if you can search effectively with a NN for even a relatively short time, you can get performance on par with a model hundreds or thousands of times larger; and wouldn't it be nice to be able to train on data generated by an advanced model from the future? Sounds like good training data to have!"
The Cursor data point def suggest a product focus pivot to me - that coupled with the twitter competitor thing and with the newly announced RAG compressions of all prior chats all points that way, to my mind.
somehow i'd missed that gwern article, that's a good link to have handy.
> The Cursor data point def suggest a product focus pivot to me - that coupled with the twitter competitor thing and with the newly announced RAG compressions of all prior chats all points that way, to my mind.
Also possible that OAI is just feeling some pressure to justify its valuation. Maybe they just have to give their product teams something to do while waiting for their AI teams to get to GPT 17 or whatever
Apple doesn’t need to copycat others to sell products.
Sorry, what are you responding to exactly?
Nice rundown. One additional point in the "Google advantages" column - Sutskever's SSI is apparently using Google TPU's, which implies they have some narrower and higher variance strategy that they're doubling down on to try to push the SOTA frontier - and of course, if they find it and demonstrate it, Google will know it exists, and could immediately throw 10x TPU's at it when they figure out what it is.
The Windsurf price is obviously crazy - but it seems to me what's happening is that OpenAI is pivoting to a more traditional products focus and are counting on that to be their moat. Windsurf probably fit one of the product niches they've mapped out, and they overpaid for it in a context where there's lots of capital chasing AI wrapper companies right now.
What a "product pivot" means overall for the overall OpenAI AGI goal is interesting to me - I never really bought any talk of slowdowns or hitting the wall that was going around a few months ago, and indeed soon after everyone was worried about that, 2.5 came out and cooked, and now o3 is out and impressive, and you know every single company has a model at least one gen advanced on what's public internally. But why worry about a product focus and customer retention now, when you have the biggest DAU's and the most name recognition by far? GPT is literally the "kleenex" of AI for normies. Were they spooked by 2.5? Is it a play to get more user data for further training?
> What a "product pivot" means overall for the overall OpenAI AGI goal is interesting to me - I never really bought any talk of slowdowns or hitting the wall
Interesting, I also hadn't thought of this angle. Personally I think it's unlikely that it's this though, Google really does seem to just be heads down improving things like context windows and reasoning, and DeepMind is the OG RL group. There were a few other answers that were interesting to me from the HackerNews post:
- telemetry data on how people use IDEs to get better RL training data
- distribution, as mentioned in the article
- competition with other AI IDEs like firebase studio
- aquihire the talent (this felt the weakest to me)
Still...OpenAI really just needs data and compute or its going to have a really tough time. I guess maybe they feel the compute stuff is handled as best it can be with Stargate, and maybe I'm just wrong on the data side?
Related: apparently OAI had gone to Cursor to try to buy them, _twice_, and cursor turned them down. So Windsurf was the second option. Which clarifies at least why they didn't just go after cursor instead of windsurf.
> Still...OpenAI really just needs data and compute or its going to have a really tough time. I guess maybe they feel the compute stuff is handled as best it can be with Stargate, and maybe I'm just wrong on the data side?
Yeah, I think they're basically okay on the compute front via Stargate: https://www.lesswrong.com/posts/SoWperLCkunB9ijGq/can-ai-scaling-continue-through-2030-epoch-ai-yes
In terms of data, Gwern has made a pretty good argument that now we can ladder upwards on data - essentially that each extant model can create good enough synthetic reasoning patterns for the next generation.
"Every problem that an o1 solves is now a training data point for an o3 (eg. any o1 session which finally stumbles into the right answer can be refined to drop the dead ends and produce a clean transcript to train a more refined intuition). As Noam Brown likes to point out, the scaling laws imply that if you can search effectively with a NN for even a relatively short time, you can get performance on par with a model hundreds or thousands of times larger; and wouldn't it be nice to be able to train on data generated by an advanced model from the future? Sounds like good training data to have!"
Comment here: https://www.lesswrong.com/posts/HiTjDZyWdLEGCDzqu/?commentId=MPNF8uSsi9mvZLxqz
The Cursor data point def suggest a product focus pivot to me - that coupled with the twitter competitor thing and with the newly announced RAG compressions of all prior chats all points that way, to my mind.
somehow i'd missed that gwern article, that's a good link to have handy.
> The Cursor data point def suggest a product focus pivot to me - that coupled with the twitter competitor thing and with the newly announced RAG compressions of all prior chats all points that way, to my mind.
Also possible that OAI is just feeling some pressure to justify its valuation. Maybe they just have to give their product teams something to do while waiting for their AI teams to get to GPT 17 or whatever
How on earth is openai buying windsurf reeks of jealousy and lack of understanding of large language model applications and how to build them.
Who hurt you?