Noodleface
A Mod Real Quick
- 40,524
- 20,610
Locally hosted AI models. I work at an enterprise server company
- 3
AI: The Rise of the Machines... Or Just a Lot of Overhyped Chatbots?
- Thread starter Sanrith Descartes
- Start date
Runnen
Ahn'Qiraj Raider
- 2,522
- 6,189
rhinohelix
Dental Dammer
- 3,648
- 6,208
Look, I am a huge AI fan, its amazing but there is no doubt we are in bubble territory: Bubbles don't last, and things that can't go on forever, don't go on forever. RAM, hard drives, etc aren't going to be this expensive forever and that was my original point: While it may indeed transition to that eventually, its going to be decades. For future generations, it may be but that's 40+ years in the future. My kids' kids may not sit down with a local desktop, but there's a lot of other things to worry about between here and there.
We've circled around this argument a number of times but its the same thing: AI isn't going to end up replacing people, its going to automate low-end processes and typically going to allow/require more hiring. Switching from horses to cars to led to more employment, not less. This kind of speculation is 50's sci-fi writing: "AI is going to EVERYTHING!!1!" Well, it going to do some things great but probably less than starry-eyed speculation leads one to think. It's just like magic but even magic has limits, and every new tool unlocks productivity that allows people and companies to power new engines of creation. One of the bigger things that is happening right now is how AI is changing software, where Moats and Walls are being torn down because people are being empowered to create things previously gated behind Subs in perpetuity, and these previous entitlements are moving back to local machines.
Control
Avatar of War Slayer
- 6,434
- 17,846
For this particular topic, I'm barely even talking about AI. How many pieces of software or media have you purchased in the last year, that you actually own? I hate phones and streaming and basically 100% of my screentime is at a destop, and a few days ago I was trying to think of the last piece of software that I bought that wasn't through some sort of an app store that just sells you a subscription or a pull-the-plug-at-our-whim license, and had I really hard time remembering. I hate all of these things, but I still don't actually buy things that I own very often at all. In the last year, basically just books. It's not that I don't use things or spend money on them. It's just that fewer and fewer things are available in an ownable form. Open source is really the last bastion, but that's very niche despite how common it seems to developers.
They're close enough that the extra control makes it worthwhile? I can't run anything too fat locally, but I have a GLM sub I use when tokens are short, and it feels like a pretty big step back. Of course, maybe that's a bad comparison or just a skill issue. Just curious in case I should be poking something I'm not.
Tuco
I got Tuco'd!
- 51,435
- 94,263
Yep. Most experienced modders started their first mod because they wanted it to exist. They then evolved into doing it for the love of modding and often don't even play the game much. Modders are also extremely territorial, prideful and prone to sudden absence, which causes all kinds of drama in every modding community I've ever seen. So when a concept is introduced like AI that challenges their little fiefdoms and their presumed status, they will be extremely hostile toward it.
They'll have to adapt though, because AI-driven modding is here to stay.
Last edited:
Noodleface
A Mod Real Quick
- 40,524
- 20,610
I think it's the cost. Enterprise API usage is insane. Plus, locally hosted ensures anthropic is not training it's models on your data. Keeping it in house opens a lot of doors.
- 2
For sure, its TBD on to what extent AI replaces people vs. augmenting them. AI changing software is one of the bigger things happening right now in terms of industry disruption and AI usage, but bigger picture that is not enabling any new technology. Its enabling people with creativity to try and do things they weren't able to do before, which is awesome.
But AI coding is just a byproduct of what these companies have really been striving for from the start: AGI and then ASI. Along the way, they realized that focusing on coding can help generate much more $$$ needed for that quest and can speed up model development at the same time. There are good arguments to be made that AGI has been achieved in the top internal models OpenAI and Anthropic have locked away right now.
Those models have surpassed people in mathematics, its not really debatable. Relatively few people out in the real world is using AI for applications like these, but it is the type of application AI researchers have been striving for from the beginning. These models are better at math than the best mathematicians in the world. It is a crisis situation for the "mathematics profession" because they are seeing that they will no longer have the opportunity to find solutions to unsolved problems on their own. As soon as somebody thinks up a new problem that may have a difficult solution, AI will solve it. Will AI be able to start creating new problems on its own (big question here) ? Its looking like mathematicians of the future may be relegated to trying to understand the solutions AI comes up with.
Is physics next? There is early evidence of this.
Point of all this is, the real AI-driven breakthroughs are just beginning, and that was probably the biggest driver of all the massive investment from the start. Breakthroughs like these are not going to allow things to slow down or a bubble to pop anytime soon - they are going to lead to more and more high - powered model usage and more investment, because every breakthrough will lead to potential applications which will require their own AI-assisted research and investment. Its fun to watch people create wacky games and videos by themselves, but the real shit is just starting to get crazy.
- 4
Intrinsic
Person of Whiteness
- 17,395
- 18,024
Not as cool as making games or anything but a thing popped up on Instagram and decided to give it a shot. Was looking for interesting ways to document our home build now that we’re at framing and drywall in a few weeks. Took about an hour start to finish with no experience doing anything like this. That includes installing Xcode, Codex, doing all the app stuff, taking pictures and then having Codex do the model.
Didn’t use high res photos and only took a minute or less scanning my room as a test. Want to go back and really take my time and see how accurate it gets.
Was midnight last night and didn’t spend a lot of time tweaking other than this one quick try. The model is crazy good for me not taking my time and putting in effort. Dimensionally it is very close, not exact. The whole project was 50 photos and fairly small on the phone like 60 meg. Office isn’t “as cluttered” as it looks, I didnt focus on any of the individual desk objects so looks a mess, also missed most of the floor by the window.
Didn’t use high res photos and only took a minute or less scanning my room as a test. Want to go back and really take my time and see how accurate it gets.
Was midnight last night and didn’t spend a lot of time tweaking other than this one quick try. The model is crazy good for me not taking my time and putting in effort. Dimensionally it is very close, not exact. The whole project was 50 photos and fairly small on the phone like 60 meg. Office isn’t “as cluttered” as it looks, I didnt focus on any of the individual desk objects so looks a mess, also missed most of the floor by the window.
- 2
rhinohelix
Dental Dammer
- 3,648
- 6,208
That's all true but it doesn't mean that it's not a bubble. Internet was a bubble and it was 1.3% of GDP; When the rail spending bubble of the post-Civil war late 1860's spending popped, it was like 2.2% of GDP; there is no doubt AI is doing amazing, civilization-changing things (there is no bigger fan/evangelist than me) but that doesn't mean there isn't over-investment in it. Right now, its at 3.0% of GDP. No one is arguing its capacity or ability (we're a LOOOONG way from AGI/ASI, but that's a wholly different discussion); like rail and fiber, it's the amount of spending and the ability to monetize/capitalize on that spending. It's just too much and still going up. Companies are investing in each other now trying to keep it going, all showing late-stage bubble signs. It's just not sustainable.
Even if Chinese psyops and Democrat Big Government-regulation Operations weren't trying to close the doors on AI, there is no way for there to be a reasonable return on investment with that level of spending: It's a bubble and will have to deflate/pop/slow down and we will have to grow into this level of investment.
- 6,484
- 11,177
That's cool. Also if you are not already, take as many photos of your walls pre-drywall as you can. If you ever need to go back to do something they are invaluable.
- 2
Haus
I am Big Balls!
- 21,988
- 91,292
Yeah, a lot of companies are looking into how to get good performing general purpose modes with their own infrastructure and internal to their networks. There are a lot of reasons to do it. But there are still performance hurdles.
My company is working on things that, in the long run, will require us to have our own specifically trained and built models to run with high speed, low latency, and minimal resource drag. Won't be impossible because the use case is specific, targeted, and narrow. But it will still be a challenge to do all the training.
- 1
- 1
rhinohelix
Dental Dammer
- 3,648
- 6,208