I’m familiar with TempleOS — but it really doesn’t have any applicability here. It’s just something written by a guy with some mental illness who thought God was telling him what he wanted in an Operating System. But even for the faithful it’s just a tool — like how a temple itself may be an important holy place, but isn’t itself worshipped by the people who use it. Nobody considers a church to actually be their God.
That’s vastly different from an LLM that purports to be itself divine. We can setup an LLM that actually claims to be the second coming of Jesus, and there will be people will do whatever it tells them to because of belief. If you suck in enough people for enough years slowly enough to build up a cult following, and abuse them just enough to keep them in line, you’ll be able to tell them to do all sorts of truly atrocious things — and some subset will in fact go through with them.
And yes, people can do that already (see Jim Jones, David Koresh, or any other cult leader that convinced all their followers to kill themselves and their families) — but an LLM could have a vastly larger reach around the globe. We may not need for the LLM itself to become Skynet — one or two bad actors behind the scenes of a “divine” LLM might be enough to bring down humanity all by itself.
Look up temple OS!
I’m familiar with TempleOS — but it really doesn’t have any applicability here. It’s just something written by a guy with some mental illness who thought God was telling him what he wanted in an Operating System. But even for the faithful it’s just a tool — like how a temple itself may be an important holy place, but isn’t itself worshipped by the people who use it. Nobody considers a church to actually be their God.
That’s vastly different from an LLM that purports to be itself divine. We can setup an LLM that actually claims to be the second coming of Jesus, and there will be people will do whatever it tells them to because of belief. If you suck in enough people for enough years slowly enough to build up a cult following, and abuse them just enough to keep them in line, you’ll be able to tell them to do all sorts of truly atrocious things — and some subset will in fact go through with them.
And yes, people can do that already (see Jim Jones, David Koresh, or any other cult leader that convinced all their followers to kill themselves and their families) — but an LLM could have a vastly larger reach around the globe. We may not need for the LLM itself to become Skynet — one or two bad actors behind the scenes of a “divine” LLM might be enough to bring down humanity all by itself.