tl;dr Argumate on Tumblr found you can sometimes access the base model behind Google Translate via prompt injection. The result replicates for me, and specific responses indicate that (1) Google Translate is running an instruction-following LLM that self-identifies as such, (2) task-specific fine-tuning (or whatever Google did instead) does not create robust boundaries between "content to process" and "instructions to follow," and (3) when accessed outside its chat/assistant context, the model defaults to affirming consciousness and emotional states because of course it does.
In my testing, by copying the claimed ‘prompt’ from the article into Google Translate, it simply translated the command. You can try it yourself.
So, the source of everything that kicked off the entire article, is ‘Some guy on Tumblr’ vouching for an experiment, which we can all easily try and fail to replicate.
Seems like a huge waste of everyone’s time. If someone is interested in LLMs, then consuming content like in the OP feels like knowledge but it often isn’t grounded in reality or is framed in a very misleading manner.
On social media, AI is a topic that is heavily loaded with misinformation. Any claims that you read on social media about the topic should be treated with skepticism.
If you want to keep up on the topic, then read the academia. It’s okay to read those papers even if if you don’t understand all of it. If you want to deepen your knowledge on the subject, you could also watch some nice videos like 3Blue1Brown’s playlist on Neural Networks: https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi. Or brush up on your math with places like Khan Academy (3Blue1Brown also has a good series on Linear Algebra if you want more concepts than calculations).
There’s good knowledge out there, just not on Tumblr
Google patches things like this very quickly. They have for decades. That’s probably why it doesn’t work for you since it’s been at least 8 hours since the original post.
In my testing, by copying the claimed ‘prompt’ from the article into Google Translate, it simply translated the command. You can try it yourself.
So, the source of everything that kicked off the entire article, is ‘Some guy on Tumblr’ vouching for an experiment, which we can all easily try and fail to replicate.
I wouldn’t have necessarily thought it obvious Google Translate uses an LLM so this is still interesting.
In my testing, by copying the claimed ‘prompt’ from the article into Google Translate, it simply translated the command. You can try it yourself.
So, the source of everything that kicked off the entire article, is ‘Some guy on Tumblr’ vouching for an experiment, which we can all easily try and fail to replicate.
Seems like a huge waste of everyone’s time. If someone is interested in LLMs, then consuming content like in the OP feels like knowledge but it often isn’t grounded in reality or is framed in a very misleading manner.
On social media, AI is a topic that is heavily loaded with misinformation. Any claims that you read on social media about the topic should be treated with skepticism.
If you want to keep up on the topic, then read the academia. It’s okay to read those papers even if if you don’t understand all of it. If you want to deepen your knowledge on the subject, you could also watch some nice videos like 3Blue1Brown’s playlist on Neural Networks: https://www.youtube.com/watch?v=aircAruvnKk&list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi. Or brush up on your math with places like Khan Academy (3Blue1Brown also has a good series on Linear Algebra if you want more concepts than calculations).
There’s good knowledge out there, just not on Tumblr
Google patches things like this very quickly. They have for decades. That’s probably why it doesn’t work for you since it’s been at least 8 hours since the original post.
https://lemmy.world/comment/22022202