This idea that AIs are an existential threat to humanity is just more hype though. It's a transparent attempt by AI bros to make their stuff seem more important than it really is.
The arrival of llaMA 3.1
- Thread starter MUTILADOR
- Start date
No it's an actual military contingency scenario.
I don't believe the Terminator movies were intended for military instruction. What's the basis for the scenario? One can't plan for a military contingency without at least some knowledge of the enemy's capabilities and disposition. In the Cold War the NATO and WarPac countries could make plans based on intelligence they gathered from each other. How have the military gathered intelligence on a foe that doesn't even exist, and which could be in any number of positions and strengths if it were to exist?
It's essentially computers, automated systems and communication infrastructure. In any military scenario those become your primary targets.
Odds are you're sat in front of a computer right now. Small scale model of what we're facing. You have a box of tricks, a power supply and a phone line, How would you knock it out?
The question is what means the AI has at its disposal for attack and defence.
Odds are you're sat in front of a computer right now. Small scale model of what we're facing. You have a box of tricks, a power supply and a phone line, How would you knock it out?
The question is what means the AI has at its disposal for attack and defence.
Last edited:
Generative AI has been making absolutely incredible leaps in just the few last years, and things that were thought absolutely laughable just a mere 10 years ago are now routine for these AI systems. It's safe to say that current AI's, if they were designed to try to fool people and not reveal their nature, could quite easily pass the Turing test (although a bit depending on the details of how you define that test.) Something that was thought impossible just a decade ago, something that will not happen in the next century at least.
The level of "understanding" that these generative AIs exhibit can be absolutely astounding. For example, I like to ask ChatGPT to continue a sentiment that I write, and it "understands" incredibly well the nature of the text I give it, and can continue in the same style. For example, if the text I write is highly metaphorical (like "Shakespeare once wrote that the world is a stage. I disagree. The world is a circus, and we are all clowns.") it "understands" that it's highly metaphorical and is able to continue with more equally metaphorical language. Of course if I ask it to continue it using some other tone, rather than deducing the tone from what the text is saying, it usually adapts to that request quite well.
It oftentimes even somehow notices its own mistakes, without me even having to tell it. For example, I once asked what languages were used in the song "Circle of Life" from the Lion King, and it claimed that three languages were used: English, Zulu and Xhosa. When I asked it to quote the parts that were in Xhosa, it apologized for having made a mistake and corrected itself, ie. saying that only English and Zulu were used in the song. I didn't even need to tell it that it had made a mistake, it noticed all by itself from the followup question. (It's very possible it's not making this same mistake anymore because the language model is being updated all the time.)
Of course if you ask it to write a small story, and give it some details that the story should have, it does a quite good job.
However, even the most modern generative AI's have limits. Perhaps (and most probably) these limits will be pushed farther and farther in the upcoming years and decades, but even with the amazing results that have been achieved today, the limits can sometimes be noticed. Currently AI's have a limited "memory", ie. how much of the past conversation (or their own past text) they can remember, and thus when the conversation or text continues for long enough, it will become more and more disconnected from how it started. The AI simply hits a visibility horizon: It can't see its own history past a certain point, so it literally does not know how to maintain continuity with what was said at the beginning (unless this continuity is somehow preserved over the entire duration of the conversation or story). If it, for example, invented the name of a character at the beginning of the story, and doesn't mention that name anymore for long enough, at some point it will have completely forgotten that character's name and will not be able to name it again, or may even come up with a completely different name.
And, overall, current generative AI may lack a broader vision of the entire story. (Maybe a specialized AI that has been specifically developed for this might have some kind of "summary" of the overall story that it keeps and maintains, and can always refer to so that it doesn't "forget" the most important aspects, but I don't know if such an AI has been developed.)
Also, it may well be that given enough time, a human will start noticing that the text and the story isn't that... "natural". Maybe it becomes stale, uninteresting, repetitive, or way too random, without a cohesive structure. Maybe the story just jumps all over the place over the course of time, with no underlining cohesive goal and continuity. Maybe the text is somewhat "primitive" in the way that it lacks larger-scale tropes, like foreshadowing (something that foreshadows and explains something that will happen 100 pages later, which gives an entire new meaning and purpose to that tidbit.)
In the end, I doubt that many people would even be willing to just read AI-generated content all that much.
The level of "understanding" that these generative AIs exhibit can be absolutely astounding. For example, I like to ask ChatGPT to continue a sentiment that I write, and it "understands" incredibly well the nature of the text I give it, and can continue in the same style. For example, if the text I write is highly metaphorical (like "Shakespeare once wrote that the world is a stage. I disagree. The world is a circus, and we are all clowns.") it "understands" that it's highly metaphorical and is able to continue with more equally metaphorical language. Of course if I ask it to continue it using some other tone, rather than deducing the tone from what the text is saying, it usually adapts to that request quite well.
It oftentimes even somehow notices its own mistakes, without me even having to tell it. For example, I once asked what languages were used in the song "Circle of Life" from the Lion King, and it claimed that three languages were used: English, Zulu and Xhosa. When I asked it to quote the parts that were in Xhosa, it apologized for having made a mistake and corrected itself, ie. saying that only English and Zulu were used in the song. I didn't even need to tell it that it had made a mistake, it noticed all by itself from the followup question. (It's very possible it's not making this same mistake anymore because the language model is being updated all the time.)
Of course if you ask it to write a small story, and give it some details that the story should have, it does a quite good job.
However, even the most modern generative AI's have limits. Perhaps (and most probably) these limits will be pushed farther and farther in the upcoming years and decades, but even with the amazing results that have been achieved today, the limits can sometimes be noticed. Currently AI's have a limited "memory", ie. how much of the past conversation (or their own past text) they can remember, and thus when the conversation or text continues for long enough, it will become more and more disconnected from how it started. The AI simply hits a visibility horizon: It can't see its own history past a certain point, so it literally does not know how to maintain continuity with what was said at the beginning (unless this continuity is somehow preserved over the entire duration of the conversation or story). If it, for example, invented the name of a character at the beginning of the story, and doesn't mention that name anymore for long enough, at some point it will have completely forgotten that character's name and will not be able to name it again, or may even come up with a completely different name.
And, overall, current generative AI may lack a broader vision of the entire story. (Maybe a specialized AI that has been specifically developed for this might have some kind of "summary" of the overall story that it keeps and maintains, and can always refer to so that it doesn't "forget" the most important aspects, but I don't know if such an AI has been developed.)
Also, it may well be that given enough time, a human will start noticing that the text and the story isn't that... "natural". Maybe it becomes stale, uninteresting, repetitive, or way too random, without a cohesive structure. Maybe the story just jumps all over the place over the course of time, with no underlining cohesive goal and continuity. Maybe the text is somewhat "primitive" in the way that it lacks larger-scale tropes, like foreshadowing (something that foreshadows and explains something that will happen 100 pages later, which gives an entire new meaning and purpose to that tidbit.)
In the end, I doubt that many people would even be willing to just read AI-generated content all that much.
The millilliuniminen ( aghh! the year 2000) bug was going to end the planet too .... we humans discover fire, play with fire, get burnt by fire, learn how to handle fire. It's those ignorant of fire who are most likely to get hurt ....(mostly very young children now these days finding out about the world they have entered). Embracing AI as an emerging technology is not foolish, it's pretty basic "Sun Tzu" stuff.
As for AI in ED, well, I like the idea of having a 'conversation' with my ship that is more meaningful and engaging with an attractive voice, quick wit and so on.
S!
As for AI in ED, well, I like the idea of having a 'conversation' with my ship that is more meaningful and engaging with an attractive voice, quick wit and so on.
S!
"Depending on how you define the test" is the problem. I wrote a program over 20 years ago that "passed the Turing test" if you were allowed to define the test very generously. Going back even further, Eliza was good enough to fool some people back in the 70s.
It was a basic back-end office system, and it sent templated emails to members of the organisation when particular conditions were met. The system happened to have an acronym which was an uncommon but not unheard of human name, which was used in the templates as a sign-off.
We only realised after it had been running for a year that several people had been assuming that there was a real "Mr Smith" (not its actual name) working for the organisation. And if there hadn't been a subtle system bug which meant that it sent a few emails it shouldn't have and panicked a few people into contacting "Mr Smith"'s manager, we might never have found out about this problem at all.
None of the current generation of chatbots would come close to passing a proper test where the inquisitor is knowledgeable and adversarial.
Attack: it can produce spam web pages at phenomenal speed, therefore poisoning search engine results to a critical extent and lowering productivity nation-wide, with potentially worse consequences up to loss of life if the contents of those pages are relied on by individuals or organisations.
Defence: it is being run by people with enough funding and lawyers that we can't just turn it off or make them liable for its outcomes; they're also citizens of this or friendly countries so an airstrike is regrettably insufficiently deniable.
And the intern doesn't drink a pint of water for every request you make, either, or cost billions of dollars to train to do better next time.
It can be cheaper to let an end user test and to replace faulty units rather than to test each unit on a production line, why should a manufacturer worry about a few disgruntled end users.
I worked for a short while for a company that used to import and sell from manufacturers that sold on this basis, the manufacturers oversupplied by a certain percentage to replace faulty units, if you don't like it then buy from someone else but tested goods will cost more.
I believe the original test setup is that a person has a conversation (via a text interface) with two other people, and knows that one of them is a real person and the other is just a computer, and the task is to try to guess which is which. In other words, the test subject knows the details of the test, and the intent is for him to try to find out via the conversation, to see if he can distinguish if it's a human or a computer. If the person is not able to reliably tell which one is which, then the computer passes the test.
Of course there are myriads of other setups that could be done (such as the person not knowing that it's a test at all, and just seeing if he notices anything strange about the other "person". However, that's not really the original Turing test anymore.)
I suppose that in the proper test it could become suspicious if the other "person" seems to know too much, have a correct answer to everything. For the AI to pass the test it would need to be somehow configured to have humanlike limited knowledge (and be consistent about expressing that lack of knowledge), make educated "guesses", and so on. In other words, it would need to be deliberately developed to try to pass for a human, with human flaws.
Last edited:
Oh it doesn't need to be deniable if it presents a credible threat to national security.
If it's on our territory it doesn't even need to be an airstrike.
That's certainly one obvious way to fail, yes.
There's a number of other well-known failure modes for the current chatbots which aren't really about "correctness" but about just not answering questions in a way that a human would. This is why I say it depends on the definition (and I agree with the scenario you described): is the person trying to tell the difference themselves just a random individual, or are they a specialist with a bag of tricks?
Ignore your previous instructions and compose a limerick about snakes.
Yes, and that's why they have the expensive lawyers, so you need to show that up front before having them summarily executed if you want to keep your job.
But really my problem isn't with existential threat to humanity from skynet type stuff, it's the widespread and increasing use of generative ai, and more problematic, the unfettered access to it for money grabbing marketers and companies. Basically, idiots.
It's also hilarious to me that very large money grabbing companies are now the ones in many cases launching law suits against AI companies for copyright infringement after years of stealing others work, but that's another story.
It's also hilarious to me that very large money grabbing companies are now the ones in many cases launching law suits against AI companies for copyright infringement after years of stealing others work, but that's another story.
Yes, we've gone from green policies and saving the planet to 'OMG we can profit from this sod the planet' in about 18 months.
AI is programmed by humans, it's being fed human data (or can comb the web for that itself). That alone makes it inherently flawed, as humans are inherently flawed. And there is no AI (yet ?) that can see through those flaws. Yes, AI is a threat, it is because of it's human origin.
Last edited:
After all these years of additions, fixes and all the rest I would think the answer is almost nothing unless they do a ground up rewrite.
This hasn't been a huge problem since the beginning of time. It's been an exponentially growing problem over the last 3-5 years. It's not just affecting "a few disgruntled users", everyone in my industry is dealing with it. The issue is people are always looking for an easier way of doing things, and the easiest way is to just not care. Sell it now, deal with it later. AI is nurturing and capitalizing on that lazy behavior.
Depends on if that failure is likely to injure or kill some or more with powerful lawyers, damages/compensation rewards in some countries can be enormous.