Wondering if Modern LLMs like GPT4, Claude Sonnet and llama 3 are closer to human intelligence or next word predictor. Also not sure if this graph is right way to visualize it.
I think the real differentiation is understanding. AI still has no understanding of the concepts it knows. If I show a human a few dogs they will likely be able to pick out any other dog with 100% accuracy after understanding what a dog is. With AI it’s still just stasticial models that can easily be fooled.
I disagree here. Dogs breeds are so diverse, there’s no way you could show some pictures of a few dogs and they’d be able to pick other dogs, but also rule out other dog like creatures. Especially not with 100 percent accuracy.
for example, wolves, hyenas, and african wild dogs certainly won’t ever reach 100% consensus on dog-or-not within human groups
Sorry, really.
Hyenas aren’t closely related to dogs.
This is entirely presumptive, we simply do not and cannot know how much they understand, this all boils down to if it looks like a duck and quacks like a duck is it a duck?
we do, and anybody telling you “it’s complicated” has an agenda.
Shouldn’t those be opposite sides of the same axis, not two different axes? I’m not sure how this graph should work.
Lemmy is full of AI luddites. You’ll not get a decent answer here. As for the other claims. They are not just next token generators anymore than you are when speaking.
There’s literally dozens of these white papers that everyone on here chooses to ignore. Am even better point being none of these people will ever be able to give you an objective measure from which to distinguish themselves from any existing LLM. They’ll never be able to give you points of measure that would separate them from parrots or ants but would exclude humans and not LLMs other than “it’s not human or biological” which is just fearful weak thought.
you use “luddite” as if it’s an insult. History proved luddites were right in their demands and they were fighting the good fight.
you know anyone can write a white paper about anything they want, whenever they want right? A white paper is not authoritative in the slightest.
Here’s an easy way we’re different, we can learn new things. LLMs are static models, it’s why they mention the cut off dates for learning for OpenAI models.
Another is that LLMs can’t do math. Deep Learning models are limited to their input domain. When asking an LLM to do math outside of its training data, it’s almost guaranteed to fail.
Yes, they are very impressive models, but they’re a long way from AGI.
I know lots of humans who can’t do maths. At least I think they’re human. Maybe there LLMs, by your definition.
There’s a preprint paper out that claims to prove that the technology used in LLMs will never be able to be extended to AGI, due to the exponentially increasing demand for resources they’d require. I don’t know enough formal CS to evaluate their methods, but to the extent I understand their argument, it is compelling.
Somewhere on the vertical axis. 0 on the horizontal. The AGI angle is just to attract more funding. We are nowhere close to figuring out the first steps towards strong AI. LLMs can do impressive things and have their uses, but they have nothing to do with AGI
https://www.youtube.com/watch?v=KKF7kL0pGc4 what’s your take on this?
A next word predictor algorithm is still a next word predictor algorithm even if you change it’s training algorithm. To think that a LLM will eventually lead to intelligence inherently asserts that intelligence comes from the ability to use language.
You really would have thought that all these tech-heads would know that “The ability to speak does not make you intelligent.”
We know, through studies on actual humans, that language filters, constrains and quantises our thoughts process, and that different languages do this in different ways. Language harms our ability to reason. We’ve internalised it to such a degree that it now forces our ideas to fit into what the language can express. However, the ability to share our thoughts with others and collaborate is a massive boon for us as a species.
The whole this field is drawing pictures on the walls of Plato’s cave, trying to mimick the shadows being cast in from outside. Their drawings might look superficially similar to their inspiration, but they’re a poor imitation and that’s all they will ever be.
Is it not the case that predicting the next word often requires reasoning about the next word?
And that if you select for better and better prediction, you have to also select for reasoning?
This is true, but it’s specifically not what LLMs are doing here. It may come to some very limited, very specific reasoning about some words, but there’s no “general reasoning” going on.
Did you watch the video I linked?
It seems to be essentially about a way to trick them into doing general reasoning, and a direct response to your comment.
It’s not a direct response.
First off, the video is pure speculation, the author doesn’t really know how it works either (or at least doesn’t seem to claim to know). They have a reasonable grasp of how it works, but what they believe it implies may not be correct.
Second, the way O1 seems to work is that it generates a ton of less-than-ideal answers and picks the best one. It might then rerun that step until it reaches a sufficient answer (as the video says).
The problem with this is that you still have an LLM evaluating each answer based on essentially word prediction, and the entire “reasoning” process is happening outside any LLM; it’s thinking process is not learned, but “hardcoded”.
We know that chaining LLMs like this can give better answers. But I’d argue this isn’t reasoning. Reasoning requires a direct understanding of the domain, which ChatGPT simply doesn’t have. This is explicitly evident by asking it questions using terminology that may appear in multiple domains; it has a tendency of mixing them up, which you wouldn’t do if you truly understood what the words mean. It is possible to get a semblance of understanding of a domain in an LLM, but not in a generalised way.
It’s also evident from the fact that these AIs are apparently unable to come up with “new knowledge”. It’s not able to infer new patterns or theories, it can only “use” what is already given to it. An AI like this would never be able to come up with E=mc2 if it hasn’t been fed information about that formula before. It’s LLM evaluator would dismiss any of the “ideas” that might come close to it because it’s never seen this before; ergo it is unlikely to be true/correct.
Don’t get me wrong, an AI like this may still be quite useful w.r.t. information it has been fed. I see the utility in this, and the tech is cool. But it’s still a very, very far cry from AGI.
That’s literally how llma work, they quite literally are just next word predictors. There is zero intelligence to them.
It’s literally a while token is not “stop”, predict next token.
It’s just that they are pretty good at predicting the next token so it feels like intelligence.
So on your graph, it would be a vertical line at 0.
What is intelligence though? Maybe I’m getting through life just by being pretty good at predicting what to say or do next…
yeah yeah I’ve heard this argument before. “What is learning if not like training.” I’m not going to define it here. It doesn’t “think”. It doesn’t have nuance. It is simply a prediction engine. A very good prediction engine, but that’s all it is. I spent several months of unemployment teaching myself the ins and outs, developing against llms, training a few of my own. I’m very aware that it is not intelligence. It is a very clever trick it pulls off, and easy to fool people that it is intelligence - but it’s not.
Agreed
This is true if you describe a pure llm, like gpt3
However systems like claude, gpt4o and 1o are far from just a single llm, they are a blend of tailored llms, machine learning some old fashioned code to weave it all together.
Op does ask “modern llm” so technically you are right but i believed they did mean the more advanced “products”
Though i would not be able to actually answer ops questions, ai is hard to directly compare with a human.
In most ways its embarrassingly stupid, in other it has already surpassed us.
That is just next word prediction with extra steps.
Now that is fair.
None of which are intelligence, and all of which are catered towards predicting the next token.
All the models have a total reliance on data and structure for inference and prediction. They appear intelligent but they are not.
How is good old fashioned code comparing outputs to a database of factual knowledge “predicting the next token” to you. Or reinforcement relearning and token rewards baked into models.
I can tell you have not actually tried to work with professional ai or looked at the research papers.
Yes none of it is “intelligent” but i would counter that with neither are human beings, we dont even know how to define intelligence.
Intelligence is a measure of reasoning ability. LLMs do not reason at all, and therefore cannot be categorized in terms of intelligence at all.
LLMs have been engineered such that they can generally produce content that bears a resemblance to products of reason, but the process by which that’s accomplished is a purely statistical one with zero awareness of the ideas communicated by the words they generate and therefore is not and cannot be reason. Reason is and will remain impossible at least until an AI possesses an understanding of the ideas represented by the words it generates.
They’re still word predictors. That is literally how the technology works
Yeah, the only question is whether human brains are also just that.
no, they are not. try showing an ai a huge number of pictures of cars from the front. Then show them one car from the side, and ask them what it is.
Show a human one picture of a car from the front, then the one from the side and ask them what it is.
What if the human had never seen or heard of anything similar to cars?
I bet it’d be confused as much as the llm.
That’s why you show him one, before asking what that same car viewed from a different angle is.
I had never seen a recumbent bike before. I only needed to see one to know and recognize one whenever I see one. Even one with a different color or make and model. The human brain definitely works differently.
You know what bicycle are though. And you’re heard of recumbent bikes or things similar to it.
If you had never heard of anything similar at all to bikes, and saw a picture of a recumbent bike from the front only, you’d probably think “ I have no fucking idea what that is”.
Idk man, weird for you to think humans can kinda learn fully about something without all the required context.
you keep missing the fact that I don’t know out of nowhere. You would have just shown me one and told me what it was. Yes of course I’d be able to tell you what it was. You just taught me. With one example.
lol, you got me, i definitely hadn’t thought of that.
I’ll preface by saying I think LLMs are useful and in the next couple years there will be some interesting new uses and existing ones getting streamlined…
But they’re just next word predictors. The best you could say about intelligence is that they have an impressive ability to encode knowledge in a pretty efficient way (the storage density, not the execution of the LLM), but there’s no logic or reasoning in their execution or interaction with them. It’s one of the reasons they’re so terrible at math.
i think the first question to ask of this graph is, if “human intelligence” is 10, what is 9? how you even begin to approach the problem of reducing the concept of intelligence to a one-dimensional line?
the same applies to the y-axis here. how is something “more” or “less” of a word predictor? LLMs are word predictors. that is their entire point. so are markov chains. are LLMs better word predictors than markov chains? yes, undoubtedly. are they more of a word predictor? um…
honestly, i think that even disregarding the models themselves, openAI has done tremendous damage to the entire field of ML research simply due to their weird philosophy. the e/acc stuff makes them look like a cult, but it matches with the normie understanding of what AI is “supposed” to be and so it makes it really hard to talk about the actual capabilities of ML systems. i prefer to use the term “applied statistics” when giving intros to AI now because the mind-well is already well and truly poisoned.
what is 9?
exactly! trying to plot this is in 2D is hella confusing.
plus the y-axis doesn’t really make sense to me. are we only comparing humans and LLMs? where do turtles lie on this scale? what about parrots?
the e/acc stuff makes them look like a cult
unsure what that acronym means. in what sense are they like a cult?
Effective Accelerationism. an AI-focused offshoot from the already culty effective altruism movement.
basically, it works from the assumption that AGI is real, inevitable, and will save the world, and argues that any action that slows the progress towards AGI is deeply immoral as it prolongs human suffering. this is the leading philosophy at openai.
their main philosophical sparring partners are not, as you might think, people who disagree on the existence or usefulness of AGI. instead, they take on the other big philosophy at openai, the old-school effective altruists, or “ai doomers”. these people believe that AGI is real, inevitable, and will save the world, but only if we’re nice to it. they believe that any action that slows the progress toward AGI is deeply immoral because when the AGI comes online it will see that we were slow and therefore kill us all because we prolonged human suffering.
That just seems like someone read about Roko’s basilisk and decided to rebrand that nightmare as the mission/vision of a company.
What a time to be alive!
can you give an example of any third data point such as a rock or a chicken
rockegg
This should just be a 1D spectrum line.
</dataviz>
All A no I
Allanoi is going to be the name of a 0 INT Warforged character I’ll create ^^
Sure, they ‘know’ the context of a conversation but only by which words are most likely to come next in order to complete the conversation. That’s all they’re trained to do. Fancy vocabulary and always choosing the ‘best’ word makes them really good at appearing intelligent. Exactly like a Sales Rep who’s never used a product but knows all the buzzwords.
Modern LLMs are basically really fancy Markov chains.
Markov chains, but without being Markovian.
