New Theory Suggests Chatbots Can Understand Text | Quanta Magazine
www.quantamagazine.org
Far from being “stochastic parrots,” the biggest large language models seem to learn enough skills to understand the words they’re processing.

I’ve been saying this for about a year, since seeing the Othello GPT research, but it’s great to see more minds changing on the subject.

  • beetus@lemmy.world
    2·
    11 months ago

    Neat to see, but I don’t buy it yet. I think their assumptions are reasonable but without verifying the training data or working directly with these LLM firms on these studies, I don’t see how they can claim it’s producing novel non-trained output.

    Regardless, it’s a step in the right direction for us better understanding the seemingly black box we created.

  • akrot@lemmy.world
    1·
    11 months ago

    Interesting read, basically the demonstrated that Gpt_4 can understand causality, using random graphs. Interesting take-away though is this excerpt:

    “And indeed, as the math predicts, GPT-4’s performance far outshines that of its smaller predecessor, GPT-3.5 — to an extent that spooked Arora. “It’s probably not just me,” he said. “Many people found it a little bit eerie how much GPT-4 was better than GPT-3.5, and that happened within a year.”