• SirGolan@lemmy.sdf.org
    link
    fedilink
    arrow-up
    3
    arrow-down
    1
    Ā·
    1 year ago

    They are saying the internal vector space that LLMs use is too complicated and too unrelated to the output to be understandable to humans.

    Yes, thatā€™s exactly what Iā€™m saying.

    That doesnā€™t mean theyā€™re having thoughts in there

    I mean. Not in the way we do, and not with any agency, but I hadnā€™t argued either way on thoughts because I donā€™t know the answer to that.

    we know exactly what theyā€™re doing inside that vector space ā€“ performing very difficult math that seems totally meaningless to us.

    Huh? We know what they are doing but we donā€™t? Yes, we know the math, people wrote it. I coded my first neural network 35 years ago. I understand the math. We donā€™t understand how the math is able to do what LLMs do. If thatā€™s what youā€™re saying then we agree on this.

    The vectors do not represent concepts. The vectors are math. When the vectors are sent through language decomposition they become words, but they were never concepts at any point.

    ā€œThe neurons are cells. When neurotransmitters are sent through the synapses, they become words, but they were never concepts at any point.ā€

    What do you mean by ā€œthey were never conceptsā€? Concepts of things are abstract. Nothing physical can ā€œbeā€ an abstract concept. If you think about a chair, there isnā€™t suddenly a physical chair in your head. Thereā€™s some sort of abstract representation. Thatā€™s what word vectors are. Different from how it works in a human brain, but performing a similar function.

    A word vector is an attempt to mathematically represent the meaning of a word.

    From this page. Or better still, this article explaining how they are used to represent concepts. Like this is the whole reason vector embeddings were invented.

    • Veraticus@lib.lgbt
      link
      fedilink
      English
      arrow-up
      2
      arrow-down
      2
      Ā·
      edit-2
      1 year ago

      We do understand how the math results in LLMs. Reread what I said. The neural network vectors and weights are too complicated to follow for an individual, and do not relate on a 1:1 mapping with the words or sentences the LLM was trained on or will output, so individuals cannot deduce the output of an LLM easily by studying its trained state. But we know exactly what theyā€™re doing conceptually, and individually, and in aggregate. Read your own sources from your previous post, thatā€™s what theyā€™re telling you.

      Concepts are indeed abstract but LLMs have no concepts in them, simply vectors. The vectors do not represent concepts in anything close to the same way that your thoughts do. They are not 1:1 with objects, they are not a ā€œthought,ā€ and anyway there is nothing to ā€œthinkā€ them. They are literally only word weights, transformed to text at the end of the generation process.

      Your concept of a chair is an abstract thought representation of a chair. An LLM has vectors that combine or decompose in some way to turn into the word ā€œchair,ā€ but are not a concept of a chair or an abstract representation of a chair. It is simply vectors and weights, unrelated to anything that actually exists.

      That is obviously totally different in kind to human thought and abstract concepts. It is just not that, and not even remotely similar.

      You say you are familiar with neural networks and AI but these are really basic underpinnings of those concepts that you are misunderstanding. Maybe you need to do more research here before asserting your experience?

      Edit: And in relation to your links ā€“ the vectors do not represent single words, but tokens, which indeed might be a whole word, but could just as well be part of a word or an entire phrase. Tokens do not represent the meaning of a word/partial word/phrase, just the statistical use of that word given the data the word was found in. Equating these vectors with human thoughts oversimplifies the complexities inherent in human cognition and misunderstands the limitations of LLMs.

      • SirGolan@lemmy.sdf.org
        link
        fedilink
        arrow-up
        1
        Ā·
        edit-2
        1 year ago

        But we know exactly what theyā€™re doing conceptually, and individually, and in aggregate.

        Can you define and give examples of what you mean at each level here? Maybe weā€™re just not understanding each other and mean the same thing.

        Read your own sources from your previous post, thatā€™s what theyā€™re telling you.

        The Anthropic one is saying they think they have a way to figure it out, but it hasnā€™t been tested on large models. This is their last paragraph:

        Our next challenge is to scale this approach up from the small model we demonstrate success on to frontier models which are many times larger and substantially more complicated. For the first time, we feel that the next primary obstacle to interpreting large language models is engineering rather than science.

        They are literally only able to do this on a small one layer transformer model. GPT 3 has 96 layers and 175 billion parameters.

        Also, in their linked paper:

        A key challenge to our agenda of reverse engineering neural networks is the curse of dimensionality: as we study ever-larger models, the volume of the latent space representing the modelā€™s internal state that we need to interpret grows exponentially. We do not currently see a way to understand, search or enumerate such a space unless it can be decomposed into independent components, each of which we can understand on its own.

        Under the Future Work heading:

        Scaling the application of sparse autoencoders to frontier models strikes us as one of the most important questions going forward. Weā€™re quite hopeful that these or similar methods will work ā€“ Cunningham et al.'s work [17] seems to suggest this approach can work on somewhat larger models, and we have preliminary results that point in the same direction. However, there are significant computational challenges to be overcome.

        How are you getting from that that this is a solved problem?

        Concepts are indeed abstract but LLMs have no concepts in them, simply vectors. The vectors do not represent concepts in anything close to the same way that your thoughts do. They are not 1:1 with objects, they are not a ā€œthought,ā€ and anyway there is nothing to ā€œthinkā€ them. They are literally only word weights, transformed to text at the end of the generation process.

        Again, you arenā€™t making sense here. Word/sentence vectors are literally a way to represent the concept of those words/sentences. Thatā€™s what they were built for. Thatā€™s how they are described. Letā€™s take a step back to try to understand each other.

        Are you trying to say that only human minds can understand concepts? I donā€™t buy the human brains are magic bit, and neither does our current understanding of physics. Are you assuming Iā€™m saying that LLMs are sentient, conscious, have thoughts or similar? Iā€™m not. Juryā€™s out on the thought thing, but I certainly donā€™t believe the other two things. Thereā€™s no magic with them, same with human brains. We just donā€™t fully understand what happens inside either. Anthropic in the work I quoted is making good progress at that, and I think they may be pretty close, but in terms of LLMs (and not Small LMs), they are still a black box. We know the math behind them, the software, etc. We have some theories. We still do not understand. If you can prove otherwise, please provide me with a source. Stuff is happening really fast in AI, and maybe I blinked and missed something.

        I think youā€™re maybe having a hard time with using numbers to represent concepts. While a lot less abstract, we do this all the time in geometry. ((0, 0), (10, 0), (10, 10), (0, 10), (0, 0)) Whatā€™s that? Itā€™s a square. Word vectors work differently but have the same outcome (albeit in a more abstract way).

        the vectors do not represent single words, but tokens

        I was talking word vectors where the vectors DO represent words. Itā€™s in the name. LLMs donā€™t specifically use word vectors, but the embeddings they do use work similarly.

        Tokens do not represent the meaning of a word/partial word/phrase, just the statistical use of that word given the data the word was found in.

        You are correct tokens donā€™t represent the meaning of a word. However, tokens are scalars. You are conflating tokens and embeddings / word vectors here. Tokens are used to simplify converting a string into a format a neural network can understand (a vector). If we used each ascii character in the input/output string as a vector input to the network, weā€™d have to have a lot more parameters than if we combine the characters in some way (i.e. tokens). As you said, they can be a word or a part of a word. Thereā€™s no statistics embedded with the tokens (there are some methods of using statistics to choose what tokens to use, but thatā€™s decided before even training the model and can not ever change [with our current approach]). You can read here for more information on tokens. Or you can play around with the gpt3 tokenizer.

        Your concept of a chair is an abstract thought representation of a chair. An LLM has vectors that combine or decompose in some way to turn into the word ā€œchair,ā€ but are not a concept of a chair or an abstract representation of a chair. It is simply vectors and weights, unrelated to anything that actually exists.

        If you know Python, you should grab nltk and experiment with gensim, their word vectors.

        model.most_similar(positive=[ā€˜womanā€™,ā€˜kingā€™], negative=[ā€˜manā€™], topn = 1) [(ā€˜queenā€™, 0.71181ā€¦)]

        king + woman - man = queen

        Seems like an abstract representation of those things as concepts using math. For the record, word vectors are actually pretty understandable/understood by people because you can visualize them easily. When you do, you find similar concepts clustered together (this is how vector search works except with text embeddings). Anyway, it just really seems like linking numbers to concepts is not clicking with you, or you somehow think itā€™s not possible. Reading up on computational linguistics might help.

        That is obviously totally different in kind to human thought and abstract concepts. It is just not that, and not even remotely similar.

        Yes, neural networks (although initially built thinking they were a computer version of a neuron), are a lot different from how actual brains work as weā€™ve learned in however many decades since they were invented. If youā€™re saying that intelligence and understanding is limited to the human mind, then please point to some non-religious literature that backs up your assertion.

        You say you are familiar with neural networks and AI but these are really basic underpinnings of those concepts that you are misunderstanding. Maybe you need to do more research here before asserting your experience?

        Iā€™m pretty confident in my understanding, though Iā€™m always open to new ideas that are backed with peer reviewed research. Iā€™m not going to get into a dick waving contest here, so I guess weā€™ll have to agree to disagree.

        As a side note, going back to your definition of intelligence. That was for psychology. Iā€™ll note that the Wikipedia page for Intelligence has this to say:

        The definition of intelligence is controversial, varying in what its abilities are and whether or not it is quantifiable.

        And so Iā€™ll reiterate that we donā€™t have a good definition of intelligence.

        • Veraticus@lib.lgbt
          link
          fedilink
          English
          arrow-up
          1
          Ā·
          1 year ago

          The Anthropic one is saying they think they have a way to figure it out, but it hasnā€™t been tested on large models. This is their last paragraph:

          Again, all your quotes indicate that what theyā€™ve figured out is a way to inspect the interior state of models and transform the vector space into something humans can understand without analyzing the output.

          I think your confusion is you believe that because we donā€™t know what the vector space is on the inside, we donā€™t know how AI works. But we actually do know how it accomplishes what it accomplishes. Simply because its interior is a black box doesnā€™t mean we donā€™t understand how we built that black box, or how it operates and functions.

          For an overview of how many different kinds of LLMs function, hereā€™s a good paper: https://arxiv.org/pdf/2307.06435.pdf Youā€™ll note that nowhere is there any confusion about the process of how they generate input or produce output. It is all extremely well-understood. You are correct that we cannot interrogate their internals, but that is also not what I mean, at least, when I say that we can understand them and how they work.

          I also canā€™t inspect the electrons moving through my computerā€™s CPU. Does that mean we donā€™t understand how computers work? Is there intelligence in there?

          I think youā€™re maybe having a hard time with using numbers to represent concepts. While a lot less abstract, we do this all the time in geometry. ((0, 0), (10, 0), (10, 10), (0, 10), (0, 0)) Whatā€™s that? Itā€™s a square. Word vectors work differently but have the same outcome (albeit in a more abstract way).

          No, that is not my main objection. It is your anthropomorphization of data and LLMs ā€“ your claim that they ā€œhave intelligence.ā€ From your initial post:

          But also, can you define what intelligence is? Are you sure it isnā€™t whatever LLMs are doing under the hood, deep in hidden layers?

          I think youā€™re getting caught up in trying to define what intelligence is; but I am simply stating what it is not. It is not a complex statistical model with no self-awareness, no semantic understanding, no ability to learn, no emotional or ethical dimensionality, no qualiaā€¦

          ((0, 0), (10, 0), (10, 10), (0, 10), (0, 0)) is a square to humans. This is the crux of the problem: it is not a ā€œsquareā€ to a computer because a ā€œsquareā€ is a human classification. Your thoughts about squares are not just more robust than GPTā€™s, they are a different kind of thing altogether. For GPT, a square is a token that it has been trained to use in a context-appropriate manner with no idea of what it represents. It lacks semantic understanding of squares. As do all computers.

          If youā€™re saying that intelligence and understanding is limited to the human mind, then please point to some non-religious literature that backs up your assertion.

          Iā€™m disappointed that youā€™re asking me to prove a negative. The burden of proof is on you to show that GPT4 is actually intelligent. I donā€™t believe intelligence and understanding are for humans only; animals clearly show it too. But GPT4 does not.

          • SirGolan@lemmy.sdf.org
            link
            fedilink
            arrow-up
            1
            Ā·
            edit-2
            1 year ago

            Simply because its interior is a black box doesnā€™t mean we donā€™t understand how we built that black box, or how it operates and functions.

            Wait a sec. I think weā€™re saying the same thing here. I guess depending on what you mean by how it operates and functions. Iā€™ve said multiple times we understand the math and the code. We understand how values propagate through it because again, thatā€™s all the math and code people wrote. What we donā€™t understand is how it uses that math and code to actually do thinks that seem intelligent (putting aside the point of whether it is or is not intelligent). If thatā€™s what youā€™re arguing then great, weā€™re on the same page!

            I also canā€™t inspect the electrons moving through my computerā€™s CPU. Does that mean we donā€™t understand how computers work? Is there intelligence in there?

            Well, I donā€™t have the equipment to look at electrons either (I donā€™t think that tech exists), but I can take a logic probe and get some information that I could probably understand, or someone who designs CPUs could look at the gates and whatever and tell you what they did and how they relate to whatever higher level operations. Youā€™re bringing up something completely different here. Computers are not a black box at all. LLMs are-- you just said that yourself.

            No, that is not my main objection. It is your anthropomorphization of data and LLMs

            Iā€™m not anthropomorphisizing them. What are you talking about? I keep saying they donā€™t work like human brains. I just said I donā€™t think theyā€™re sentient or conscious. I said they donā€™t have agency.

            I think youā€™re getting caught up in trying to define what intelligence is; but I am simply stating what it is not.

            How do you know what itā€™s not if we canā€™t define what it is?

            It is not a complex statistical model with no self-awareness, no semantic understanding, no ability to learn, no emotional or ethical dimensionality, no qualiaā€¦

            Juryā€™s still out on whether human brains are complex statistical models. I mean (from here)ā€¦

            Our brains have learned, through evolution and experience, the statistical properties of our natural environments and exploit this knowledge when performing perceptual tasks.

            I donā€™t make any claim to understanding neuroscience, and I donā€™t think that article is saying for sure we know that.

            Anyway, in-context learning is a thing for LLMs. Maybe one day weā€™ll figure out how to have them adjust their weights after training, but thatā€™s not happening now (well people are experimenting with it).

            New research is showing they do have semantic understanding.

            They donā€™t by themselves have self-awareness, but a software framework built up around them can generally do that to some extent.

            They do understand emotions and ethics. Someone built a fun GPTrolley web site a while ago. I think it died pretty quickly because it was too expensive for them, but it had GPT 3(?) answering Trolley Problem questions. It did (in my memory of it) like to save any ā€œAGIā€ on one track over humans, which was amusing. They donā€™t have emotions, no. Does something have to have emotions to be intelligent?

            And no, Iā€™ve said all along they arenā€™t conscious, so no qualia. Again, is that required for intelligence?

            This is the crux of the problem: it is not a ā€œsquareā€ to a computer because a ā€œsquareā€ is a human classification. Your thoughts about squares are not just more robust than GPTā€™s, they are a different kind of thing altogether. For GPT, a square is a token that it has been trained to use in a context-appropriate manner with no idea of what it represents. It lacks semantic understanding of squares. As do all computers.

            No. A square to GPTs is not just a token. Itā€™s associated with some meaning. Iā€™m not going to re-hash embedding and word vectors and whatever since I feel like Iā€™ve explained that to death.

            If youā€™re saying that intelligence and understanding is limited to the human mind, then please point to some non-religious literature that backs up your assertion.

            Iā€™m disappointed that youā€™re asking me to prove a negative.

            Iā€™m literally not. ā€œIntelligence is limited to the human mindā€ is not a negative.

            The burden of proof is on you to show that GPT4 is actually intelligent. I donā€™t believe intelligence and understanding are for humans only; animals clearly show it too. But GPT4 does not.

            I feel like Iā€™ve laid out my argument for that mostly through the Microsoft and Max Tegmark papers. Are you saying intelligence is only the domain of biological life?

            Hereā€™s a question-- are you conflating ā€œintelligenceā€ with ā€œgeneral intelligenceā€ like AGI? I find a lot of people think ā€œAIā€ means ā€œAGI.ā€ It doesnā€™t help that some people do say those things interchangeably. I was just reading a recent argument between Yann LeCun and Yoshua Bengio and they were both totally doing that. Anyway, I donā€™t at all believe GPT4 is AGI or that LLMs could even be AGI.

            For an overview of how many different kinds of LLMs function, hereā€™s a good paper: https://arxiv.org/pdf/2307.06435.pdf

            Looks like a great paper-- I hadnā€™t seen it yet. I know how LLMs are constructed (generally-- while I could go and write some code for a multi-layer neural network with back propagation without looking anything up, I couldnā€™t do that for an LLM without looking at a diagram of the layers or whatnot).

      • BitSound@lemmy.world
        link
        fedilink
        arrow-up
        2
        arrow-down
        2
        Ā·
        1 year ago

        Your concept of a chair is an abstract thought representation of a chair. An LLM has vectors that combine or decompose in some way to turn into the word ā€œchair,ā€ but are not a concept of a chair or an abstract representation of a chair. It is simply vectors and weights, unrelated to anything that actually exists.

        Just so incredibly wrong. Fortunately, Iā€™ll have save myself time arguing with such a misunderstanding. GPT-4 is here to help:

        This reads like a misunderstanding of how LLMs (like GPT) work. Saying an LLMā€™s understanding is ā€œsimply vectors and weightsā€ is like saying our brainā€™s understanding is just ā€œneurons and synapsesā€. Both systems are trying to capture patterns in data. The LLM does have a representation of a chair, but itā€™s in its own encoded form, much like our neurons have encoded representations of concepts. Oversimplifying and saying itā€™s unrelated to anything that actually exists misses the point of how pattern recognition and information encoding works in both machines and humans.

        • Veraticus@lib.lgbt
          link
          fedilink
          English
          arrow-up
          2
          arrow-down
          2
          Ā·
          1 year ago

          Are you kidding me? I sourced GPT4 itself disagreeing with you that it is intelligent and you told me itā€™s lying. And here you are, using it to try to reinforce your point? Are you for real or is this some kind of complicated game?

            • Veraticus@lib.lgbt
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              2
              Ā·
              edit-2
              1 year ago

              Here, letā€™s ask GPT4 itself since youā€™ve decided itā€™s suddenly an okay source:

              Your statement is correct in asserting that the vector representation in a language model is not an abstract representation. Itā€™s purely a mathematical construct. However, saying itā€™s ā€œunrelated to anything that actually existsā€ might be an overstatement. These vectors do capture statistical patterns in human language, which are reflections of human thought and culture. Theyā€™re just not capable of the deep, nuanced understanding that comes from human experience.

              I accept itā€™s an overstatement. But it is neither ā€œincredibly wrong,ā€ nor is it thought. (Or intelligence.)

              • SirGolan@lemmy.sdf.org
                link
                fedilink
                arrow-up
                1
                Ā·
                1 year ago

                Iā€™d just like to step in here and mention that asking an LLM is probably not a good proof (and this is directed at both of you). Its understanding of AI is from before it was trained, so it is wildly out of date at this point given how much has happened in the space since.

                  • SirGolan@lemmy.sdf.org
                    link
                    fedilink
                    arrow-up
                    1
                    Ā·
                    edit-2
                    1 year ago

                    Care to provide some proof of that? They did update their system prompt to include a few things like it is now GPT4 (it used to always say GPT3). Other than that, I donā€™t think it knows anything. But in general, I was more talking about developments in AI since it was trained which it certainly does not know.

                    Edit: hmm I just reviewed our discussion and I note you only provided one link which was to the psychological definition of intelligence. You otherwise are providing no sources to back up your claims while my responses are full of them. Please start backing up your assertions, or provide some evidence you are an expert in the field.

                • Veraticus@lib.lgbt
                  link
                  fedilink
                  English
                  arrow-up
                  1
                  arrow-down
                  2
                  Ā·
                  1 year ago

                  I was in this case ā€“ but the overall point I made is still correct. If winning this minor battle is what you were seeking, congratulations. You are no closer to understanding the truth of this or what we were actually talking about. Not that that was either your point or within your capabilities.