ChatGPT is bullshit - Ethics and Information Technology
link.springer.com
Recently, there has been considerable interest in large language models: machine learning systems which produce human-like text and dialogue. Applications of these systems have been plagued by persistent inaccuracies in their output; these are often called “AI hallucinations”. We argue that these falsehoods, and the overall activity of large language models, is better understood as bullshit in the sense explored by Frankfurt (On Bullshit, Princeton, 2005): the models are in an important way indifferent to the truth of their outputs. We distinguish two ways in which the models can be said to be bullshitters, and argue that they clearly meet at least one of these definitions. We further argue that describing AI misrepresentations as bullshit is both a more useful and more accurate way of predicting and discussing the behaviour of these systems.

Here’s a good & readable summary paper to pin your critiques on

  • Frank [he/him, he/him]@hexbear.netEnglish
    12·
    6 months ago

    Hard agree. A hallucination results from a damaged mind perceiving things that aren’t there. Llms have no mind, no perception, and a thing has to work before you can call it damaged. Llms are exploring brave new frontiers in garbage in garbage out.

    • itappearsthat@hexbear.netOPEnglish
      14·
      6 months ago

      I would not necessarily say that is true, and the article summarizes a philosophically interesting reason why:

      The basic architecture of these models reveals this: they are designed to come up with a likely continuation of a string of text. It’s reasonable to assume that one way of being a likely continuation of a text is by being true; if humans are roughly more accurate than chance, true sentences will be more likely than false ones.

    • fuckwit_mcbumcrumble@lemmy.dbzer0.comEnglish
      5·
      6 months ago

      Have you actually used ChatGPT? The vast majority of the time it spits out good enough info. We use it at work frequently to write more tedious code. Ex: It’s written approximately 7 trillion queryselectors for me, and as long as I hand hold it it will do a good job.

      The biggest problem is when it comes to anything involving human safety. You also have to know that you have to hand hold it to get it to spit out something that’s more or less exactly what you intended. But if you use it to draft a custom cover letter for you it’s probably gonna do a good enough job, and it’s not like anyone is actually reading that shit. It’s great at doing basic math equations that involve a lot of conversions for me. It sure as hell aint the end all be all that every tech company seems to be pushing, but it’s sure as hell not wrong 50% of the time.

      • itappearsthat@hexbear.netOPEnglish
        6·
        6 months ago

        For me it is wrong more than 95% of the time. I stopped using it because it was just a waste of time. I am not doing particularly difficult or esoteric programming work and it just could not hack it at all. Often the ways it was wrong were quite subtle. And it presents wrong answers with the exact same confidence it presents right answers.

  • jaden@lemmy.zipEnglish
    1·
    6 months ago

    Yeah I just don’t see how it’s really any different from a human in that respect

    • itappearsthat@hexbear.netOPEnglish
      1·
      6 months ago

      Humans are capable of metacognition: having levels of confidence about the accuracy of their beliefs. They are also capable of communicating this uncertainty, usually through tone & phrasing.