• Pogogunner@sopuli.xyz
      link
      fedilink
      arrow-up
      24
      ·
      4 days ago

      This is just the thing - if you don’t understand the subject, the AI output seems perfectly reasonable, and you don’t see the need to look further.

      If you understand the subject the AI is spouting off about, you immediately see that it’s full of shit and shouldn’t be trusted.

      • Rhaedas@fedia.io
        link
        fedilink
        arrow-up
        5
        ·
        4 days ago

        Which should be a warning if you use it for a subject you aren’t familiar with. Why would it suddenly become very good at output on something you’re not sure of? It can be useful as a sounding bound of your own ideas, as it’s very good at taking pieces of what you input and completing them in various ways. Keeping that within the context window to prevent wandering, you are modeling your own ideas. But that’s not how lots of (most) people are using it.

      • jj4211@lemmy.world
        link
        fedilink
        arrow-up
        3
        ·
        4 days ago

        That’s remarkably like watching movies or television when the writers get near your area of expertise…

    • Sheridan@lemmy.world
      link
      fedilink
      English
      arrow-up
      14
      ·
      4 days ago

      G to E is a major sixth, not a major seventh. That’s the mistake. It then misidentifies the chord because of this.

    • phaedrus@piefed.world
      link
      fedilink
      English
      arrow-up
      7
      ·
      4 days ago

      Just bolstering one of the other comments with a more visual approach to show just how simple the deduction would be, even if you don’t understand music.

      Notes are only A - G and they repeat (i.e., G loops back to A). In the example, G is the ‘root’ and considered note #1, so when you get to F it loops back to G to complete the scale/octave. Armed with that knowledge, you can see more clearly how claude bungled it by laying the notes out like below. It got B and D right, but couldn’t do simple arithmetic to place E.

      1. G
      2. A
      3. B
      4. C
      5. D
      6. E
      7. F

      It’s basic deduction for a human English speaker knowing that E immediately follows D, and therefore should be 5+1 = 6. Such a tiny, simple thing but shows just how scary it is that people trust this stuff blindly and don’t corroborate the info given. Now imagine a young, fresh chemist or physicist fully trusting the output because they’ve been taught to by their professors.

      • ChaoticNeutralCzech@feddit.org
        link
        fedilink
        English
        arrow-up
        1
        ·
        3 days ago

        This isn’t the G major scale, though. You need to add flats and sharps to get the right number of semitones (two per step except for 3-4 and 7-8), in this case F♯.

        For example, the major 3rd from D is not F but F♯ because the D major scale has F♯ and C♯. Therefore, the basic 1-3-5 D-major chord is D-F♯-A.

        • phaedrus@piefed.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          2 days ago

          I wasn’t depicting a specific scale, just keeping it simple for those that don’t know music and showing what the obvious flaw was in the screengrab. The specifics aren’t necessary for the point to be valid and only serves to muddy the waters for folks that don’t know theory.