For example, does a 13B parameter model at 2_K quantiation perform worse than a 7B parameter model at 8bit or 16bit?

      • rufus
        link
        fedilink
        English
        arrow-up
        5
        ·
        1 year ago

        Well, a few of those extra numbers are my fault. I edited my answer a few times. And lemmy reportedly counts every edit as an additional comment. (When user and community are on different instances.) I hope they fix that soon.

        • noneabove1182@sh.itjust.worksM
          link
          fedilink
          English
          arrow-up
          2
          ·
          1 year ago

          ahh makes sense, i just made a post and deleted the comment i made on it but it glitched and deleted twice so now my post has -1 comments lmao