• webghost0101@sopuli.xyz
    link
    fedilink
    English
    arrow-up
    2
    ·
    1 year ago

    12GB of VRAM is still an upgrade away for most people and a 4bit quantized 13B model is barely going to be a tech demo. When open source ai is proclaimed to be near/on par/better then gpt4 they are talking about nothing else than their biggest models in a prime environment.

      • webghost0101@sopuli.xyz
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        I usually don’t think much about companies and cloud instances when it comes to Fossai but fair enough.

        For me its all about locally run consumer models. If we cannot archive that it means we will always need to rely on the wims and decisions of others to acces the most transforming technology ever invented.