Trying something new, going to pin this thread as a place for beginners to ask what may or may not be stupid questions, to encourage both the asking and answering.

Depending on activity level I’ll either make a new one once in awhile or I’ll just leave this one up forever to be a place to learn and ask.

When asking a question, try to make it clear what your current knowledge level is and where you may have gaps, should help people provide more useful concise answers!

  • SuperSpruce@lemmy.zipEnglish
    3·
    3 months ago

    Late to the party, I never got FOSAI working until I found LMStudio, but I have 2 questions:

    1. Is there any way I could utilize my GPU, a Radeon RX6800M (12GB VRAM)? I got Mistral-7B doing 5 tokens/s but it’s all running on the CPU.

    2. Is there any model specifically for programming questions? This could be of immense help to my projects without having to ask ChatGPT.

    • OpticalMooseEnglish
      3·
      3 months ago

      Have you tried the guide on AMD’s site? It looks like it’s for Windows, and I don’t know what you’re running. Plus, I use Ollama, so I probably can’t be of much help.

      For programing, my favorite is Dolphin-Mixtral, but I’ve had good results with Dolphin-Mistral and Llama2.