haxor@derp.fooMB to Hacker News@derp.fooEnglish · 1 year agoGPU-Accelerated LLM on a $100 Orange Piblog.mlc.aiexternal-linkmessage-square1fedilinkarrow-up13arrow-down10file-textcross-posted to: hackernews@lemmy.smeargle.fansbootstrappable@slrpnk.nettechnews@radiation.party
arrow-up13arrow-down1external-linkGPU-Accelerated LLM on a $100 Orange Piblog.mlc.aihaxor@derp.fooMB to Hacker News@derp.fooEnglish · 1 year agomessage-square1fedilinkfile-textcross-posted to: hackernews@lemmy.smeargle.fansbootstrappable@slrpnk.nettechnews@radiation.party
minus-squarebeenglinkfedilinkEnglisharrow-up1·1 year agoI cloned and compiled the alpaca.cpp on my Android Oneplus 9, I can get it started but dies due to not enough RAM. I have 8 GB total, but cant get the required 4 GB free to run the trimmed Llama (alpaca) model. I might splash out on the orange pi for some fun.
I cloned and compiled the alpaca.cpp on my Android Oneplus 9, I can get it started but dies due to not enough RAM. I have 8 GB total, but cant get the required 4 GB free to run the trimmed Llama (alpaca) model.
I might splash out on the orange pi for some fun.