makeasnek@lemmy.ml to AI@lemmy.mlEnglish · 4 months agoLLM ASICs on USB sticks?lemmy.mlimagemessage-square12fedilinkarrow-up126arrow-down16file-textcross-posted to: aicompanions@lemmy.world
arrow-up120arrow-down1imageLLM ASICs on USB sticks?lemmy.mlmakeasnek@lemmy.ml to AI@lemmy.mlEnglish · 4 months agomessage-square12fedilinkfile-textcross-posted to: aicompanions@lemmy.world
Source: nostr https://snort.social/nevent1qqsg9c49el0uvn262eq8j3ukqx5jvxzrgcvajcxp23dgru3acfsjqdgzyprqcf0xst760qet2tglytfay2e3wmvh9asdehpjztkceyh0s5r9cqcyqqqqqqgt7uh3n Paper: https://arxiv.org/abs/2406.02528
minus-squareFischlinkfedilinkEnglisharrow-up2·20 hours agoBeing able to run 7B quality models on your phone would be wild. It would also make it possible to run those models on my server (which is just a mini pc), so I could connect it to my Home Assistant voice assistant, which would be really cool.
Being able to run 7B quality models on your phone would be wild. It would also make it possible to run those models on my server (which is just a mini pc), so I could connect it to my Home Assistant voice assistant, which would be really cool.