Nazrin@burggit.moe to AI discussions@burggit.moeEnglish · 1 year agoBeeeg GPT model (176B) (94GB) (150GB of vram needed)huggingface.coexternal-linkmessage-square9fedilinkarrow-up18arrow-down10
arrow-up18arrow-down1external-linkBeeeg GPT model (176B) (94GB) (150GB of vram needed)huggingface.coNazrin@burggit.moe to AI discussions@burggit.moeEnglish · 1 year agomessage-square9fedilink
minus-squareMarimfisher@burggit.moelinkfedilinkEnglisharrow-up3·1 year agoLooks like they recommend 2 or 3 80/48GB GPUs combined to run the model. Something like one of these server/professional GPUs https://www.tomshardware.com/news/nvidia-rtx-a6000-48gb-benchmarked String a few dozen of those together and that’s sorta how crypto mining rigs operate. They cost a ton, but when you need dedotated wam, you need it bad. Plus they’re not for general use, they rarely even have connection ports on them (sad!)
Looks like they recommend 2 or 3 80/48GB GPUs combined to run the model. Something like one of these server/professional GPUs https://www.tomshardware.com/news/nvidia-rtx-a6000-48gb-benchmarked
String a few dozen of those together and that’s sorta how crypto mining rigs operate.
They cost a ton, but when you need dedotated wam, you need it bad. Plus they’re not for general use, they rarely even have connection ports on them (sad!)