Lugh@futurology.todayM to Futurology@futurology.todayEnglish · 2 days agoWithin 24 hours, OpenAI's Deep Research has been replicated by an open-source version that already scores 54% on the same validation set OpenAI's scored 67%.huggingface.coexternal-linkmessage-square10fedilinkarrow-up141arrow-down11
arrow-up140arrow-down1external-linkWithin 24 hours, OpenAI's Deep Research has been replicated by an open-source version that already scores 54% on the same validation set OpenAI's scored 67%.huggingface.coLugh@futurology.todayM to Futurology@futurology.todayEnglish · 2 days agomessage-square10fedilink
minus-squareBronzebeard@lemm.eelinkfedilinkEnglisharrow-up7·2 days agoThat’s 22% worse. That’s basically wrong half the time vs wrong 1/3 of the time.
minus-squareLugh@futurology.todayOPMlinkfedilinkEnglisharrow-up20·2 days agoYes, but GPT-4 was at 7% and regarded as world best only months ago. The true significance here, is that they’ve replicated the industry leader so easily and so quickly.
That’s 22% worse.
That’s basically wrong half the time vs wrong 1/3 of the time.
Yes, but GPT-4 was at 7% and regarded as world best only months ago.
The true significance here, is that they’ve replicated the industry leader so easily and so quickly.