The first model set to be opensourced that actually comes close to ChatGPT, and is just 13B (that's small enough for a laptop). The 51 page report from Micro...

Video about this paper. Impressive results close to GPT4 but with a much smaller model. The interesting note in the paper is:

We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA’s release policy to be published at https://aka.ms/orca-lm

  • rastilin@kbin.social
    2·
    1 year ago

    Very impressive. Yeah, ChatGPT4 is an absolute beast, but it’s impressive to see how well Orca stacks up against it in the harder trials.

    One of the best things about ChatGPT4 is that it has general knowledge, which I think a lot of the open source models don’t represent properly in their training sets. It’s not just about language processing, it’s about context awareness. Hopefully Orca as an open source model can bring context to the table.