7 billion / 13 billion parameters, Microsoft Research released Orca 2 LLM: performance comparable to 10 times parameter model

CTOnews.com, November 22, Microsoft Research (Microsoft Research) recently issued a press release to launch Orca 2 LLM, which is smaller than the mainstream language model, but can still answer some complex questions.

Microsoft Orca 2 comes in 7 billion and 13 billion sizes, partially incorporating Llama 2 LLM parameters to provide more accurate and excellent composite data by combining customized high-quality composite data.

Microsoft says Orca 2 uses extended, highly customized synthetic data sets for training. Orca 2 supports various reasoning techniques such as step-by-step processing, recall then generation, recall-cause-generation, extraction-generation and direct answer, and can also choose different solution strategies for different tasks.

Compared with large language models such as Llama 2 and WizardLM, Orca 2 model is better in pain understanding, common sense reasoning, multi-step reasoning, mathematical problem solving, reading comprehension and so on.

"our preliminary results show that Orca 2 performs significantly better than models of similar size," Microsoft said. "it also achieves a level of performance similar to or better than models at least 10 times larger, demonstrating the potential to equip smaller models with better reasoning capabilities."

A link to the introduction of the Microsoft Orca 2 model is attached to CTOnews.com, which can be read in depth by interested users.