Microsoft launches XOT method to improve the reasoning ability of language model

CTOnews.com November 15 news, Microsoft recently launched a method called "Everything of Thought"(XOT), inspired by Google DeepMind AlphaZero, using compact neural networks to enhance AI model reasoning ability.

Microsoft collaborated with Georgia Institute of Technology and East China Normal University to develop the algorithm, which integrates reinforcement learning and Monte Carlo Tree Search (MCTS) capabilities to further improve the effectiveness of problem solving in complex decision-making environments.

CTOnews.com Note: Microsoft research team says XOT method allows language models to be extended to unfamiliar problems, with significant improvement in Game of 24, 8-Puzzle and Pocket Cube rigorous tests. The results show that XOT is obviously superior to other methods and even solves the problems that other methods fail. However, XOT does not achieve 100% reliability.

The XOT framework includes the following key steps:

Pre-training phase: MCTS modules are pre-trained on specific tasks to learn domain knowledge about effective thought search. Lightweight policies and value networks guide search. Thought Search: In the inference process, pre-trained MCTS modules use policy/value networks to efficiently explore and generate LLM thought trajectories.

Thought Correction: LLM reviews MCTS thoughts and identifies any errors. The modified idea was generated by additional MCTS simulations.

LLM Reasoning: Provide the modified idea to the LLM as the final clue to solve the problem.

CTOnews.com attaches the [PDF] address of the paper here, and interested users can read it in depth.