.png){kind=logo}
- Insights
- Cryptocurrencies
- Stocks
- White Papers
- Industry
- Geography
- Insights
- Cryptocurrencies
- Stocks
- White Papers
- Industry
- Geography
Researchers of Arizona State University, Tempe, show their study when it comes to planning and thinking methodically, large language models (LLMs) perform very poorly. LLM is trained on an enormous amount of data. Some of the other examples of LLMs are Google's BERT and OpenAI's GPT-2 and GPT-3. GPT-3 is the largest language model known at the time with 175 billion parameters trained on 570 gigabytes of text. It has become difficult to measure the limits of their capabilities.
LLM suffers from many of the same failures observed in current deep learning systems. Language Model for Dialogue Application is an example of LLM. In fact, the few big companies that have the required resources to train and maintain LLMs refuse or show no interest in investigating them. The recent advances in LLMs have transformed the field of NLP. From GPT-3 to PaLM, the state-of-the-art performance on natural language tasks is being pushed forward with every new LLM.
The team developed their benchmark based on the domains used in the International Planning Competition. And the effectiveness of LLMs in generating plans from text descriptions. Most benchmarks depend on a shallow type of reasoning, as well as tasks for which there is sometimes no actual ground truth. The goal of the project is to put the benchmark out and give an idea of where the current baseline is.
LLMs are large neural networks trained on lots of data and generate text that's far more fluent and coherent. Reasoning can be emergent from LLMs even without any special mechanisms such as world models and reasoning about dynamics, which can use the benchmark to support researchers' point of view. The researchers hope that their work opens new windows for developing planning and reasoning capability for current AI systems.
