LLM leaderboard

Large Language Models (LLMs) are a type of AI system trained on massive amounts of text data. They learn patterns, relationships, and structures in language, allowing them to generate human-like text, translate languages, answer questions, and perform various other language-based tasks. They typically use neural networks (often transformer models) as their underlying architecture. Modern LLMs can perform a wide range of tasks including writing, translation, summarization, question answering, code generation, and reasoning.

LLMs are large neural networks, specifically transformer architectures, that process and generate text. During training, the neural networks learn to identify patterns and relationships within massive datasets of text, by accomplishing tasks like predicting the next word in a sequence. When prompted, the neural networks use their learnt parameters to generate coherent and relevant text given the provided context.

LLM training is a two-stage process: pre-training and fine-tuning. During pre-training, models learn general language patterns. During fine-tuning, models learn to accurately accomplish a variety of tasks like answering questions, summarizing text, generating code, identifying entities and importantly, following human instructions.

LLMs are capable of solving a diversity of natural language understanding and generation tasks. They can of course generate content, from business emails to children stories. They are capable of extracting information from the provided context, summarizing and analyzing text, and translating between languages.

We can access LLMs in multiple ways. Either directly through the provider’s console or API, but also by self-hosting and on cloud providers’ platforms.

Using the provider console : ChatGPT, Gemini AI studio, Claude allow us to chat with their state-of-the-art LLM models either as a free or paid method.

APIs : Almost all providers (OpenAI, Gemini, Claude, etc) allow users to use their LLMs via paid service using APIs. This is the easiest way for one to integrate with their applications and leverage AI capabilities.

Self-hosting : Research teams have released open-source LLMs (Deepseek-R1, Gemma 3, Llama 3.3, Mistral, Phi-4) allowing us to download and run them on our local machine or in cloud instances. This gives more control over the model, but may require significant computational resources and technical expertise. That said, almost all open-source versions have smaller model flavours (with fewer parameters) that are distilled from the larger models, such as Deepseek R1 1.5B and Llama 3 7B. These can be more easily hosted on local machines with decent specifications (sometimes without GPUs as well). One can interact with them either on the command line using tools like llama.cpp and ollama, or with a web interface like open-webui or text-generation-webui.

Cloud-based platforms : Cloud providers (e.g., AWS, Azure, GCP) offer managed services that allow you to deploy and run LLMs on their infrastructure. This provides a balance between control and ease of use.

It depends on the deployment method and expected usage.

Utilizing LLMs through provider APIs incurs costs based on usage, billed by token consumption.

Self-hosting open-source LLMs may require substantial computational resources, particularly GPUs. This approach offers greater control and privacy, but GPUs need to be highly utilized for it to be cheaper than using provider APIs.

Cloud providers offer managed LLM services that balance cost and control. Pricing models vary, but often involve a combination of compute and usage-based charges.

LLMs represent a significant advancement over traditional Natural Language Processing (NLP) techniques. They generalize better, which allows them to solve very diverse tasks effectively. Being trained on large text corpuses, they embed strong priors about natural language, producing human-like text. Finally, they are computationally efficient and can use GPUs (which are highly parallel processors) very well.

GPT (Generative Pre-trained Transformer) is a specific family of Large Language Models developed by OpenAI. The term LLM is a more generic term referring to any large language model. GPT designates a specific implementation and architecture, and therefore GPT is an LLM.

These are the number of parameters in the LLM, in billions. LLMs are neural networks that are built from matrices: the no. parameters is the total no. elements in these matrices. Parameters are learnt during the training process and then used to make predictions. The higher the number of parameters, the more complex patterns the model can learn. In most cases, they are also directly proportional to the model’s performance. However, more parameters require higher computational resources to run the models (essentially GPU memory and bandwidth).

While previous iterations of LLMs were primarily trained on vast amounts of text, newer models are also trained on non-textual data like images, audio and video thanks to specialized encoders that act as translators. These encoders convert images or audio into numerical representations (vector embeddings) that the LLM can understand. On the output side, corresponding decoders generate text, but may also generate images and audio. Therefore, modern iterations can support various modalities, combining them to solve complex tasks.