Artificial intelligence

Abstract: The article offers a comprehensive analysis of the stages of evolution of large language models between 2020 and 2025. We analyzed the strategic focus, approaches, architecture, and key stages of development of the AI industry, focusing on the contributions of ten leading companies: OpenAI, Anthropic, Google, Meta, Mistral AI, DeepSeek AI, AI21 Labs, Qwen, Cohere, and xAI. We analyzed the main paradigms of LLM development: scaling, proposed with the release of the GPT-3 model; aligning model responses with user prompts, characterised by the emergence of techniques such as RLHF; spreading models with open weights; reducing model costs through optimisation; and the current stage, focused on architectural efficiency, multimodality, and agent behaviour. Key trends that have shaped the current landscape of generative AI are identified, including competition between proprietary and open models, the rise of specialised agents over universal models, and the growing importance of economic efficiency. The paper summarises the current state of the industry on a proposed hierarchical model of technology development levels and outlines prospects for further development of the industry, emphasising the role of agent ecosystems and high-quality data.

Keywords: large language models, generative AI, artificial intelligence, AI evolution, model architecture, agentic reasoning, review

References:

1. Nykonenko, A. (2023). The impact of artificial intelligence on modern education: prospects and challenges. Artificial Intelligence, 2, 10-15.
2. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training.
3. Radford, A., Wu, J., Child, R., et al. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9.
4. Brown, T., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
5. Wu, J., Ouyang, L., Ziegler, D. M., Stiennon, N., Lowe, R., Leike, J., & Christiano, P. (2021). Recursively summarizing books with human feedback. arXiv preprint arXiv:2109.10862.
6. Ouyang, L., Wu, J., Jiang, X., et al. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35, 27730-27744.
7. Achiam, J., Adler, S., Agarwal, S., et al. (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
8. OpenAI. (2024). Learning to reason with LLMs. OpenAI. https://openai.com/index/learning-to-reason-with-llms/
9. OpenAI. (2025). Introducing OpenAI o3 and o4-mini. OpenAI. https://openai.com/index/introducing-o3-and-o4-mini/
10. Anthropic. (2021). A Mathematical Framework for Transformer Circuits. Anthropic. https://www.anthropic.com/research/a-mathematical-framework-for-transformer-circuits
11. Anthropic. (2022). Toy Models of Superposition. Anthropic. https://www.anthropic.com/research/toy-models-of-superposition
12. Anthropic. (2023). Claude's Constitution. Anthropic. https://www.anthropic.com/news/claudes-constitution
13. Anthropic. (2025). Tracing the thoughts of a large language model. Anthropic. https://www.anthropic.com/research/tracing-thoughts-language-model
14. Verma, S., Prasun, P., Jaiswal, A., & Kumar, P. (2025). RAIL in the Wild: Operationalizing Responsible AI Evaluation Using Anthropic's Value Dataset. arXiv. https://arxiv.org/html/2505.00204v1
15. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT (pp. 4171-4186).
16. Raffel, C., Shazeer, N., Roberts, A., et al. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140), 1-67.
17. Pichai, S. (2023). An important next step in our AI journey. Google Blog. https://blog.google/intl/en-africa/products/explore-get-answers/an-important-next-step-on-our-ai-journey/
18. Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H. T., ... & Le, Q. (2022). Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239.
19. Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., ... & Fiedel, N. (2023). Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240), 1-113.
20. Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., ... & Wu, Y. (2023). Palm 2 technical report. arXiv preprint arXiv:2305.10403.
21. Team, G., Anil, R., Borgeaud, S., Alayrac, J. B., Yu, J., Soricut, R., ... & Blanco, L. (2023). Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
22. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M. A., Lacroix, T., ... & Lample, G. (2023). Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.
23. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., ... & Scialom, T. (2023). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
24. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., ... & Ganapathy, R. (2024). The llama 3 herd of models. arXiv e-prints, arXiv-2407.
25. Meta. (2024). Llama 3.1. Llama.com. https://www.llama.com/llama3_1/
26. Lee, J., Song, K. U., Yang, S., Lim, D., Kim, J., Shin, W., ... & Kim, T. H. (2025). Efficient LLaMA-3.2-Vision by Trimming Cross-attended Visual Features. arXiv preprint arXiv:2504.00557.
27. Meta. (2024). Llama-3.3-70B-Instruct. Hugging Face. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct
28. Meta AI. (2025). The Llama 4 Herd: The Beginning of a New Era of Natively Multimodal AI Innovation. Meta AI. https://ai.meta.com/blog/llama-4-multimodal-intelligence/
29. Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L.R., Lachaux, M., Stock, P., Scao, T.L., Lavril, T., Wang, T., Lacroix, T., & Sayed, W.E. (2023). Mistral 7B. ArXiv, abs/2310.06825.
30. Mistral AI. (2024). Mistral Large. Mistral AI. https://mistral.ai/news/mistral-large
31. Mistral AI. (2025). Medium is the new large. Mistral AI. https://mistral.ai/news/mistral-medium-3
32. Rastogi, A., Jiang, A. Q., Lo, A., Berrada, G., Lample, G., Rute, J., ... & Tang, Y. (2025). Magistral. arXiv preprint arXiv:2506.10910.
33. Bi, X., Chen, D., Chen, G., Chen, S., Dai, D., Deng, C., ... & Zou, Y. (2024). Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954.
34. Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., ... & Xu, Z. (2024). Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434.
35. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., ... & Piao, Y. (2024). Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437.
36. Sallam, M., Al-Mahzoum, K., Sallam, M., & Mijwil, M. M. (2025). DeepSeek: Is it the end of generative AI monopoly or the mark of the impending doomsday? Mesopotamian Journal of Big Data, 2025, 26-34.
37. Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., ... & He, Y. (2025). Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948.
38. Karpas, E., Abend, O., Belinkov, Y., Lenz, B., Lieber, O., Ratner, N., ... & Tenenholtz, M. (2022). MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv preprint arXiv:2205.00445.
39. Lieber, O., Lenz, B., Bata, H., Cohen, G., Osin, J., Dalmedigos, I., ... & Shoham, Y. (2024). Jamba: A hybrid transformer-mamba language model. arXiv preprint arXiv:2403.19887.
40. Wang, P., Yang, A., Men, R., Lin, J., Bai, S., Li, Z., ... & Yang, H. (2022). Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In International conference on machine learning (pp. 23318-23340). PMLR.
41. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., ... & Qiu, Z. (2025). Qwen3 technical report. arXiv preprint arXiv:2505.09388.
42. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
43. Derakhshani, M.M., Varghese, D., Fadaee, M., & Snoek, C.G. (2025). NeoBabel: A Multilingual Open Tower for Visual Generation. arXiv e-prints, arXiv-2507.
44. Singh, S., Nan, Y., Wang, A., D'Souza, D., Kapoor, S., Üstün, A., ... & Hooker, S. (2025). The leaderboard illusion. arXiv preprint arXiv:2504.20879.
45. Dzikowski, R. (2025). Grok 3: A Threat to Human-AI Interaction and Technological Control. ResearchGate. https://www.researchgate.net/publication/389395726_Grok_3_A_Threat_to_Human-AI_Interaction_and_Technological_Control_licensed_under_CC_BY-SA_40