Back to projects
OngoingSince 2024

Study of emerging deep learning methods and techniques (PVPG851-2024)

Study of emerging deep learning methods and techniques, focusing on Transformer architectures, large language models, and GPU-accelerated processing for applications involving text, images, audio, and video.

About the project

AI systems with human-like reasoning capabilities have existed since the 1950s, but they only achieved widespread adoption with the advent of large language models (LLMs). According to the Forbes article cited in the project description, “Transformers Revolutionized AI. What Will Replace Them?”, an important advance occurred in 2017 when the Google Brain team introduced the Transformer algorithm architecture. This deep learning model replaced traditional recurrent and convolutional structures with an architecture particularly effective at understanding and contextualizing language and generating text, images, audio, and computer code.LLMs based on Transformer architectures have enabled new areas of AI capability. A well-known example is OpenAI's ChatGPT, described in the source as “Chatbot Generative Pretrained Transformer.” The CNN article cited in the project description, “Microsoft confirms it is investing billions in the creator of ChatGPT,” illustrates support for developing increasingly large LLMs, some of which may incorporate hundreds of billions of parameters to generate coherent and contextually relevant responses.Another important factor in the evolution of AI is the emergence of accelerated hardware systems known as graphics processing units (GPUs). While central processing units (CPUs) are designed for general-purpose computing, GPUs were initially developed for graphics rendering and have proved well suited to machine learning because of their architecture. GPUs have many cores capable of processing multiple tasks simultaneously.Transformers use GPUs to process multiple information pathways in parallel, accelerating the training of AI models that handle text, images, audio, and video. This parallel processing capability is crucial for computationally intensive machine learning calculations, such as matrix operations. GPUs can perform these calculations much faster than CPUs, reducing training and inference times and improving the overall performance of machine learning algorithms.

Project information

  • Period: 2024 — Present
  • Status: Ongoing
  • Nature: Research

Student participation

Undergraduate: (4) / Academic master’s: (1).

Team