Curated papers that shaped the foundation of Generative AI. These are the breakthroughs that defined artificial intelligence as we know it today.
The research that transformed artificial intelligence
DeepSeek-R1: A novel approach using Reinforcement Learning to enhance reasoning in LLMs, achieving performance comparable to OpenAI's o1-1217. It introduces pure-RL training and distillation techniques for smaller, efficient models.
Phi-4 is a 14B parameter language model that prioritizes data quality and synthetic data, surpassing its teacher model GPT-4o in STEM reasoning and achieving competitive performance against much larger models.
Llama 3, Meta's new family of open-source language models, boasts impressive scale, performance rivaling GPT-4, and a commitment to responsible AI development through its open release and safety features. Multimodal extensions are under development.
Mixtral 8x7B is a high-performance, open-source Sparse Mixture of Experts (SMoE) language model that rivals and, in many cases, surpasses the performance of much larger models like Llama 2 70B and GPT-3.5, while using significantly fewer active parameters.
GPT-4 is a multimodal model excelling in benchmarks and scaling predictably. It marks a leap towards safer, more useful AI, but faces limitations requiring careful mitigation and societal consideration.
PaLM is a 540B parameter language model by Google, trained with Pathways, achieving SOTA few-shot learning and breakthrough reasoning. It pushes scaling limits, while also addressing ethical concerns like bias and toxicity in very large models.
Chinchilla demonstrates that current LLMs are undertrained. By scaling model size and training data equally, Chinchilla achieves superior performance with fewer parameters, reducing inference costs and improving accessibility.
InstructGPT uses human feedback to align language models with user intent, improving helpfulness, honesty, and harmlessness. It outperforms GPT-3 with fewer parameters, showing the potential of alignment techniques.
Latent Diffusion Models (LDMs) drastically improve the efficiency of high-resolution image synthesis by operating in a compressed latent space. This approach maintains image quality while significantly reducing computational costs, enabling new possibilities for AI-driven content creation.
LoRA enables efficient adaptation of large language models by freezing pre-trained weights and training low-rank matrices. It reduces parameters and memory needs while maintaining performance and introducing no inference latency.
GPT-3, a massive 175B parameter language model, achieves impressive few-shot learning across diverse NLP tasks. It rivals fine-tuned models but also raises ethical concerns about misuse and bias.
BERT, a novel bidirectional transformer, has revolutionized NLP by achieving state-of-the-art results across various tasks. Its deep bidirectional pre-training approach and unified architecture have significantly impacted the field.
The Transformer model uses only attention mechanisms, achieving state-of-the-art results in translation while being faster and more parallelizable than previous models. It has revolutionized sequence modeling and opened new possibilities for AI.
Explains Generative Adversarial Networks (GANs), a novel approach to generative modeling where two networks, a generator and a discriminator, compete to generate realistic data. It highlights the core concepts, advantages, and potential applications of GANs.
AEVB (VAE) enables efficient training of generative models with continuous latent variables using a reparameterization trick and neural networks, offering a scalable approach to unsupervised learning and data generation.
