DreamLab
Latent Diffusion Transformer for Text2Video generation
DreamLab is a smaller-scale text-to-video model inspired by OpenAI’s Sora. I experimented with multiple autoencoder architectures, training one that combines perceptual and reconstruction losses to minimize quality loss, drawing on DDPM, LDM, VAE/VQ-VAE, Vision Transformer, and Diffusion Transformer (DiT) research. I implemented DiT-based denoising for images without text conditioning, and am now building a DiT model with spatio-temporal attention and CLIP-based text conditioning, trained on a custom Minecraft gameplay dataset as a proof of concept.
