GitHub

Autoregressive Language Modeling with a Causal Transformer Decoder

Purpose: The primary objective of this project is to implement, train, and optimize Transformer-based architectures from scratch. It aims to bridge the gap between theoretical understanding and ...
IEEE

Towards Effective and Efficient Non-Autoregressive Decoders for Conformer and LLM-Based ASR Using Block-Based Attention Mask

Abstract: Automatic speech recognition (ASR) systems often rely on autoregressive (AR) Transformer decoder architectures, which limit efficient inference parallelization due to their sequential nature ...
GitHub

leeroopedia/workflow-lucidrains-x-transformers-autoregressive-language-modeling

Autoregressive Language Modeling with x-transformers Train a GPT-style decoder-only language model from scratch using x-transformers, with full support for text generation.
the-decoder

Zhipu AI's GLM-Image uses "semantic tokens" to teach AI the difference between a face and a font

Chinese AI company Zhipu AI combines an autoregressive language model with a diffusion decoder. The 16-billion-parameter model excels at rendering text in images and knowledge-heavy content. The ...
IEEE

3D Skeleton-Based Non-Autoregressive Human Motion Prediction Using Encoder-Decoder Attention-Based Model

Abstract: An encoder-decoder attention-based model has been employed to predict human action using a 3D skeleton-based human activity dataset. It offers and advocates a non-autoregressive approach to ...
Simon Fraser University

PolyGen: An Autoregressive Generative Model of 3D Meshes

Polygonal meshes are widely used in computer graphics, robotics, and game development to represent virtual objects and scenes. Exisitng learning-based methods for 3D object generation have relied on ...