This repository accompanies the preprint Residual Stream Analysis with Multi-Layer SAEs (https://arxiv.org/abs/2409.04185). See References for related work. We define ...

Sparse tensor operations are increasingly important in diverse applications such as social networks, deep learning, diagnosis, crime, and review analysis. However, a major obstacle in sparse tensor ...

While major microcontroller (MCU) suppliers like Infineon, Renesas, and STMicroelectronics have been incorporating artificial intelligence (AI) capabilities into their chips to facilitate applications ...

Generative Large Multimodal Models (LMMs), such as LLaVA and Qwen-VL, excel in vision-language (VL) tasks like image captioning and visual question answering (VQA). However, these models face ...

We introduce Permuted Block-Sparse Attention (PBS-Attn), a plug-and-play method that leverages the permutation properties of attention to increase block-level sparsity and boost efficiency of LLM ...

Introduction: With the rapid advancement of industrialization and the prevalent occurrence of haze weather, P M 2.5 contamination has emerged asa significant threat to public health and environmental ...