Denny AI Digest

AI Digest

A knowledge base exploring neural architectures, attention mechanisms, and the building blocks of modern deep learning.

Concept Constellation

Attention Is All You Need Attention Residuals Me... Attention Residuals Block Attention Residuals Deep Residual Learning... DenseFormer Depth-Weighted Averaging Depth-Wise Attention Gating Mechanism Highway Networks Hyper-Connections Kimi Linear Layer Normalization Multi-Stream Residual ... PreNorm Dilution On Layer Normalization... Pre-Norm vs Post-Norm Residual Connections Self-Attention Transformer Architecture

Paper Summaries

Attention Is All You Need #transformer#attention#machine-translation Attention Residuals #residual-connections#transformer#depth-wise-attention Deep Residual Learning for Image Recognition #resnet#residual-learning#deep-learning DenseFormer #transformer-architecture#depth-weighted-averaging#information-flow Highway Networks #deep-learning#network-architecture#gating Hyper-Connections #residual-connections#transformer#depth On Layer Normalization in the Transformer Architecture #transformer#layer-normalization#pre-norm

Concepts

Attention Residuals Mechanism #residual-connections#attention#depth-wise-attention Block Attention Residuals #residual-connections#attention#depth-wise-attention Depth-Weighted Averaging #transformer-architecture#information-flow#cross-layer-connectivity Depth-Wise Attention #attention#residual-connections#depth Gating Mechanism #gating#gradient-flow#deep-learning Layer Normalization #normalization#transformer#deep-learning Multi-Stream Residual Architectures #residual-connections#transformer#architecture Pre-Norm vs Post-Norm #transformer#layer-normalization#pre-norm PreNorm Dilution #prenorm#layer-normalization#residual-connections Residual Connections #residual-connections#skip-connections#identity-mapping Self-Attention #attention#self-attention#scaled-dot-product Transformer Architecture #transformer#architecture#encoder-decoder

Other

Kimi Linear