DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion Paper • 2406.06567 • Published Jun 3, 2024 • 1
NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time Paper • 2408.03675 • Published Aug 7, 2024 • 1
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging Paper • 2410.01610 • Published Oct 2, 2024 • 1
Dual Modalities of Text: Visual and Textual Generative Pre-training Paper • 2404.10710 • Published Apr 16, 2024 • 2
CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models Paper • 2604.04780 • Published Apr 6 • 11
MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions Paper • 2410.02743 • Published Oct 3, 2024 • 9
Mixture of Universal Experts: Scaling Virtual Width via Depth-Width Transformation Paper • 2603.04971 • Published Mar 5 • 4
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer Paper • 2012.15688 • Published Dec 31, 2020 • 1
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora Paper • 2012.15674 • Published Dec 31, 2020 • 1
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Paper • 2112.12731 • Published Dec 23, 2021 • 1
ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages Paper • 2212.06742 • Published Dec 13, 2022 • 4
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction Paper • 2601.05966 • Published Jan 9 • 23
MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions Paper • 2410.02743 • Published Oct 3, 2024 • 9