What is an Adapter

Adapter is a parameter-efficient fine-tuning technique that introduces small, trainable neural network modules into frozen pre-trained models, enabling task-specific customization without modifying the original model weights. These lightweight components consist of down-projection and up-projection layers with non-linear activations that learn task-specific representations while preserving the general knowledge encoded in the base model. Adapters are inserted between layers of transformer architectures, allowing the model to adapt to new domains, languages, or tasks by training only the adapter parameters (typically 1-5% of total parameters) while keeping the pre-trained backbone frozen. This approach dramatically reduces computational requirements, storage costs, and training time compared to full fine-tuning while maintaining comparable performance across diverse tasks. Modern adapter implementations include bottleneck adapters, parallel adapters, and LoRA (Low-Rank Adaptation) variants that optimize the trade-off between parameter efficiency and model expressiveness. Enterprise applications leverage adapters for domain adaptation, multilingual models, personalization systems, and multi-task learning scenarios where organizations need to customize foundation models for specific business requirements without extensive computational overhead. Advanced adapter architectures support compositional learning, enabling combination of multiple adapters for complex tasks while maintaining modular, interpretable model components.