Adapter

Adapter is a parameter-efficient fine-tuning technique that introduces small, trainable neural network modules into frozen pre-trained models, enabling task-specific customization without modifying the original model weights. These lightweight components consist of down-projection and up-projection layers with non-linear activations that learn task-specific representations while preserving the general knowledge encoded in the base model. Adapters are inserted between layers of transformer architectures, allowing the model to adapt to new domains, languages, or tasks by training only the adapter parameters (a small fraction of the model's total: 3.6% per task in the original formulation by Houlsby et al., 2019, and less in later bottleneck variants) while keeping the pre-trained backbone frozen. This approach dramatically reduces computational requirements, storage costs, and training time compared to full fine-tuning while maintaining comparable performance across diverse tasks. Adapter variants include bottleneck adapters, parallel adapters, and multi-adapter setups that trade parameter efficiency against expressiveness. Related parameter-efficient methods such as LoRA (Low-Rank Adaptation) reach the same goal differently, by learning low-rank updates to existing weight matrices instead of inserting new modules; because such an update can be merged back into the base weights, it adds no inference latency. Enterprise applications leverage adapters for domain adaptation, multilingual models, personalization systems, and multi-task learning scenarios where organizations need to customize foundation models for specific business requirements without extensive computational overhead. Advanced adapter architectures support compositional learning, enabling combination of multiple adapters for complex tasks while maintaining modular, interpretable model components.

Related terms

• Adapter Layers

Related services: Agentic AI consulting.

← Back to the AI Glossary