FedReFT: Federated Representation Fine-Tuning with All-But-Me Aggregation

Md Anwar Hossen*, Fatema Siddika*, Juan Pablo Muñoz, Tanya Roosta, Anuj Sharma, and Ali Jannesari

EACL 2026 (19th Conf. of the European Chapter of the ACL) · *Equal contribution

TL;DR. FedReFT brings representation fine-tuning to the federated setting. Instead of sharing full models, clients adapt hidden representations, and an All-But-Me aggregation scheme lets each client benefit from peers without diluting its own specialization.

Motivation

Fine-tuning large language models across distributed clients is expensive to communicate and risky for privacy when full model weights are exchanged. Representation fine-tuning offers a lighter-weight alternative but adapting it to heterogeneous, federated clients raises new aggregation questions.

Approach

FedReFT fine-tunes a small set of interventions on a model's hidden representations rather than its full parameters, dramatically reducing what must be communicated. Its All-But-Me aggregation forms, for each client, an update from all other clients' contributions, sharing cross-client knowledge while preserving the client's own locally tuned signal and avoiding harmful averaging across heterogeneous data.

Representation-level fine-tuning instead of full-model sharing.
All-But-Me aggregation that balances collaboration with personalization.
Parameter- and communication-efficient federated adaptation of LLMs.

Importance

FedReFT makes federated adaptation of foundation models practical in bandwidth- and privacy-constrained environments, while respecting the heterogeneity that defines real federated deployments.