The suggested preprocessing step
Picture three hospitals that want to train one model together. Their tables share a schema, so every site has the same columns, but each holds different patients and none may send its rows to anyone else. This is horizontal federated learning: the features are shared, the rows are split across sites, and the model is trained where the data lives. The standard recipe is FedAvg (McMahan et al., 2017), in which each site trains locally, sends its weights to a server, and the server averages them, weighted by each site’s sample count.
On a pooled table, feature selection is routine. I would fit a Lasso, check random forest importances or call SelectKBest, and drop some unnecessary columns. In federated learning there is no pooled table to run those tools on, and FedAvg never asks which features belong in the model. It averages whatever weights it receives, so every irrelevant feature rides along in every update, every round, and adds noise that the smaller sites feel most.
This article is about Feature Election, a method I developed for that gap. Most research on federated feature selection asks you to adopt a new purpose-built algorithm. Fed-FiS (Banerjee et al., 2021) and Fed-mRMR (Hermo et al., 2024) federate mutual-information filters, the latter by having each site share a matrix of co-occurrence counts for feature values. FedSDG-FS (Li et al., 2023) learns stochastic feature gates under homomorphic encryption for vertically partitioned data, and other work searches for feature subsets with particle swarm optimization (Zhang et al., 2026). Feature Election asks for less: each site keeps the ordinary selector it already trusts, runs it on its own rows and sends a vote, and the server settles the disagreement with one tunable rule. Below I work through the rule by hand, implement it in NumPy, run it as a real NVIDIA FLARE (Roth et al., 2022) job, report our published results on model size and F1 alongside an independent benchmark, and show how to run it in Flower (Beutel et al., 2020) and NVIDIA FLARE.
The idea explained
Each site runs a feature selector of its choice on its local data. For every feature it produces a yes or no (together these form the mask) and a number saying how strongly it prefers that feature (the score), and it sends both to the server along with its sample count. Features that every site selected are kept and features that no site selected are dropped, whatever their scores. The contested ones go to a vote weighted by site size, and a single parameter called the freedom degree decides how many of them get in. Every site applies the resulting global mask to its columns, and ordinary federated training such as FedAvg runs on the reduced table. No rows or gradients leave a site during the election.
How the election works
For a table with d features, client i sends:
-
S_i, a binary mask of lengthd, where 1 means “my selector kept this feature”. -
M_i, a vector ofdpreference scores (for example absolute Lasso coefficients, random forest importances or mutual information values). -
n_i, its number of training samples.
What leaves the devices during a Feature Election: masks, scores and row counts go up, one global mask returns. Data rows will not leave the site
The server splits the features into three groups. The intersection holds the features every client selected, and these are always kept. Features no client selected are always dropped. The disputed set is the union of all selections minus the intersection: features that at least one site wanted and at least one rejected.
The freedom degree fd, a number in [0, 1], sets how much of the disputed set is admitted. At fd = 0 the global mask is the intersection and at fd = 1 it is the union. In between, the server min-max normalizes each client’s scores separately, so that a Lasso coefficient and a mutual information value land on the same 0 to 1 scale. It computes client weights w_i = n_i / sum(n) and an aggregated vote per feature, V = sum_i w_i * M_i, over the normalized scores. It then keeps the top ceil(fd * |disputed|) disputed features by V, on top of the intersection.
The freedom degree slides the global mask from the intersection (`fd = 0`) to the union (`fd = 1`). Filled dots are features kept in the worked example below.
The freedom degree trades off two failure modes. At fd = 0 every site holds a veto, so one site with a strict selector shrinks the model for everyone, including sites whose data needed the vetoed features. At fd = 1 every site can pollute the mask, so the noise features of one loose selector reach everyone. In between, the sites with more data and stronger preferences decide the contested cases.
A small example makes this concrete. Three clients hold 50%, 30% and 20% of the data, so their weights are 0.5, 0.3 and 0.2. Feature f1 was selected by all three. Feature f2 was selected by C1 (score 0.9) and C2 (0.6), f3 by C2 (0.8) and C3 (0.6), and f4 only by C3 (0.4). The table shows every client’s normalized score for every feature, with the scores of features that client selected in bold. A client’s score counts in the vote whether or not it selected the feature: how much C1 likes f3 can help or hurt f3 even though only C2 and C3 put it in their masks.
|
Feature |
C1 |
C2 (w =0.3) |
C3 (w =0.2) |
Vote V |
Outcome at fd = 0.4 |
|---|---|---|---|---|---|
|
f1 |
selected |
selected |
selected |
n/a (Unanimous) |
kept (intersection) |
|
f2 |
0.9 |
0.6 |
0.3 |
0.5×0.9+0.3×0.6+0.2×0.3 = 0.69 |
kept (rank 1) |
|
f3 |
0.2 |
0.8 |
0.6 |
0.5×0.2+0.3×0.8+0.2×0.6 =0.46 |
kept (rank 2) |
|
f4 |
0.1 |
0.2 |
0.4 |
0.5×0.1+0.3×0.2+0.2×0.4=0.19 |
dropped (rank 3) |
The disputed set is {f2, f3, f4}, and at fd = 0.4 the server admits ceil(0.4 × 3) = 2 of them, f2 and f3, giving the global mask {f1, f2, f3}. At fd = 0 the mask would be {f1} alone and at fd = 1 all four features. The weighting widens the gap: with equal site sizes f2’s vote would be 0.60 and f3’s about 0.53, so C1’s half of the data is what puts f2 clearly ahead.
The algorithm in NumPy
The rule is short enough to write out in full. The function takes lists of masks, score vectors and sample counts and returns the global mask with a few counts. Scores go through abs so that negative Lasso coefficients vote the same way as positive ones.
Note that the masks only decide which features are in the intersection, the disputed set, or out. The vote itself uses every client’s score for a disputed feature, including clients that left it out of their mask. A feature that only the smallest site selected can still win a place if the larger sites scored it just below their own cut-off, and it will lose if they scored it near zero. This is why the function normalizes and sums the full score vectors and does not zero them with the masks.
Running it as an NVIDIA FLARE job
The function above is for reading. For numbers I run the shipped implementation as a FLARE job: a FeatureElectionController on the server and a FeatureElectionExecutor on each of three sites, exchanging FLARE tasks. I launch it with FLARE’s simulator, which runs the server and the three clients on one machine; the same job object can be exported with job.export_job() and submitted to a provisioned federation. The module is in the nvflare package from release 2.8.0; I ran 2.9.0.
The table has 100 features: 12 informative, 8 redundant (linear combinations of the informative ones, so they carry signal too) and 80 noise. With shuffle=False, scikit-learn puts the 20 signal features in the first 20 columns, which makes it easy to check what the election kept. A short script, prepare_data.py writes one CSV per site with 1,500, 900 and 600 rows. Each site needs one thing from me: how to load its own data. The executor below reads the site’s CSV the first time a task arrives and leaves everything else to Feature Election.
The job gives every site a different selector: Lasso at the largest site, a random forest at the second and mutual information at the smallest.
I ran the job at five freedom degrees and once with auto-tuning. With nvflare 2.9.0 and scikit-learn 1.9.1, the last line of each run was:
Both failures from the previous section show up. On their own data the Lasso site selected 35 features with 12 of the 20 signal features among them, the random forest site 50 with all 20, and the mutual information site 50 with 15. At a freedom degree of 0 the strictest selector holds a veto, and the mask keeps only 10 signal features. At 1 every site’s noise gets in, and the mask carries 61 noise features. At 0.5 the vote recovers all 20 signal features with 28 noise features, less than half the noise of the union. I repeated the runs at 0, 0.5 and 1 with two other seeds for the table and the pattern held: 8 and 9 signal features at 0, 19 at 0.5, and 61 and 62 noise features at 1.
Auto-tuning has one limitation here. It tried freedom degrees between 0.5 and 0.7, saw validation F1 between 0.776 and 0.781, and settled on 0.6 with 55 features. The hill climb optimizes F1 alone, so when F1 is this flat it has no reason to prefer the smaller mask, and 0.5 gives the same signal with 7 fewer noise features. If model size matters to you, check the neighbouring values yourself.
What it offers
In the FLASH paper (Christofilogiannis et al., 2025) we evaluated Feature Election with three clients on 50/30/20 stratified splits, trained with FedAvg. We used three models (Gaussian Naive Bayes, a linear SGD classifier and an MLP), four datasets (Mushroom, Adult Income, Heart Disease, Wisconsin Breast Cancer) and three local selectors: Lasso (Tibshirani, 1996); Sequential Attention (Yasuda et al., 2023), which selects features greedily using attention weights inside a network; and PyImpetus, an implementation of PPFS (Hassan et al., 2025), a wrapper that searches for the target’s Markov blanket, the smallest feature set that makes the target independent of the rest, and needs no tuning.
The consistent effect was on model size. Parameter count, and with it the traffic sent every round, fell by 49% on average. The largest cut was 88.8% (SGD classifier, Adult Income, Sequential Attention), Gaussian Naive Bayes on Breast Cancer with Lasso shrank by 81.0%, and the smallest cut was 6.7% (SGD, Income, PyImpetus).
The effect on F1, measured without hyperparameter tuning, was large in some cells and absent or negative in others.
|
Model |
Dataset |
Selector |
F1, no selection |
F1, Feature Election |
|---|---|---|---|---|
|
MLP |
Breast Cancer |
PyImpetus |
0.41 |
0.94 |
|
SGD |
Adult Income |
PyImpetus |
0.02 |
0.35 |
|
Gaussian NB |
Mushroom |
PyImpetus |
0.81 |
0.88 |
|
Gaussian NB |
Breast Cancer |
Lasso |
0.93 |
0.85 |
In the first two rows the untuned model on all features barely worked and the elected features made it usable, although 0.35 is still weak. In the third, selection added seven points to a model that already worked, and in the last it cost eight. On Heart Disease the SGD and MLP scores did not move (0.88 to 0.89 with or without selection), while Gaussian Naive Bayes gained up to five points (0.83 to 0.88), and the parameter count dropped in every case. Against a plain Flower (Beutel et al., 2020) FedAvg baseline with no selection and no tuning, the best FLASH configuration (which in several cells includes hyperparameter tuning) matched or beat it in 9 of 12 model and dataset pairs and lost slightly in the other 3, for example Gaussian Naive Bayes on Breast Cancer at 0.96 against 0.98. The dependable gain is fewer parameters shipped every round; accuracy was usually preserved and sometimes much improved, depending on dataset, model and selector.
Tuning the dial automatically
The paper set the freedom degree by hand and listed automating it as future work. The later implementation adds server-side hill climbing. The server proposes a mask, each site scores it on its own validation data and returns one number, and the server adjusts direction and step size, stops when the score plateaus, and keeps the best mask it has seen.
In an NVIDIA FLARE sweep on a synthetic table with 100 columns and 3 sites using different selectors, the union kept 69 columns. Auto-tuning picked fd = 0.5 and kept 43 columns, 6 of them noise, at an F1 of 0.838. Above a freedom degree of about 0.3 the F1 curve was flat within 0.004, so the tuner found a mask with 38% fewer columns than the union at the same F1, in 5 tuning rounds.
*Validation F1 (top) and columns kept (bottom) across freedom degrees on a 100-column synthetic table with 3 sites. Auto-tuning stopped at `fd = 0.5`. Image by author.*
For an MLP with one 64-unit hidden layer across the 3 sites, that mask takes projected traffic from 153 KB to 68 KB per round (−56%). The election, tuning included, cost 3.4 KB in total, which is repaid in under one training round, and after 20 rounds the totals are 2.99 MB against 1.32 MB. These figures are a projection from that run’s parameter counts, not a measurement on a production network, where protocol overhead, compression and encryption will change them.
What an independent benchmark found
Everything so far comes from my own experiments. A paper by Sharma et al. (2026) from VTT and the University of Eastern Finland, accepted for the MICCAI 2026 workshop proceedings, gives an outside view. It introduces LAFFS, a federated LASSO method of their own, and the one existing federated feature selection method they benchmark it against is Feature Election. They ran the open-source NVIDIA FLARE example as published, with Lasso as the local selector and the freedom degree set by auto-tuning in five hill-climbing rounds. The other baselines were L1 logistic regression from scikit-learn run inside NVIDIA FLARE, a model trained by each site alone, and a centralized model on the pooled data.
The conditions are much harder than in the FLASH paper (Christofilogiannis et al., 2025). The four benchmark datasets have 5,966 to 19,993 features and only 72 to 187 samples, split across 5 to 18 sites. The table shows their reported AUROC, averaged over outer cross-validation folds. For Feature Election I show the best of its three downstream classifiers on each dataset, while the research paper reports all three.
|
Method |
ALLAML (5 sites) |
Prostate_GE (10 sites) |
SMK_CAN_187 (18 sites) |
leukemia (5 sites) |
|---|---|---|---|---|
|
Each site alone |
0.747 |
0.582 |
0.529 |
0.663 |
|
L1 logistic regression in FLARE |
0.968 |
0.925 |
0.669 |
0.983 |
|
Feature Election (Lasso, auto-tuned) |
0.985 |
0.922 |
0.710 |
0.934 |
|
LAFFS |
0.987 |
0.947 |
0.702 |
0.996 |
|
Centralized on pooled data |
0.987 |
0.964 |
0.711 |
0.974 |
LAFFS had the highest federated score on three of the four datasets. Feature Election had the highest on SMK_CAN_187, the largest setting with 18 sites and close to 20,000 features, where it reached 0.710 against 0.711 for pooled training. It was within 0.002 of LAFFS on ALLAML, level with the L1 baseline on Prostate_GE, and clearly behind on leukemia (0.934 against 0.996). The spread across folds is large, with standard deviations between 0.01 and 0.2, so most gaps between the federated methods are within one standard deviation. Every federated method was far ahead of sites working alone.
The paper also reports a weakness. On a synthetic table with 20 informative features out of 200, LAFFS selected only informative ones, while Feature Election with Lasso let some uninformative features through (fewer than with its other selectors). That is the authors’ argument for optimizing one sparse model jointly, and on selection purity their result supports it.
My reading is that a general voting scheme, run by an independent team, stayed close to a purpose-built method and to pooled training at six times the number of sites I had tested. The trade-off is in what each method assumes. LAFFS is built around LASSO logistic regression, and its authors disclose a patent application on the algorithm. Feature Election works with any local selector and model, and is open source under Apache 2.0.
Run it today
There are two ports of the same election rule, one for each framework. The Flower version is a Flower Hub app, @christofilojohn/feature-election [2]; create it with flwr new @christofilojohn/feature-election and run it in local simulation with flwr run .. The NVIDIA FLARE version is the module nvflare.app_opt.feature_election, with a runnable example in the NVFlare repository from release 2.8.0 onwards [9]. Both are open source under Apache 2.0, and the READMEs list the supported selectors, settings and deployment options.
Limitations
My published evaluation used three clients and trained only with FedAvg. The independent benchmark above went to 18 sites, but scaling beyond that and strongly non-IID data are untested. In that benchmark a jointly optimized method also selected fewer noise features than the election did. There is no formal proof for the algorithm, for example of how close the elected mask comes to what a selector on the pooled table would choose. The paper did not report variance across runs or convergence rates, so small F1 differences in the tables above should be read as single-configuration results. On privacy, masks and scores reveal far less than raw rows, but they still reveal something about each site’s data, and unlike differential privacy, this is not a formal guarantee.
History and future
The FLASH paper received the Best Student Paper Award at IEEE FLTA 2025 in Dubrovnik. I showed the mixed-selector results as a poster at the Flower AI Summit 2026 in London (15–16 April 2026) and gave the talk “Feature Election in NVFlare: Private feature selection for tabular data” at NVIDIA FLARE Day 2026 (24 September 2026, virtual). Next is the Federate AI Summit in Boston on 19–20 October 2026, where I am giving a short talk about Feature Election and the next steps. If you have a tabular problem split across sites, or a single table you can split to simulate one, run feature election on it and compare the elected mask with the one your usual selector picks on the pooled table.
Citing this work
If you use or build on Feature Election in your research, please cite the FLASH paper (Christofilogiannis et al., 2025), where the algorithm was introduced.
References
1. I. Christofilogiannis, G. Valavanis, A. Shevtsov, I. Lamprou, S. Ioannidis, “FLASH: A Framework for Federated Learning with Attribute Selection and Hyperparameter Optimization,” 2025 3rd International Conference on Federated Learning Technologies and Applications (FLTA), Dubrovnik, Croatia, 2025, pp. 93–100. https://doi.org/10.1109/FLTA67013.2025.11336571
2. Feature Election app on Flower Hub. https://flower.ai/apps/christofilojohn/feature-election
3. H. B. McMahan, E. Moore, D. Ramage, S. Hampson, B. Agüera y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” AISTATS, PMLR, 2017, pp. 1273–1282.
4. R. Tibshirani, “Regression Shrinkage and Selection via the Lasso,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, 1996.
5. T. Yasuda, M. Bateni, L. Chen, M. Fahrbach, G. Fu, V. Mirrokni, “Sequential Attention for Feature Selection,” ICLR, 2023. https://arxiv.org/abs/2209.14881
6. A. Hassan, J. H. Paik, S. R. Khare, S. A. Hassan, “A wrapper feature selection approach using Markov blankets,” Pattern Recognition, vol. 158, 111069, 2025.
7. D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusmão, N. D. Lane, “Flower: A Friendly Federated Learning Research Framework,” arXiv:2007.14390, 2020. https://arxiv.org/abs/2007.14390
8. H. R. Roth, Y. Cheng, Y. Wen, I. Yang, Z. Xu, Y.-T. Hsieh, et al., “NVIDIA FLARE: Federated Learning from Simulation to Real-World,” arXiv:2210.13291, 2022. https://arxiv.org/abs/2210.13291
9. NVIDIA FLARE Feature Election example (NVFlare 2.8.0 and later). https://github.com/NVIDIA/NVFlare/tree/main/examples/advanced/feature_election
10. G. Sharma, E. Moradi, J. Pajula, M. Hilvo, J. Tohka, “LAFFS: LASSO Assisted Federated Feature Selection,” in Medical Image Computing and Computer Assisted Intervention – MICCAI 2026 Workshops and Challenges, LNCS 17252, Springer Nature Switzerland, 2026 (in press, https://papers.miccai.org/miccai-2026-sat/DeCaF_015.html).
11. S. Banerjee, E. Elmroth, M. Bhuyan, “Fed-FiS: A Novel Information-Theoretic Federated Feature Selection for Learning Stability,” International Conference on Neural Information Processing (ICONIP), 2021.
12. J. Hermo, V. Bolón-Canedo, S. Ladra, “Fed-mRMR: A lossless federated feature selection method,” Information Sciences, vol. 669, 120609, 2024. https://doi.org/10.1016/j.ins.2024.120609
13. A. Li, H. Peng, L. Zhang, J. Huang, Q. Guo, H. Yu, Y. Liu, “FedSDG-FS: Efficient and Secure Feature Selection for Vertical Federated Learning,” IEEE INFOCOM, 2023.
14. S. Zhang, H. Jin, Z. Ye, J. Yang, J. Zhang, D. Wu, X. Zheng, D. Song, “Federated multi-label feature selection via dual-layer hybrid breeding cooperative particle swarm optimization with manifold and sparsity regularization,” Computers, Materials & Continua, vol. 86, no. 1, 2026. https://doi.org/10.32604/cmc.2025.068044
Disclosure: the FLASH work was funded by the Horizon Europe project SYNAPSE (Grant Agreement No. 101120853). I am the first author of FLASH, and the author of both the Flower Hub app and the NVIDIA FLARE contribution described here. All images are by the author. Tables were generated by Python code and some were enhanced for the article using GenAI (Claude).