CoChat
Sign Up
Cited by 1,355Privacy-Preserving Technologies in DataStochastic Gradient Optimization TechniquesDomain Adaptation and Few-Shot Learning

Clustered Federated Learning: Model-Agnostic Distributed Multitask Optimization Under Privacy Constraints

Open access
Sattler, Felix, Muller, Klaus-Robert, Samek, Wojciech·2021·IEEE Transactions on Neural Networks and Learning Systems
DOI 10.1109/tnnls.2020.3015958
Open this paper in CoChat — free accountOpen PDF

Abstract

Federated learning (FL) is currently the most widely adopted framework for collaborative training of (deep) machine learning models under privacy constraints. Albeit its popularity, it has been observed that FL yields suboptimal results if the local clients' data distributions diverge. To address this issue, we present clustered FL (CFL), a novel federated multitask learning (FMTL) framework, which exploits geometric properties of the FL loss surface to group the client population into clusters with jointly trainable data distributions. In contrast to existing FMTL approaches, CFL does not require any modifications to the FL communication protocol to be made, is applicable to general nonconvex objectives (in particular, deep neural networks), does not require the number of clusters to be known a priori, and comes with strong mathematical guarantees on the clustering quality. CFL is flexible enough to handle client populations that vary over time and can be implemented in a privacy-preserving way. As clustering is only performed after FL has converged to a stationary point, CFL can be viewed as a postprocessing method that will always achieve greater or equal performance than conventional FL by allowing clients to arrive at more specialized models. We verify our theoretical analysis in experiments with deep convolutional and recurrent neural networks on commonly used FL data sets.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces clustered federated learning (CFL), a model-agnostic federated multitask framework that recursively bipartitions clients using cosine similarity of weight-updates at a stationary FL solution, without changing the FL protocol or requiring the number of clusters a priori. Experiments on deep convolutional and recurrent networks show large gains over standard FL when client data distributions are incongruent.

Key findings

  • At a stationary FL solution, cosine similarity between client updates separates hidden distributions: under equality assumptions, same-cluster similarity is 1 and different-cluster similarity is −1; Theorem 1 bounds cross-cluster and intra-cluster similarities, and a positive separation gap \(g(\alpha)>0\) guarantees correct bipartitioning.
  • CFL only splits incongruent clients: it checks whether the average update norm is near zero and the maximum client update norm exceeds \(\epsilon_2\); in experiments, the maximum client norm converged to zero for congruent clients but stagnated or increased for incongruent clients over 1000 communication rounds.
  • On CIFAR-10 with \(M=20\) clients and \(K=4\) clusters, FL accuracy stagnated around 20%; after CFL splits at rounds 50, 100, and 150, accuracy more than doubled to close to 60%.
  • On Ag-News with 20 clients and four topics, FL perplexity stagnated around 42–43; after splits at rounds 30, 60, and 90, client perplexity dropped to less than 36. On MNIST label-swap, correct bipartitioning was achieved with as few as 20 data points per client using weight-updates, while CIFAR required around 500; correct separation occurred after about 10 communication rounds.

Methods

  • Theoretical design: formalizes the FL congruence assumption, generalizes it to CFL Assumption 2, derives a separation theorem based on cosine similarity of gradients/weight-updates, and presents a recursive top-down bipartitioning algorithm with \(O(M^3)\) clustering overhead.
  • Experiments: MNIST and CIFAR label-swap problems with \(M=20\) clients, \(K=4\) clusters, and multilayer convolutional networks; Ag-News language modeling with 20 clients, four topic clusters, and a two-layer LSTM; three local epochs and batch size 100 in CFL experiments.
  • Analysis: compares gradient-based versus weight-update-based cosine similarity, varies local data points (10–200 on MNIST, 100–2400 on CIFAR) and communication rounds, and verifies the norm-based splitting criteria and separation gap.

Limitations

  • Paper-stated: future work is needed to explore CFL with differential privacy mechanisms, parameter update compression, and explainable AI; the paper also notes that CFL exposes a privacy issue because client data similarity can be inferred from weight-updates, though it argues the privacy loss is tolerable in most situations.
  • Paper-stated: CFL relies on assumptions about stationary solutions, empirical risk approximation, and the separation gap, and its splitting behavior depends on hyperparameters \(\epsilon_1\) and \(\epsilon_2\), for which the paper gives practical recommendations rather than a fully automatic selection rule.
  • Evident from the text: the experiments use synthetic label-swap/permuted-label constructions and topic-split Ag-News rather than naturally occurring heterogeneous client data, and no formal differential privacy guarantees are evaluated experimentally.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2020.3015958

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 12 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § Remark 1:

    For d=10^2, M=3K, and normally distributed gradients, the probability of correct clustering is approximately 1 even for large K>10 and γ_max>1.

    “if we set d = 10 2 (which is still many orders of magnitude smaller than typical modern neural networks), M = 3K , and assume ∇ R I (i) (θ * ) and ∇ R I (i) (θ * ) -∇r i (θ * ) to be normally distributed for all i = 1, . . ., M; then, experimentally, we find (see Fig. 3) that P[”
    Quote in the paperNot a substantial claim
  • Findingown data
    § Algorithm 5 CFL With Privacy Preservation and Weight-Updates

    On the MNIST problem, as little as 20 data points on every client are sufficient to achieve correct bipartitioning in the sense of Definition 1.

    “On the MNIST problem, as little as 20 data points on every client are sufficient to achieve correct bipartitioning in the sense of Definition 1.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § A. Weight-Updates as Generalized Gradients

    Computing cosine similarities based on weight-updates in practice achieves even better separations than computing cosine similarities based on gradients.

    “Our experiments in Section V will demonstrate that computing cosine similarities based on weight-updates, in practice, surprisingly achieves even better separations than computing cosine similarities based on gradients.”
    Quote in the paperNot a substantial claim
  • Findingreasoning
    § Algorithm 5 CFL With Privacy Preservation and Weight-Updates

    From Corollary 1, CFL will always find a correct bipartitioning if g(α) > 0.

    “From Corollary 1, we know that CFL will always find a correct bipartitioning if g(α) > 0.”
    Quote in the paperSubstantial claim
  • Findingown data
    § Algorithm 5 CFL With Privacy Preservation and Weight-Updates

    The separation gap g(α) grows monotonically with increasing data set size.

    “As expected, g(α) grows monotonically with increasing data set size.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 20
  • Classified as probably not a substantial claim. 17
  • The quote only partly states this claim. 2

  • May combine two claims.

    Findingreasoning
    § A. Practical Considerations

    The cosine similarity criterion distinguishes different incongruent clients under three conditions: FL has converged to a stationary point; every client holds enough data that the empirical risk approximates the true risk; and cosine similarity is computed between the full gradients of the empirical risk.

    “In Section II-A, we showed that the cosine similarity criterion does distinguish different incongruent clients under three conditions: 1) FL has converged to a stationary point θ * ; 2) every client holds enough data subject to the empirical risk approximates the true risk; and 3) cosine similarity is computed between the full gradients of the empirical risk.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Remark 1:

    In practice, α_min_intra typically will be much larger and α_max_cross typically much smaller than the worst-case estimate, especially if the parameter dimension d is large.

    “In practice, α min intra typically will be much larger, and α max cross typically will be much smaller, especially if the parameter dimension d := dim() is large.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given
    § A. Practical Considerations

    Cosine similarity can be computed between weight-updates instead of full gradients, which improves performance.

    “Furthermore, we will see that cosine similarity can be computed between weight-updates instead of full gradients, which even improves the performance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § B. Distinguishing Congruent and Incongruent Clients

    In both congruent and incongruent cases, the average update norm tends to zero, indicating convergence to a stationary point.

    “In both cases, the average update norm tends to zero, indicating convergence to a stationary point (see Fig. 8).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § Image Classification on CIFAR-10:

    In the CIFAR-10 experiment, after the initial 50 communication rounds, training has converged to a stationary point of the FL objective, and client test accuracies stagnate at around 20%.

    “After these initial 50 rounds, training has converged to a stationary point of the FL objective, and the client test accuracies stagnate at around 20%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Algorithm 5 CFL With Privacy Preservation and Weight-Updates

    Weight-updates provide even better separation g(α) with fewer data points and at a greater distance to a stationary solution than full gradients.

    “Interestingly, weight-updates seem to provide even better separation g(α) with fewer data points and at a greater distance to a stationary solution.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § B. Distinguishing Congruent and Incongruent Clients

    In the incongruent case, the maximum client norm stagnates and even increases over time.

    “As predicted by the theory, in the congruent case, the maximum client norm converges to zero, while, in the incongruent case, it stagnates and even increases over time.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § Assumption 2 (CFL):

    By identifying the hidden clustering structure, CFL allows clients with similar data to profit from one another while minimizing harmful interference between clients with dissimilar data.

    “By identifying the hidden clustering structure, CFL allows clients with similar data to profit from one another while minimizing the harmful interference between clients with dissimilar data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Limited Model Complexity:

    An insufficiently expressive model will not be able to fit the data of all clients at the same time.

    “An insufficiently expressive model will not be able to fit the data of all clients at the same time”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § Assumption 1 (FL):

    The FL assumption is violated if clients have disagreeing conditional distributions φ_i(y|x) ≠ φ_j(y|x).

    “or 2) clients have disagreeing conditional distributions ϕ i (y|x) = ϕ j (y|x).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § Assumption 1 (FL):

    The FL assumption is violated if the model fθ is not expressive enough to fit all distributions at the same time.

    “Concretely, it is violated if either: 1) the model f θ is not expressive enough to fit all distributions at the same time”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § Assumption 2 (CFL):

    CFL is able to deal with FMTL problems that satisfy Assumption 2.

    “we present clustered FL (CFL), a novel algorithmic framework that is able to deal with FMTL problems that satisfy Assumption 2.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § B. Distinguishing Congruent and Incongruent Clients

    If client distributions are congruent, federated optimization will jointly optimize all clients' local risk functions, and the norm of the clients' gradients will tend toward zero as the stationary point is approached.

    “If, conversely, the client distributions are congruent, federated optimization will be able to jointly optimize all clients' local risk functions, and hence, the norm of the clients' gradients will tend toward zero as we are approaching the stationary point.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § C. Algorithm

    The optimal bipartitioning problem at the core of CFL can be solved in O(M^3) using Algorithm 1.

    “This optimal bipartitioning problem at the core of CFL can be solved in O(M 3 ) using Algorithm 1.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § C. Algorithm

    Ghosh et al.'s method is not able to distinguish congruent from incongruent settings and will incorrectly split up clients in the conventional congruent non-i.i.d. setting.

    “Their method also is not able to distinguish congruent from incongruent settings.This means that the method will incorrectly split up clients in the conventional congruent non-i.i.d.setting described in [1].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10]

      Not checked in the free run.

    • cites [1]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § A. Cosine Similarity-Based Bipartitioning

    Theorem 1 also bounds the minimum intra-cluster cosine similarity from below by (19).

    “At the same time, the similarity between the updates from clients that share the same data generating distribution can be bounded from below by α min intra := min i, j I (i)=I ( j ) α(∇ θ r i (θ * ), ∇ θ r j (θ * )) ≥ min i, j I (i)=I ( j ) H i, j .”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The local training data of each client is sampled from one of K data generating distributions.
    • The authors state this as an assumption: The empirical risk on every client approximates the true risk at every stationary solution of the FL objective subject to condition (15).

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationown data
    § Assumption 2 (CFL):

    CFL can seamlessly adapt to varying client populations, training with formal privacy guarantees, and communication of weight-updates instead of gradients.

    “we investigate several practical concerns (varying client populations, training with formal privacy guarantees, and communication of weight-updates instead of gradients) and demonstrate that CFL can seamlessly adapt to these conditions/constraints”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § B. Preserving Privacy

    Other multitask learning approaches require direct access to the client's data and cannot be used together with encryption, so CFL has a distinct advantage in privacy-sensitive situations.

    “Other multitask learning approaches require direct access to the client's data and, hence, cannot be used together with encryption, which means that CFL has a distinct advantage in privacy-sensitive situations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § C. Algorithm

    If criterion (36) is satisfied, CFL recursively reapplied to each of the two separate groups starting from θ*; splitting continues until none of the subclusters violate the stopping criterion, at which point all groups of mutually congruent clients have been identified and the CFL problem characterized by Assumption 2 is solved.

    “If criterion (36) is satisfied, CFL is recursively reapplied to each of the two separate groups starting from the stationary solution θ * .Splitting recursively continues on until (after at most K -1 recursions) none of the subclusters violate the stopping criterion anymore; at that point, all groups of mutually congruent clients C = {c 1 , . . ., c K } have been identified, and the CFL problem”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § A. Cosine Similarity-Based Bipartitioning

    In the incongruent two-distribution case, the cosine similarity between gradient updates of any two clients is 1 if they share the same data generating distribution and -1 if they do not.

    “= 1, if I (i ) = I ( j ) -1, if I (i ) = I ( j ).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § Assumption 2 (CFL):

    A criterion can be derived that ensures clustering only takes place in the incongruent case, where benefits can be expected.

    “We address the question of when to cluster and derive a criterion that ensures that clustering only takes place in the incongruent case, where benefits can be expected”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § B. Preserving Privacy

    Both the cosine similarity between two clients' weight-updates and the norms of these updates are invariant to orthonormal transformations P.

    “As both the cosine similarity between two clients' weight-updates and the norms of these updates are invariant to orthonormal transformations P (such as the permutation of the indices)”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § B. Distinguishing Congruent and Incongruent Clients

    Splitting should only take place if the FL objective is close to a stationary point and the individual clients are far from a stationary point of their local empirical risk.

    “Splitting should only take place if it holds that both: 1) we are close to a stationary point of the FL objective”
    • Also stated in: § B. Distinguishing Congruent and Incongruent Clients · plain
      “and 2) the individual clients are far from a stationary point of their local empirical risk”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § C. Varying Client Populations and Parameter Trees

    When a new client joins the training, it can get assigned to a leaf cluster by iteratively traversing the parameter tree from the root to a leaf, always moving to the branch that contains the more similar client updates according to Algorithm 4.

    “When a new client joins the training, it can get assigned to a leaf cluster by iteratively traversing the parameter tree from the root to a leaf, always moving to the branch that contains the more similar client updates according to Algorithm 4.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation
    § C. Algorithm

    Other distributed training frameworks are facing the same issues as conventional FL with incongruent data distributions.

    “Other distributed training frameworks [19]- [22] are facing the same issues.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19]- [22]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § B. Distinguishing Congruent and Incongruent Clients

    CFL will always improve the FL performance or perform equally well at worst.

    “In this sense, CFL will always improve the FL performance (or perform equally well at worst).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § B. Distinguishing Congruent and Incongruent Clients

    When client distributions are incongruent, the stationary solution of the FL objective cannot be stationary for the individual clients, so the norm of the clients' gradients is strictly greater than zero.

    “When client distributions are incongruent, the stationary solution of the FL objective by definition cannot be stationary for the individual clients.Hence, the norm of the clients' gradients has to be strictly greater than zero.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § A. Weight-Updates as Generalized Gradients

    It is commonly conjectured that accumulated minibatch gradients approximate the full-batch gradient of the objective function.

    “It is commonly conjectured (see [28]) that accumulated minibatch gradients approximate the full-batch gradient of the objective function.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [28]

      Not checked in the free run.

  • The quote only partly states this claim.

    Interpretationreasoning
    § A. Cosine Similarity-Based Bipartitioning

    Theorem 1 (Separation Theorem): For M clients with data from K distributions, at a stationary solution satisfying condition (15), there exists a bipartitioning such that the maximum cross-cluster cosine similarity is bounded above by (17).

    “Then, there exists a bipartitioning c * 1 ∪c * 2 = {1, . . ., M} of the client population such that the maximum similarity between the updates from any two clients from different clusters can be bounded from above according to”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The local training data of each client is sampled from one of K data generating distributions.
    • The authors state this as an assumption: The empirical risk on every client approximates the true risk at every stationary solution of the FL objective subject to condition (15).

  • May combine two claims.

    Premisecitation
    § C. Algorithm

    Corinzia and Buhmann's method can handle nonconvex models but is expensive to generalize to large federated networks because client models are refined sequentially.

    “Although their method can handle nonconvex models, it is expensive to generalize to large federated networks as the client models are refined sequentially.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Algorithm

    MOCHA relies on alternating biconvex optimization and is thus only applicable to convex objective functions and limited in its ability to scale to massive client populations.

    “However, their method relies on alternating biconvex optimization and is, thus, only applicable to convex objective functions and limited in its ability to scale to massive client populations.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [9]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § B. Preserving Privacy

    Every machine learning model carries information about the data it has been trained on.

    “Every machine learning model carries information about the data it has been trained on.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Interpretationreasoning
    § C. Algorithm

    α_min_intra can only be estimated using Theorem 1 according to α_min_intra ≥ 1 - 2γ_max^2.

    “α min intra can only be estimated using Theorem 1 according to”
    • Also stated in: § C. Algorithm · plain
      “≥ 1 -2γ 2 max .”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § B. Distinguishing Congruent and Incongruent Clients

    The smaller the value of ε2, the more likely it is that the client population will be separated by CFL.

    “The smaller the value of ε 2 , the more likely it is that the client population will be separated by CFL.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § Algorithm 5 CFL With Privacy Preservation and Weight-Updates

    Using weight-updates allows leaving the FL communication protocol unchanged.

    “This comes in very handy as it allows us to leave the FL communication protocol unchanged.”
    Quote in the paperNot a substantial claim
35 of 35

  • The authors state this as an assumption.

    FL assumes there exists a parameter configuration θ* that locally minimizes the risk on all clients' data generating distributions at the same time.

    “Assumption 1 (FL): There exists a parameter configuration θ * ∈ that (locally) minimizes the risk on all clients' data generating distributions at the same time”

  • The authors state this as an assumption.

    There exists a partitioning C of the client population such that every subset of clients satisfies the conventional FL assumption.

    “Assumption 2 (CFL): There exists a partitioning C = {c 1 , . . ., c K }, K k=1 c k = {1, . . ., M} of the client population, such that every subset of clients c ∈ C satisfies the conventional FL assumption.”

  • The authors state this as an assumption.

    For demonstration purposes, the empirical risk on every client equals the true risk.

    “For demonstration purposes, let us first assume equality in (9).”

  • The authors state this as an assumption.

    In FL, the server has far greater computational power than the clients.

    “Since, in FL, it is assumed that the server has far greater computational power than the clients, the overhead of clustering will typically be negligible.”

  • The authors state this as an assumption.

    All clients participate from the beginning of training.

    “Up until now, we always made the assumption that all clients participate from the beginning of training.”

  • The authors state this as an assumption.

    The local training data of each client is sampled from one of K data generating distributions.

    “Let D 1 , . . ., D M be the local training data of M different clients, each data set sampled from one of K different data generating distributions ϕ 1 , . . ., ϕ K such that D i ∼ ϕ I (i) (x, y).”

  • The authors state this as an assumption.

    The empirical risk on every client approximates the true risk at every stationary solution of the FL objective subject to condition (15).

    “Let the empirical risk on every client approximate the true risk at every stationary solution of the FL objective θ * subject to ∇ R I (i) (θ * ) > ∇ R I (i) (θ * ) -∇r i (θ * )”
7 of 7

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • New-client assignment refers to a nonexistent child

    Minor

    Algorithm 4 names the two children v0 and v1, but its similarity calculation refers to v2. The stated procedure therefore cannot assign a client using the two children it defines.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. alistarh, 2017, QSGD: Communication-efficient SGD via gradient quantization and encoding, Proc Adv Neural Inf Process Syst, 1709
  2. krizhevsky, 2014, CIFAR-10 Dataset
  3. lecun, 1998, The MNIST Database of Handwritten Digits
  4. yu, 2018, arXiv 1807 06629
  5. li, 2019, Federated learning: Challenges, methods, and future directions, arXiv 1908 07873
  6. smith, 2017, Federated multi-task learning, Proc Adv Neural Inf Process Syst, 4424

Cited by

Sorted by citations

Related

Authors
Sattler, F., Muller, K.-R., & Samek, W.
Affiliations
  • Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute (DE)
  • Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany
ORCID
Venue
IEEE Transactions on Neural Networks and Learning Systems
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021-08
Volume
32
Issue
8
Pages
3710-3722
ISSN
  • 2162-237X (ISSN-L)
  • 2162-2388
DOI
10.1109/tnnls.2020.3015958
OpenAlex
W2977814456
Cited by
1,355
References
44
Citations by year
2026: 221 · 2025: 326 · 2024: 319 · 2023: 250 · 2022: 132 · 2021: 83 · 2020: 24
FWCI
47.62
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Neural Networks and Learning Systems (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion, other-oa)
  • Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Berlin Institute for the Foundations of Learning and Data (01IS18025A, 01IS18037)
  • German Research Foundation (EXC 2046/1, 390685689)
  • Information & Communications Technology Planning & Evaluation
  • Korea Government (2017-0-00451)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
clustered federated learning, client clustering, non-IID data, privacy-preserving machine learning, non-convex optimization, deep neural network

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review