An Accurate and Single-Communication Federated Inference Algorithm

The evolution of multi-center studies in medicine and public health emphasizes the importance of collaborative analysis. Traditional pooled data analysis faces privacy and governance barriers, prompting the development of federated learning (FL) methods like FedAvg, which exchange model parameters iteratively. While effective, these methods are communication-intensive and less accurate with small or skewed datasets. Bayesian federated inference (BFI) approaches, such as BFI2, improve privacy and efficiency by sharing local posterior summaries, but rely on second-order approximations, which are inadequate for small or skewed data. Recent efforts like ODAL reduce communication but still depend on quadratic approximations, limiting accuracy. Addressing these limitations, this paper proposes a high-order (third-order) Taylor expansion method, aiming for more precise inference in challenging data scenarios.

Existing federated inference techniques primarily use second-order Taylor approximations, which inadequately capture the local log-likelihood's skewness and higher moments, especially when local samples are small. This leads to biased or imprecise global posteriors, affecting downstream inference and decision-making. Additionally, iterative methods incur high communication costs, and single-round approaches often sacrifice accuracy. The core challenge is to develop a method that enhances approximation fidelity without increasing communication rounds or compromising privacy, particularly under small-sample, skewed data conditions common in rare disease studies.

The key innovation is the integration of third-order derivatives into the Taylor expansion of local log-posterior functions, enabling a more accurate global approximation. Each center computes local MAP estimates and their derivatives up to third order, transmitting these tensors to the server. The server constructs a third-order Taylor expansion, maximizing it to estimate parameters. This approach captures skewness and higher moments, reducing bias inherent in second-order methods. It maintains a single communication round, preserves privacy, and improves finite-sample accuracy, representing a significant step forward in federated statistical inference.

• �� Each site computes local MAP estimate and derivatives: first, second, and third order, of the local log-posterior.
• �� These summaries include the MAP, Hessian matrix, and third derivative tensor, transmitted to the server.
• �� The server constructs a third-order Taylor approximation of the global log-posterior using these derivatives.
• �� The global estimate is obtained by numerically maximizing this third-order expansion.
• �� The approach requires only one communication round, with local sites transmitting derivatives once.
• �� Validation involves simulations with real datasets, comparing MSE and prediction accuracy against existing methods.

Using a dataset of 371 trauma patients, simulations mimic multi-center settings with varying sample sizes and skewness. The algorithms compared include second-order (FI2), third-order (FI3), and their two-round variants, alongside ODAL2. Metrics include MSE for parameter estimates and predictions, evaluated across multiple runs. The experiments assess robustness in small samples, skewed distributions, and imbalanced data, with hyperparameters tuned for fairness. Results demonstrate the superiority of third-order methods in accuracy and stability, especially in challenging scenarios.

Third-order Taylor expansion consistently outperforms second-order in parameter estimation and prediction accuracy, reducing MSE by 20-30%. In small, skewed datasets, the method maintains stability and low bias, outperforming ODAL2, which shows instability and higher errors. Two-round communication further refines estimates, with error reductions of over 15%. Real data analysis confirms the method’s robustness, making it suitable for sensitive applications like rare disease studies, where data scarcity and skewness are prevalent.

The method is ideal for privacy-sensitive, multi-center clinical research, especially in rare diseases with limited samples. It enables accurate joint analysis without data sharing, facilitating collaborative studies, meta-analyses, and personalized medicine. Its efficiency makes it suitable for large-scale public health surveillance, where data heterogeneity and privacy are critical. Future extensions could include deep learning models and non-i.i.d. data scenarios, broadening its impact.

Computationally, calculating and transmitting third derivatives (tensors) is intensive, especially in high-dimensional settings. Storage and processing costs increase significantly. The approximation may still be biased in highly heterogeneous data. Extending to complex models like neural networks remains challenging. Further research is needed to optimize tensor compression, handle non-i.i.d. data, and reduce computational overhead for practical deployment.