Zhaoye Pan

publications

1. Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach

   Hongye Wang, Zhaoye Pan, Chang He, Jiaxiang Li, Bo Jiang

   arXiv 2507.22855, 2025

   Abs HTML

   collaborative model training across distributed clients while preserving data privacy. However, existing FL algorithms predominantly focus on unconstrained optimization problems with exact gradient information, limiting its applicability in scenarios where only noisy function evaluations are accessible or where model parameters are constrained. To address these challenges, we propose a novel zeroth-order projection-based algorithm on Riemannian manifolds for FL. By leveraging the projection operator, we introduce a computationally efficient zeroth-order Riemannian gradient estimator. Unlike existing estimators, ours requires only a simple Euclidean random perturbation, eliminating the need to sample random vectors in the tangent space, thus reducing computational cost. Theoretically, we first prove the approximation properties of the estimator and then establish the sublinear convergence of the proposed algorithm, matching the rate of its first-order counterpart. Numerically, we first assess the efficiency of our estimator using kernel principal component analysis. Furthermore, we apply the proposed algorithm to two real-world scenarios: zeroth-order attacks on deep neural networks and low-rank neural network training to validate the theoretical findings.

2. Riemannian accelerated zeroth-order algorithm: improved robustness and lower query complexity

   Chang He, Zhaoye Pan, Xiao Wang, Bo Jiang

   Proceedings of the 41st International Conference on Machine Learning 2024

   Abs HTML

   Optimization problems with access to only zeroth-order information of the objective function on Riemannian manifolds arise in various applications, spanning from statistical learning to robot learning. While various zeroth-order algorithms have been proposed in Euclidean space, they are not inherently designed to handle the challenging constraints imposed by Riemannian manifolds. The proper adaptation of zeroth-order techniques to Riemannian manifolds remained unknown until the pioneering work of \cite{li2023stochastic}. However, zeroth-order algorithms are widely observed to converge slowly and be unstable in practice. To alleviate these issues, we propose a Riemannian accelerated zeroth-order algorithm with improved robustness. Regarding efficiency, our accelerated algorithm has the function query complexity of $\mathcal{O}(\epsilon^{-7/4}d)$ for finding an $\epsilon$-approximate first-order stationary point. By introducing a small perturbation, it exhibits a function query complexity of $\tilde{\mathcal{O}}(\epsilon^{-7/4}d)$ for seeking a second-order stationary point with a high probability, matching state-of-the-art result in Euclidean space. Moreover, we further establish the almost sure convergence in the asymptotic sense through the Stable Manifold Theorem. Regarding robustness, our algorithm requires larger smoothing parameters in the order of $\tilde{\mathcal{O}}(\epsilon^{7/8}d^{-1/2})$, improving the existing result by a factor of $\tilde{\mathcal{O}}(\epsilon^{3/4})$.

3. An Efficient Hierarchical Block Coordinate Descent Method for Time-Varying Graphical Lasso

   Zhaoye Pan, Xiaolu Wang, Huikang Liu, Jun Zhang

   In 2024 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP) 2024

   Abs HTML

   Time-varying graphical LASSO (TVGL) aims to infer a sequence of graphs from time series data and has been widely used in many statistical inference problems. The existing algorithms usually suffer from high computational cost when solving large-scale TVGL problems. In this paper, we develop an efficient and scalable hierarchical block coordinate descent (HBCD) method for solving TVGL with smooth temporal difference prior. The proposed HBCD method contains both outer-loop and inner-loop BCD iterations. The outer loops seperate the original TVGL problem into a sequence of subproblems, which are variants of the static graphical LASSO problems. Then, we propose an efficient BCD method to solve the inner-loop subproblems. We provide theoretical analysis that indicates the linear convergence of our proposed method. Furthermore, numerical experiments on both synthetic and real datasets show that our method significantly outperforms the state-of-the-art algorithm in terms of both the required iterations and CPU time to reach the target precision.

4. Utilizing second-order information in noisy information-sharing environments for distributed optimization

   Zhaoye Pan, Haoqi Yang, Huikang Liu

   In 2024 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP) 2024

   Abs HTML

   Decentralized optimization aims to cooperatively solve a global finite-sum loss function, where each agent only possesses knowledge of its own local function. Real-world applications introduce challenges such as unstable channels and differential privacy concerns, necessitating the development of more robust algorithms. This paper proposes a framework that explores second-order information using two approaches: global Newton tracking and local Newton preconditioning. Furthermore, we adapt a generic convergence result for gradient tracking methods to demonstrate the almost sure convergence property of both schemes under specific conditions. To validate the effectiveness of the proposed algorithm, numerical experiments are conducted on synthetic and real datasets, showcasing its robustness and superior accuracy compared to existing first-order methods.

5. A Fully Asynchronous Newton Tracking Method for Decentralized Optimization

   Zhaoye Pan, Huikang Liu

   In 2023 62nd IEEE Conference on Decision and Control (CDC) 2023

   Abs HTML

   We consider fully asynchronous decentralized optimization over a directed graph. While various algorithms have been proposed, real-world applications require relaxing the assumption and considering communication networks with asynchronous and heterogeneous nodes. To meet these challenges, we propose an efficient and robust Newton tracking mechanism for fully asynchronous optimization. Our proposed mechanism can be adapted to different asynchronous first-order methods as required by the practical context. Through the theoretical analysis we demonstrate the R-linear rate of our method and derive an explicit expression of decaying factor under local conditions. Furthermore, numerical comparison with existing algorithms support the efficiency and robustness of our method.

6. Ind.

   Customized Energy-Optimization Solver

   Zhaoye Pan (Research Intern)

   Cardinal Operations    Jan 2023 – Jan 2024

   Designed and implemented high-performance MILP heuristics and cutting-plane strategies tailored for large-scale electricity-generation scheduling.

7. Ind.

   Large-Scale Power-System Optimization with Language & Graph Models

   Zhaoye Pan (Research Intern)

   Alibaba DAMO Academy    Mar 2024 – Jun 2024

   Built a domain-specific LLM pipeline that translates verbal dispatch instructions into structured optimization models.