Probabilistic Inference and Learning Of Things (PILOT)
Overview: My current research interests are in the broad areas of machine learning, data science, signal processing, and communications. Specifically, my current focus is on robust and adaptive Bayesian methods for uncertainty-aware decision-making using sequential and/or networked data, including but not limited to i) uncertainty quantification for high-stakes predictions using Gaussian processes, ii) Bayesian optimization and active learning, iii) Bayesian reinforcement learning, and iv) spatio-temporal inference over graphs, which find exciting applications in autonomous and networked systems. In the past, I have also worked on underwater acoustic communications, multi-target tracking, and multi-sensor fusion.
The overarching goal of my research is to create a mutually promoting ecosystem among theory, algorithms, and applications.
Ensemble Gaussian processes for online prediction with scalability, adaptivity, and robustness
Function approximation is a critical algorithmic module for numerous tasks in ML and AI. GPs are established Bayesian nonparametric framework of learning functions with uncertainty quantification, which is extremely appealing for safety-critical applications. Along with the merits of Bayesian approaches in handling uncertainty and prior information, nonparametric GPs also inherit their limitations, such as cubic complexity in the number of training samples that will not be affordable in the big data regime. In addition, the uncertain and volatile environments necessitate scalable learning algorithms that are robust to noise and adversaries, as well as adaptive to unknown dynamics and other sources of uncertainty.
In this context, I proposed to learn the unknown function via an ensemble (E) of GP learners, each associated with a unique kernel (covariance) belonging to a prescribed dictionary. Such a diversity of function spaces endows the sought learning task with adaptivity and robustness. To further effect scalability, each learner leverages the random feature (RF) based approximation of the kernel function to convert the nonparametric learning to a parametric one. Considering real-time processing, the posterior pdf of RF based parameter vector per learner is the information bottleneck that is used for the uncertainty-aware prediction on-the-fly. The EGP meta-learner then forms the ensemble decision as a weighted combination of predictions from the individual learners, where the per-learner weight is updated in a data-adaptive fashion. Dynamics of the learning environment can be accounted for by constructing structured evolution models for the weight vector at the EGP meta-learner and parameter vectors at the GP learners. To further demonstrate the robustness of EGP-based online predictor to the adversarial setting, I have conducted the so-termed regret analysis, widely adopted in online convex optimization. The proposed approach also empirically outperforms existing alternatives in real prediction problems.
Related publications:
J14) Q. Lu, G. V. Karanikolas, and G. B. Giannakis, "Incremental Ensemble Gaussian Processes," submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence, revised Jan. 2022.
C14. Q. Lu, G. V. Karanikolas, Y. Shen, and G. B. Giannakis, "Ensemble Gaussian Processes with Spectral Features for Online Interactive Learning with Scalability," Proc. of 23rd Intl. Conf. on Artificial Intelligence and Statistics, Palermo, Italy, June 3-5, 2020.
C17. G. V. Karanikolas, Q. Lu, and G. B. Giannakis, "Online Unsupervised Learning using Ensemble Gaussian Processes with Random Features," Proc. of Intl. Conf. on Acoustics, Speech, and Signal Processing, Toronto, Canada, June 6-11, 2021.
Adaptive and Robust Bayesian optimization
A number of ML and AI applications boil down to optimizing an `expensive-to-evaluate' black-box function, including hyperparameter tuning, drug discovery, and policy optimization in robotics. As in hyperparameter tuning, lack of analytic expressions for the objective function and overwhelming evaluation cost discourage grid search, and adoption of gradient-based solvers. To find the global optimum under a limited evaluation budget, Bayesian optimization (BO) offers a principled framework by leveraging a statistical model to guide the acquisition of query points that can strike a balance between exploring the unknown function space and exploiting available evaluation information.
Most existing works rely on a single GP based surrogate model, whose kernel function is typically preselected using domain knowledge. To bypass such a design process, our work considers an EGP prior for the unknown function to adaptively select the surrogate model fit on-the-fly. Acquisition of the next evaluation input using this EGP-based function posterior is then enabled by Thompson sampling (TS) that requires no additional design parameters. To endow function sampling with scalability, RF-based kernel approximation is capitalized on for each GP model. Our proposed EGP-TS readily accommodates parallel operation that allows for concurrent function evaluations so as to reduce the convergence time. To further establish the convergence of EGP-TS to the global optimum, I have conducted theoretical analysis via the notion of Bayesian regret for both the sequential and parallel settings. The merits of the proposed EGP-TS have been validated using tests on synthetic functions and real hyperparameter tuning task of a feedforward neural network.
Related work:
C21) Q.Lu, K. D. Polyzos, B. Li, and G. B. Giannakis, “Surrogate modeling for Bayesian optimization beyond a single GP," submitted to 36th AAAI Conf. on Artificial Intelligence (AAAI), 2021.
Model-Free Bayesian reinforcement learning with application to IoT resource allocation
Markov decision processes (MDPs) offer a powerful tool for tackling interactive sequential decision-making problems under uncertainty. Recent efforts have been devoted to model-free MDP approaches, where the underlying Markovian dynamical model is unknown --- the central concept of RL . As the hardcore of contemporary AI, RL has a plethora of applications including robotics, computer games, and self-driving cars. While most RL methods view the unknown MDP as deterministic, Bayesian RL considers a random MDP and incorporates expert information in the prior pdf, thus offering a more robust learning paradigm. Nevertheless, decision still has to be made beforehand concerning which prior to select. This model selection task is especially challenging in the dynamically changing and volatile environments.
In this context, we consider adaptive prior selection for a family of model-free value-based Bayesian RL algorithms using GPs. Instead of postulating the sought value function using a single GP with a preselected kernel, I proposed to consider an EGP prior with adaptive model selection through the online interaction between the environment and the agent. Our work proves the robustness of the proposed EGP-based value function learning approach in the worst-case scenario. When the state-action function is sought, I have also developed a novel EGP-based SARSA approach, which, coupled with BO for action selection with the tradeoff between exploration and exploitation, has been applied to a practical and challenging resource allocation problem for Internet-of-Things.
Related publications:
C16) Q. Lu and G. B. Giannakis, "Gaussian Process Temporal-difference Learning with Scalability and Worst-Case Performance Guarantees," Proc. of Intl. Conf. on Acoustics, Speech, and Signal Processing, Toronto, Canada, June 6-11, 2021.
C20) K. D. Polyzos, Q. Lu, and G. B. Giannakis, "Ensemble Gaussian Processes over Egonet Features for Online Graph-Guided Learning," Proc. of Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, Oct. 31-Nov. 3, 2021.
J13) Q. Lu, K. D. Polyzos, and G. B. Giannakis, “Ensemble of Gaussian Processes for Temporal-Difference Learning with Scalability and Robustness,” submitted to Journal of Machine Learning Research, 2021.
Spatio-temporal inference over graphs.
Spatio-temporal inference is of great practical significance, that has found applications in a number of fields, including sociology, biology, neuroscience and economics. With spatial interdependency captured by a graph, the spatio-temporal signals refer to time-varying feature vectors over all the nodes in the graph. However, due to limited sampling cost or privacy considerations, only a subset of nodal values are observed, yielding the semi-supervised learning (SSL) task that aims to reconstruct all the nodal features. For example, individuals in social networks may be reluctant to share personal information, while acquiring nodal samples in brain networks may require invasive procedures such as electrocorticography.
Although many attempts have been made to cope with spatio-temporal inference, there remain several critical challenges. How to build an {\it interpretable} model that can account for spatial and temporal correlations is essential to the extrapolation task given the available information. Further, many spatio-temporal inference problems call for real-time processing, including traffic and weather forecasting, as well as COVID cases prediction. This online operation model entails scalable solvers with data accumulated on-the-fly. Towards addressing theses challenges, I advocate a Gaussian process (GP) based dynamical model for online scalable spatio-temporal SSL over graphs.
Related publications:
C18) Q. Lu and G. B. Giannakis, "Spatio-Temporal Inference of Dynamical Gaussian Processes over Graphs," Proc. of Asilomar Conf. on Signals, Systems, and Computers, Pacific Grove, CA, Oct. 31-Nov. 3, 2021.
J12) Q. Lu and G. B. Giannakis, “Online Spatio-Temporal Inference over Graphs via Gaussian Process Dynamical Modeling,” submitted to IEEE Transactions on Signal and Information Processing over Networks, 2021.
J10) Q. Lu and G. B. Giannakis, "Probabilistic Reconstruction of Spatiotemporal Processes over Multi-relational Graphs," IEEE Transactions on Signal and Information Processing over Networks, vol. 7, pp. 166-176, April 2021.
J9) Q. Lu, V. N. Ioannidis, and G. B. Giannakis, "Graph-adaptive semi-supervised tracking of dynamic processes over switching network modes," IEEE Transactions on Signal Processing, vol. 68, no. 1, pp. 2586-2597, December 2020.