Title: Explainable AI for Unsupervised Learning: Turning Raw Data into Scientific Insights

Abstract: Raw data usually comes unsupervised. Turning unsupervised data into supervised data requires labels, which in turn require expert knowledge. Developing methods that are not tied to labels gives more flexibility in conducting various forms of scientific investigations. In this talk, I will present novel Explainable AI techniques, based on layer-wise relevance propagation (LRP), that work with unsupervised data and unsupervised ML models. These techniques formulate questions such as "what makes data dissimilar/similar", "what makes features mutually predictable", and explain by highlighting instances, features or concepts that are relevant. Medical use cases ranging from inferring regulatory networks from omics data to detecting Clever Hans effects in medical foundation models, will be presented to highlight the actionability of unsupervised XAI.

Bio: Prof. Dr. Grégoire Montavon is a Professor at Charité Universitätsmedizin Berlin and a Research Group Lead in the Berlin Institute for the Foundations of Learning and Data (BIFOLD). He received a Master's degree in Communication Systems from the École Polytechnique Fédérale de Lausanne in 2009 and a Ph.D. in Machine Learning from the Technische Universität Berlin in 2013. His research focuses on methods and applications of Explainable AI (XAI). A particular goal is to develop approaches that integrate well with state-of-the-art machine learning (ML) models used in medical diagnosis and research, so that these models can be verified and explored for insights. Together with his colleagues, he contributed the Layer-Wise Relevance Propagation (LRP) method, which efficiently and robustly explains the predictions of popular machine learning models such as deep neural networks.

Title: Barriers to Explainability for Sustainability, and A Path Towards More Reliable Models

Abstract: The path towards a more sustainable future requires accelerating our work in monitoring global change, designing and implementing interventions, and evaluating impact. While the scale of problems are vast, with wildlife populations declining by 73% on average since 1970, we are also seeing unprecedented international willingness and cooperation—which accelerates large-scale action, but also sharpens the urgency of ensuring that the models underpinning these decisions are trustworthy.
Focusing on the global biodiversity crisis as a case study, we’ll discuss the increasing application of machine learning models for monitoring and interventions for sustainability. We’ll highlight critical opportunities where interpretable models are needed across predictive machine learning, sequential planning including reinforcement learning, and impact evaluation, and present insights from existing applications.

Bio: Lily Xu is a postdoc at Oxford with the Leverhulme Centre for Nature Recovery and an incoming assistant professor at Columbia. Her work focuses on advancing methods in machine learning, large-scale planning, and causal inference for planetary health challenges, with a focus on biodiversity conservation. She aims to enable practitioners to make effective decisions in the face of limited data, taking actions that are robust to uncertainty, effective at scale, and future-looking. She partners closely with nongovernmental organizations and interdisciplinary researchers to bridge research and practice, serving as AI Lead for the SMART Partnership to support wildlife conservation through poaching prevention. Since 2020, she co-organizes the EAAMO research initiative, committed to advancing Equity and Access in Algorithms, Mechanisms, and Optimization. Lily holds a PhD in computer science from Harvard University and her research has been recognized with best paper runner-up at AAAI, the INFORMS Doing Good with Good OR award, a Google PhD Fellowship, a Siebel Scholarship, and dissertation award runner-up at AAMAS.

Title: Neurosymbolic AI for Hypothesis Generation in Social, Chemical, and Cognitive Sciences

Abstract: Scientific discovery is a key driver of progress, and LLMs have the potential to accelerate this process. However, their ability to generate novel, valid hypotheses remains uncertain. We investigate whether LLMs can autonomously propose research hypotheses based on a given background (a research question and/or survey) without domain constraints, with a particular focus on social science and chemistry. Drawing from expert discussions, we assume that hypotheses emerge from a combination of a research background and multiple inspirations. This leads us to three key questions: (1) Can LLMs effectively retrieve useful inspirations? (2) Can they synthesize hypotheses by integrating background information and inspirations? (3) Can they rank stronger hypotheses higher, distinguishing quality and relevance? We also apply AI in cognitive science, with a particular focus on metaphor analysis. This approach allows us to explore how metaphors shape thinking and communication across different domains. Specifically, we conduct a comparative study on metaphorical cognition in ChatGPT and humans, examining similarities and differences in how AI and human minds process figurative language. Additionally, we use metaphor analysis to unveil diplomatic narratives, investigating how metaphors influence international relations and political discourse. Furthermore, we apply this method in financial metaphor analysis, studying the role of metaphors in shaping economic thought, market trends, and investor behavior.

Bio: Erik Cambria is a Professor of Artificial Intelligence at Nanyang Technological University, where he also holds the appointment of Provost Chair in Computer Science and Engineering, and a Visiting Professor at MIT Media Lab. He is also Founder of several AI companies, such as SenticNet, offering B2B sentiment analysis services, and finaXai, providing fully explainable financial insights. His research focuses on neurosymbolic AI for interpretable, trustworthy, and explainable affective computing in domains like climate resilience, mental health, and socially responsible investing. He is ranked in Clarivate's Highly Cited Researchers List of World's Top 1% Scientists, is recipient of many awards, e.g., IEEE Outstanding Early Career, was listed among the AI's 10 to Watch, and was featured in Forbes as one of the 5 People Building Our AI Future. He is an IEEE Fellow, Associate Editor of various top-tier AI journals, e.g., Information Fusion and IEEE Transactions on Affective Computing, and is involved in several international conferences as keynote speaker, program chair and committee member.

Title: Interpretable and Explainable Scientific Equation Discovery via LLM-Guided Symbolic Programming

Abstract: Scientific equation discovery is a crucial aspect of computational scientific discovery, traditionally approached through symbolic regression (SR) methods that focus mainly on data-driven equation search. Current approaches often struggle to fully leverage the rich domain-specific knowledge that scientists typically rely on. We present LLM-SR, an agentic AI framework that combines the power of large language models (LLMs) with evolutionary program search and data-driven optimization to discover scientific equations more effectively and efficiently while incorporating scientific prior knowledge. LLM-SR functions as an autonomous scientific agent, integrating several key aspects of the discovery process, namely, scientific knowledge representation and reasoning (enabled by LLM-based agents with structured prompting and prior knowledge), hypothesis generation (equation skeleton proposals produced by LLMs), data-driven evaluation and optimization, and evolutionary search for iterative refinement. The agentic nature of our system enables a closed-loop workflow that mimics how scientists generate, test, and refine hypotheses. Our framework explicitly prioritizes the interpretability and explainability of both the discovered equations and the underlying reasoning steps, ensuring that the discovery process itself is transparent to domain experts. By combining equation program search with LLM-guided reasoning, we enable transparent and auditable model behavior that facilitates post-hoc attribution and qualitative validation of scientific hypotheses. Through this integration, our approach discovers interpretable and physically meaningful equations while ensuring efficient exploration of the equation search space and generalization to out-of-domain data. We demonstrate the effectiveness of LLM-SR across various scientific domains—including nonlinear oscillators, bacterial growth, and material stress behavior—highlighting how agentic, explainable modeling frameworks can accelerate scientific insight while fostering trust and reproducibility in AI-assisted discovery.

Bio: Chandan Reddy is a Professor in the Department of Computer Science at Virginia Tech. He received his Ph.D. from Cornell University and his M.S. from Michigan State University. His primary research interests include Machine Learning and Natural Language Processing, with applications in Healthcare, Software, E-commerce, and Human Resource Management. Dr. Reddy's research has been funded by organizations such as the NSF, NIH, DOE, DOT, and various industries. He has authored over 190 peer-reviewed articles in leading conferences and journals. He received several awards for his research work including the Best Application Paper Award at the ACM SIGKDD conference in 2010, the Best Poster Award at the IEEE VAST conference in 2014, and the Best Student Paper Award at the IEEE ICDM conference in 2016. He was also a finalist in the INFORMS Franz Edelman Award Competition in 2011. Dr. Reddy serves (or has served) on the editorial boards of journals such as ACM TKDD, ACM TIST, NPJ AI, and IEEE Big Data. He is a Senior Member of the IEEE and a Distinguished Member of the ACM. More information about his work is available at https://creddy.net.

Title: Application-driven Machine Learning in Climate Change

Abstract: Machine learning is increasingly being called upon to help address climate change, from modeling Earth systems to accelerating the energy transition. Such settings represent an important frontier for machine learning innovation, where traditional paradigms of large, general-purpose datasets and models often fall short. In this talk, we show how an application-driven paradigm for algorithm design can respond to problem-specific goals, with a focus on physics-inspired and interpretable ML systems. We introduce novel techniques that leverage the structure of the problem to improve accuracy and usability across applications, including climate model emulation and downscaling, designing chemical catalysts, and optimizing the electrical grid.

Bio: David Rolnick is an Assistant Professor and Canada CIFAR AI Chair in the School of Computer Science at McGill University and at Mila – Quebec AI Institute. He is a Co-founder and Chair of Climate Change AI and serves as Scientific Co-director of Sustainability in the Digital Age and co-lead of the Global Center on AI and Biodiversity Change (ABC). Dr. Rolnick is a Sloan Research Fellow and an AI2050 Early Career Fellow and was named to the MIT Technology Review’s 2021 list of “35 Innovators Under 35” for his work in building the field of AI and climate change. He received his Ph.D. in Applied Mathematics from MIT and is a former Fulbright Scholar, NSF Graduate Research Fellow, and NSF Mathematical Sciences Postdoctoral Research Fellow.

Title: PIED: Physics-Informed Experimental Design for Inverse Problems

Abstract: In many science and engineering settings, system dynamics are characterized by governing partial differential equations (PDEs), and a major challenge is to solve inverse problems (IPs) where unknown PDE parameters are inferred based on observational data gathered under limited budget. Due to the high costs of setting up and running experiments, experimental design (ED) is often done with the help of PDE simulations to optimize for the most informative design parameters (e.g., sensor placements) to solve such IPs, prior to actual data collection. This process of optimizing design parameters is especially critical when the budget and other practical constraints make it infeasible to adjust the design parameters between trials during the experiments. However, existing experimental design (ED) methods tend to require sequential and frequent design parameter adjustments between trials. Furthermore, they also have significant computational bottlenecks due to the need for complex numerical simulations for PDEs, and do not exploit the advantages provided by physics informed neural networks (PINNs) in solving IPs for PDE-governed systems, such as its meshless solutions, differentiability, and amortized training. This work presents Physics-Informed Experimental Design (PIED), the first ED framework that makes use of PINNs in a fully differentiable architecture to perform continuous optimization of design parameters for IPs for one-shot deployments. PIED overcomes existing methods' computational bottlenecks through parallelized computation and meta-learning of PINN parameter initialization, and proposes novel methods to effectively take into account PINN training dynamics in optimizing the ED parameters. Through experiments based on noisy simulated data and even real world experimental data, we empirically show that given limited observation budget, PIED significantly outperforms existing ED methods in solving IPs, including for challenging settings where the inverse parameters are unknown functions rather than just finite-dimensional.

Bio: Dr. Bryan Low is an Associate Vice President (AI) and an Associate Professor of Computer Science at the National University of Singapore (NUS), the Director of AI Research at AI Singapore, and the Deputy Director of NUS AI Institute. He obtained the B.Sc. (Hons.) and M.Sc. degrees in Computer Science from National University of Singapore, Singapore, in 2001 and 2002, respectively, and the Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University, Pittsburgh, Pennsylvania, in 2009. His research interests include data-centric AI, probabilistic & automated machine learning, agents in ML, and AI for science.
Dr. Low is the recipient of the (1) Andrew P. Sage Best Transactions Paper Award for the best paper published in all 3 of the IEEE Transactions on Systems, Man, and Cybernetics - Parts A, B, and C in 2006; (2) National University of Singapore Overseas Graduate Scholarship for Ph.D. studies in Carnegie Mellon University (CMU) in 2004-2009; (3) Best Paper Award at the ICML 2024 Workshop on AI for Science; (4) Singapore Computer Society Prize for Best M.Sc. Thesis in School of Computing, National University of Singapore in 2003; and (5) Faculty Teaching Excellence Award in School of Computing, National University of Singapore in 2017-2018 and 2023-2024.
Dr. Low has served as a World Economic Forum’s Global Future Councils Fellow for the Council on the Future of Artificial Intelligence and Robotics from Sep 2016 to Jun 2018 and an IEEE Robotics & Automation Society (RAS) Distinguished Lecturer for the IEEE RAS Technical Committee on Multi-Robot Systems in Mar 2019. He has served as an organizing chair for the IEEE RAS Summer School on Multi-Robot Systems in Jun 2016, the AI Summer Schools in Jul 2019 and Aug 2020, and the NeurIPS 2021 Workshop on New Frontiers in Federated Learning. Dr. Low has also served as associate editors, (senior) area chairs and program committee members, and reviewers for premier AI (specifically, multiagent systems, AI planning, robotics, machine learning) conferences: IJCAI, AAAI, ECAI, AAMAS, ICAPS, RSS, IROS, ICRA, CoRL, NeurIPS, ICML, AISTATS, ICLR and journals: TKDE, JMLR, JAIR, MLJ, TNNLS, T-ASE, IJRR, T-RO, AURO, JFR, TOSN, JAAMAS. He was the top 5% reviewer for ICML 2019, top 33% reviewer for ICML 2020, expert reviewer for ICML 2021, top 25% PC member for AAAI 2021, outstanding SPC member for AAMAS 2023, and distinguished PC member for IJCAI 2023.

Title: Learning Interpretable Macroscopic Dynamics

Abstract: We discuss some recent work on constructing interpretable macroscopic dynamics from trajectory data using deep learning. We adopt a modelling approach: instead of generic neural networks as functional approximators, we use a model-based ansatz for the dynamics following a suitable generalisation of the classical Onsager principle for non-equilibrium systems. This allows the construction of macroscopic dynamics that are physically motivated and can be readily used for subsequent analysis and control. We present a case study in the analysis of polymer stretching in elongational flow.

Bio: Qianxiao Li is an assistant professor in the Department of Mathematics, and a principal investigator in the Institute for Functional Intelligent Materials, National University of Singapore. He graduated with a BA in mathematics from the University of Cambridge and a PhD in applied mathematics from Princeton University. His research interests include the interplay of machine learning and dynamical systems, control theory, stochastic optimisation algorithms and data-driven methods for science and engineering.

Title: Probing for the Unknown: Mechanistic Concept Discovery Beyond Human Expectation With SemanticLens

Abstract: Scientific AI models often learn patterns that extend beyond our assumptions — yet current explainability tools primarily probe for what we expect to find. In this talk, we argue why expectation-based probing (e.g., via Concept Activation Vectors) is not enough, and present a scalable alternative for uncovering the full spectrum of a model’s internal knowledge. We will introduce the SemanticLens method that embeds model components, such as neurons or directions, into the semantic space of a multimodal foundation model. This enables scalable concept discovery, automated labeling, and targeted probing based on human-defined expectations. With this framework, we can audit the alignment between expected and actual model behavior, and crucially, uncover spurious or surprising concepts that influence decisions. Finally, we discuss challenges such as distributed concepts and polysemantic components, and outline practical strategies to address them — paving the way for scientific discoveries and more interpretable scientific AI.

Bio: Maximilian Dreyer and Jim Berend are PhD candidates in the Explainable AI group at the Fraunhofer Heinrich Hertz Institute in Berlin, supervised by Sebastian Lapuschkin and Wojciech Samek. Their research focuses on developing concept-based explainability methods that are intuitive, scalable, and insightful — with the goal of improving the safety, robustness, and transparency of large AI models. A key focus lies in integrating explainability into the model development process itself and in leveraging foundation models to reduce manual effort in interpreting learned representations.

Title: Why Uncertainty Calibration Matters for Reliable Perturbation-based Explanations

Abstract: Perturbation-based explanations are widely utilized to enhance the transparency of modern machine-learning models. However, their reliability is often compromised by the unknown model behavior under the specific perturbations used. This paper investigates the relationship between uncertainty calibration - the alignment of model confidence with actual accuracy - and perturbation-based explanations. We show that models frequently produce unreliable probability estimates when subjected to explainability-specific perturbations and theoretically prove that this directly undermines explanation quality. To address this, we introduce ReCalX, a novel approach to recalibrate models for improved perturbation-based explanations while preserving their original predictions. Experiments on popular computer vision models demonstrate that our calibration strategy produces explanations that are more aligned with human perception and actual object locations.

Bio: Thomas Decker is a PhD student at LMU Munich and the Munich Center for Machine Learning (MCML). He is also a Research Scientist at Siemens AG, working on reliable and human-centered AI solutions for industrial applications.