Open Honours and Course Projects
I supervise graduate and undergraduate honours projects in deep learning, reinforcement learning, and physical AI. These projects are designed to provide students with hands-on experience in modern AI research, including problem formulation, implementation, evaluation, and scientific communication.
Students will gain experience with PyTorch, reinforcement learning frameworks, robotics platforms, and real-world experimental design.
Featured Projects
Featured Collaborative Project
Privacy-Preserving Synthetic Health Data Generation
Ongoing Undergraduate Research
This project focuses on developing generative models for producing high-quality synthetic health data while preserving patient privacy. The work explores trade-offs between data utility and privacy guarantees, with applications in healthcare analytics and machine learning.
Collaboration: Vector Institute, TELUS Health
Featured Collaboration Project
AI for Cybersecurity: Privacy-Preserving Synthetic Data Generation for Insider Risk Modeling
New Research Collaboration
Organizations frequently possess valuable insider-risk and cybersecurity data that could improve threat detection and resilience across sectors. However, privacy, security, and legal concerns often prevent data sharing between organizations. This project investigates how generative AI can be used to learn from heterogeneous insider-threat datasets and generate high-utility synthetic data that can be safely shared without exposing sensitive organizational information.
Research activities include evaluating transformer and diffusion-based synthetic data generation methods, studying privacy and re-identification risks, developing novel privacy-preserving generative algorithms, and measuring the trade-offs between data utility and privacy protection. The long-term goal is to create validated methods and prototype systems that enable secure knowledge sharing for insider risk management and cybersecurity applications.
Collaboration: Canadian Insider Risk Management Centre of Excellence (CIR-MCE), Carleton University
Featured Collaboration Project Sim-to-Online Reinforcement Learning for Robotics Control Ongoing Research Collaboration This project investigates sim-to-online reinforcement learning for robotic control. An initial control policy is learned efficiently in a JAX-based simulation using Soft Actor-Critic (SAC), and then An initial control policy is learned efficiently in a JAX-based simulation using Soft Actor-Critic (SAC), and then transferred to a physical robotic system. to a physical robotic system. The policy is further adapted online using incremental deep reinforcement learning approaches inspired by recent work, the Action Value Gradient (AVG) methods. The aim is faciliate efficient, adaptable and robustness embodied agents. Collaboration: Amii, National Research Council (NRC) Related Work Featured Research Project
Learning When to Look: Active Observation Reinforcement Learning for Real Robots
Open Undergraduate and MSc project opportunty
This project investigates how reinforcement learning agents can actively choose what to observe, when to observe, and how to use limited sensing to make robust decisions in real-world robotic systems. Rather than assuming that all state information is always available, the project studies robots that must operate under partial observability, sensor noise, latency, occlusions, limited viewpoints, and changing real-world conditions.
Research directions include developing active perception and observation-selection policies, integrating uncertainty estimation into reinforcement learning, improving sim-to-real transfer under practical sensing constraints, and evaluating learned policies on physical robotic platforms such as manipulators, mobile robots, or aerial robots. The goal is to build RL agents that are not only sample-efficient in simulation, but also reliable when deployed on real hardware.
Application Areas: Robot manipulation, mobile robotics, sim-to-real transfer, adaptive sensing, and robust autonomy
Past Student Projects
Reinforcement Learning with Pre-Collected Datasets
Fall 2025
Tara Denaud Joseph, Richa Kewalramani, Fay Lee
Reinforcement Learning and Its Applications on the Kinova Gen3 Lite Robotic Arm
Fall 2025
Zahra Suleymanova, Vishal Bhat
Multi-Agent Reinforcement Learning for Safe Hovering and Navigation on Crazyflie Drones
Fall 2025
Michael O’Sullivan, Aydin Yalcinkaya
Single-Agent Reinforcement Learning for Drone Robotics
Fall 2025
Kevin Naveen, Chad B. Yassin, Pronoy Fuad
Is PPO Inherently Safe?
Winter 2026
Estelle Ngounou, Eliel Beonao, Jean-Philippe Nahimana Bahenda
Available Project Areas
Reinforcement Learning (RL)
Single-Agent Reinforcement Learning
Develop and evaluate RL algorithms in simulated environments. Projects may explore:
- Sample efficiency and exploration strategies
- Safe and constrained decision-making
- Goal-conditioned learning
- Learning under limited or noisy data
Multi-Agent Reinforcement Learning
Study how multiple agents learn to cooperate or compete. Possible directions:
- Coordination and communication between agents
- Decentralized vs. centralized training
- Emergent behaviors in complex environments
Robotics and Sim-to-Real Learning
Sim-to-Real Transfer with Crazyflie Drones
Develop RL or control methods in simulation and transfer them to real Crazyflie quadrotors. Topics include:
- Domain randomization
- Robust control policies
- Perception-based control
Robotic Manipulation with Kinova Arms
Design learning-based approaches for robotic manipulation tasks:
- Grasping and object interaction
- Vision-based control
- Policy learning from limited data
Mobile Robotics with TurtleBots
Develop navigation and decision-making algorithms for ground robots:
- Autonomous navigation and planning
- Sensor-aware decision-making
- Reinforcement learning for robotics
Data-Efficient and Adaptive Learning
Fine-Grained Active Measurement in RL
Investigate how agents can selectively acquire observations when sensing is costly:
- Observation-cost-sensitive decision making
- Adaptive data collection strategies
- Exploration under sensing constraints
Continual Reinforcement Learning
Design agents that can learn across multiple tasks without forgetting:
- Lifelong learning
- Transfer across environments
- Avoiding catastrophic forgetting
Machine Learning for Challenging Data Settings
Learning from Imbalanced Data
Develop deep learning methods that perform well when data is skewed or rare:
- Class imbalance and rare event detection
- Synthetic data generation
- Robust evaluation methods
Privacy-Preserving Health Data Generation
Design generative models for sensitive datasets:
- Privacy-preserving synthetic data
- Evaluation of utility vs. privacy trade-offs
- Applications in healthcare datasets
Expectations
Students undertaking honours projects are expected to:
- Work independently with regular supervision
- Read and understand research papers
- Implement and evaluate machine learning models
- Present results in written and oral form
Strong programming skills (Python) and familiarity with machine learning are expected. Prior experience with deep learning or reinforcement learning is an asset but not required.
How to Apply
If you are interested in an honours project, please email me.
Please include:
- A transcript (unofficial is fine)
- A brief statement of interest
- Any relevant experience (courses, projects, GitHub)
