During my master's studies at TU Delft, I have had the opportunity to work on several interesting projects, most of which were completed as part of coursework and involved collaboration with fellow students in groups of two to five people. This page describes some of these projects and provides links to documentation and code repositories wherever possible.
As a quick side note, TU Delft highly values academic integrity and intends to maintain the same level of course quality across multiple academic years of the program. Students are not permitted to share project and assignment solutions/concepts online to prevent the risk of compromising the integrity of the course and other students' learning outcomes. I will hence not be able to share the exact details and implementation specifics of some of these projects.
Path Planning for Non-Holonomic Robots w/ RRT *
This course provides an overview of motion planning and decision-making techniques and their practical application in robotics. As part of the course project, I worked with a group of four students to implement the optimal rapidly exploring random tree (RRT*) planning algorithm for a non-holonomic robot using the Dubins and Reed-Shepp connectors.
RRT* is an asymptotically optimal sampling-based algorithm that finds paths in the configuration space. For non-holonomic robots planning paths in a 2D configuration space, the connector function within RRT* was modified to use the Dubins/Reed Shepp functions. Finally, an off-the-shelf implementation of MPC was used to follow the reference trajectory generated by RRT*.
Planning, Perception & Control for Autonomous Inspection w/ ROS
This is a project-based course towards the end of the first academic year that requires students to work in groups of five to develop a complete robotic system both in simulation and with real hardware. The aim of this project is to implement the various aspects of a robotic solution while also learning to work in multidisciplinary teams, carrying out systems engineering, and learning to work in the agile workflow.
Our solution involved an implementation of the ROS navigation stack to plan base paths, using the moveit ROS package to plan and execute robotic arm paths, a behaviour tree to implement cascading behaviours based on the robot's state, and a GUI element for better human-robot interaction. We worked with our client, SAM|XL to use these elements for a turbine blade inspection robot.
Predicting Action Progress in Videos
Course: CS4240 Deep Learning
This course goes into the specific field of deep learning to cover a variety of deep learning approaches. The course project for this course required students to work in groups of three to reproduce recently published research papers and document the reproduction implementation details and findings as a blog post.
My group chose a paper on predicting action progress within videos. With the ability to predict action progress, intelligent agents can better understand future states and even choose to act preemptively; first predicting when an action might take place and then following its progress until it ends. This could be especially useful for tasks such as accident avoidance, intercepting objects mid-action (such as lane changing or catching projectiles), and much more.
The reproduced deep architecture – named ProgressNet – is capable of “predicting when an action takes place in a video, where it is located within the frames, and how far it has progressed during its execution”. ProgressNet is an LSTM based architecture that is attached to an off-the-shelf Faster R-CNN backbone. It learns to predict action progress by exploiting the temporal nature of actions by modelling them as time-based sequences of images and region of interest bounding boxes.
Neural Guided Population Initialisation for Symbolic Regression
Course: CS4205 Evolutionary Algorithms
This course goes into the sub-field of machine learning called evolutionary algorithms. It covers a spectrum of topics in EAs, ranging from basic concepts to advanced, recent, and state-of-the-art research, and ranging from theoretical to applied. In particular, topics include genetic algorithms, evolution strategies, genetic programming, estimation-of-distribution algorithms, optimal mixing evolutionary algorithms, multi-objective optimization, and real-world applications.
The course project for this course involved the use of symbolic regression to learn expressions for the optimal action value function for the OpenAI Gym lunar lander environment. Students were tasked with improving their baseline solutions through modifications based on current literature. One of the proposed improvements was neural guided population initialisation – the process of replacing random initialisation within symbolic regression with some learnt approach that generates a fitter population of trees and then improves on it via genetic programming. Our solution used the deep Q networks algorithm to learn a policy that iteratively adds operators to a base tree. This involves chaining up two Markov decision processes to optimise policy networks of the first one based on the reward signal accumulated in the second.
Knowledge Enabled Task Planning w/ PDDL & Prolog
This course provides an overview of knowledge representation and reasoning (KRR) techniques and their practical application in robotics, to support the programming of the robot’s behaviour with reasoning using symbolic knowledge about the mission tasks, its capabilities, and its environment.
As part of the course project, I worked with a group of three students to develop a task planner that used Prolog as a high-level knowledge base of rules and PDDL as a low-level task planner to plan a sequence of actions. The robot's goal was to assist retail store employees with tasks such as restocking or fulfilling orders. These tasks were modelled as rules with ROS Prolog and then durative actions within PDDL were used to carry out tasks such as picking an object or navigating to waypoints.
Behaviour Cloning
This course provides a broad overview of machine learning techniques and their applications to robotics. It goes into several individual topics from the subfields of supervised and unsupervised learning and provides a deep understanding of the fundamentals of data-driven learning methods. As part of the course project, students worked in groups of two to train an agent to learn to solve a custom four-rooms Gym environment. The task is to learn a control policy through behaviour cloning, given a small dataset of human control actions.
My team's solution explored several models such as decision trees, random forest classifiers, SVMs, and deeper networks. In the end, we chose to go with a simple vanilla RNN. This was chosen to ensure a temporal connection between the agent's past and present actions and resulted in significantly better results compared to models that generate actions without considering the history of action sequences.
Comments