Hey! I’m Zihe1 (or you can call me Zach2). I do computer engineering at the University of Cambridge. Most of my time is spent thinking about how frontier AI systems learn, particularly with Reinforcement Learning (RL), how LLMs obtain goal-directed behaviour, and aligning models to intended specifications (hopefully values). This is my little corner of the Internet where I talk about what I’m currently obsessed with, cool projects I’ve worked on, stuff I’ve learnt, and some thoughts that have recently bubbled to the surface but refuse to pop.
About Me
I’m really curious about why RL works at all, and how the overoptimization problem could cause model outputs to become unintelligible to humans. I’m pretty concerned with extensive agentic/reasoning post-training leading to unintended (likely dangerous) behaviour due to reward hacking, so I’m currently working on how we can instill values via RLVR over long-horizon tasks. I’m personally curious about the geometry of optimization, especially in RL policy gradients, and am working on an efficient second-order optimizer for RL as my master’s thesis.
Previously, I worked at Amazon Prime Video building a system to root-cause error logs at scale, using CatBoost and LLMs, but turned down a return offer to work on RL research. I also poked around the high level learning dynamics of Natural Gradient Descent, researched and benchmarked various hallucination detection methods at the Singapore AI Safety Institute, and help out at the Cambridge AI Safety Hub. I did the ARENA 4.0 program in London, where we built GPT-2 from scratch and implemented PPO for RLHF. Before that, I deployed CV models for object detection and did topic modelling NLP stuff to automate web feedback at the Changi Airport Group ML team.
Singapore is where I call home, and before university I served 2 years in the Republic of Singapore Air Force as a security trooper. Outside work, you’ll probably find me dancing to hip-hop and R&B music, or rewatching old Wong Kar Wai films.