Blog

Recent algorithmic progress on two fundamental RL questions: credit assignment on the theory side, and asynchronous inference-trainer separation on the practical side. GRPO, SAO, FlashREINFORCE, BPO and KLPO each answer that pair differently.

Running Codex and Claude Code via CLI is very very slow on my computer when connected to my network at home. My initial resolution is to use a remote machine via ssh and git clone my repos there, but that feels like a circumvention rather than really fixing it. So I asked Claude to help...

I tested this question with two local LLMs: Qwen3.6-27B and DeepSeek-Prover-V2-7B.

Recently my research lab built a custom agent running on a spare lower-end Mac mini box that my mentor happens to have. We call it Murphy, in defiant spirit to Murphy’s law and tribute to the beloved Interstellar.

I will skip the notation and introduction on RL as a Markov Decision Process, for that you can refer to this blog. Here is a checklist of questions to ask yourself before proceeding: what $J(\theta)$ and $\nabla_{\theta} J(\theta)$ (recall policy gradient theorem) look like? what is $Q$, $A$, $V$? do you have 10 minutes to...

Here is a beta version of an ideal agent that I have been thinking about, this helps me personally to categorize a large amout of new ML research paper to one of the boxes or arrows, and to identify gaps.