Yasaswi Kapila

Only teach what it can learn

Fri, 23 Oct 2026 00:00:00 +0000

I have been running reinforcement learning on a small open-weight model in my own time. I train it on four kinds of tasks at once: answering questions from retrieved documents, producing flat and nested JSON, following instructions, and solving math problems.

The first run went well for three of these. Retrieval answers and structured output improved clearly, but math barely moved.

The method I use is GRPO. It samples several answers to the same question and learns by comparing them. If all the answers are wrong, or all of them are right, there is nothing to compare, and the model learns nothing from that question. Most of my math questions were too hard for this model. It got every attempt wrong, so most of the math in each batch contributed nothing to training.

The model that lost its place

Sun, 04 Oct 2026 00:00:00 +0000

Language models don’t naturally know the order of the words they read. Positional encoding is what gives them this information. We use a variant called YaRN, which stretches the model’s sense of position so that a model trained on shorter text can handle longer documents. We added YaRN to our model and checked that it worked.

To make generation faster, a GPU can record a sequence of operations once and then replay that recording again and again, which skips a lot of overhead. These recordings are called CUDA graphs, and they are one of the easiest ways to speed up inference.

243 missing tensors

Mon, 14 Sep 2026 00:00:00 +0000

After fine-tuning, my model was producing garbage, repeating the same few words over and over. I assumed something had gone wrong in training and spent a good amount of time adjusting the training setup.

To check whether the model could learn at all, I trained it on just five examples until it memorised them. It memorised all five, but generation still produced the same repeated words. So the training was working, and the problem was somewhere else.

Training where the internet isn't

Mon, 20 Jul 2026 00:00:00 +0000

Some of my work runs on machines that have no internet connection at all. On these machines, you can’t install a missing package or download a dataset you forgot, and every gap means another trip with a drive. Over time, this changed the way I set things up.

Anything that needs the internet becomes a script that I run once, on a different machine. I download the data, save it, and carry the whole folder in. I also pin every library to an exact version, because the version I test with has to be the same one that arrives on the machine.

About

Mon, 01 Jan 0001 00:00:00 +0000

I’m Yasaswi, a data scientist at Neuralix.AI. I work on large language models: model architecture, distributed pretraining, and post-training with supervised fine-tuning, preference optimisation and reinforcement learning. I also make our models work with Hugging Face and vLLM.

I’m most interested in how architecture choices affect training and inference, and in using reinforcement learning to improve how models behave on real tasks.

Projects

In my own time, I post-trained Qwen2.5-1.5B with reinforcement learning (DAPO in verl). I trained two teachers, one for grounded RAG answering and JSON output and one for math and instruction following. I then merged them into one student with multi-teacher on-policy distillation. The student beats the original model on RAG, instruction following, JSON extraction and math, and isn’t significantly worse on any of the 24 benchmarks I tracked. The code and write-up are on GitHub, and the models and training data are on Hugging Face.