It's been a few months since I last checked in and I am creeping closer to an official release date. I have to admit that getting this thing across the finish line has been way more challenging than I expected it to be. I remember when I first started building I told Shardé that it would only take me a couple of weeks.
Now, in my defense, the scope back then was much smaller. I was going to self host my home grown student management system, a more secure version of the registration system a co-op my family is a member of uses now. That old system was donated by some previous parent who was a a programmer, but by this time that family was long gone, their kids are graduated, but they still support the website out of the goodness of their hearts.
I planned to use the old site as a spec and rebuild it using modern tech, I did some perfunctory searching and didn't see any open source tools that did what I wanted and I dove in.
I faced the usual challenges of agentic coding, I'm sure I talked about those in detail in my previous post, but towards the end I started to bump up some more subtle problems that I knew existed but didn't realize how annoying it could be until I really experienced it first hand.
First, as the project gets bigger and more complex it becomes more difficult to get Claude or Codex to stick to a problem solving style. Most programmers have a handful of patterns that they like to apply, and generally you'll see problems throughout a project that is maintained by one person or a very small team, that you see the same patterns applied to various problems throughout the codebase.
LLMs know all of the patterns though so they tend to use them all and sometimes they will use one pattern to solve a problem in one place, and then an entirely different one to solve a nearly identical problem elsewhere.
The other issue I ran into this past month was LLMs weird inability to say "No", or "That's a bad idea".
It seems that even when provided with very large context windows LLMs are still lacking the ability to understand the goals and overlay that on top of the context window so that each iteration pulls us closer to the solution. Since LLMs lack that, working with them feels like pulling a random, very talented, CS student off of the street to work with every couple of hours. Just as it feels like they are beginning to understand the project you have to dismiss them and grab another random student and start fresh.
I'm going to say upfront that the following example is, in hindsight, on me. But I asked the most advanced agent I have access to at the moment (CODEX 5.5) to basically solve one of those triangle problems where you can only have 2 of your 3 needs met. You want the job done, right, fast, and cheap, you have to pick two. Agents don't really understand that paradox and will happily burn a lot of tokens trying to oblige. Eventually it will present the problem as solved, having only actually done 2 of the 3 things and when you take a look at the code for the 3rd requirement you will inevitably find some kind of awful hack job.
When I was a very junior developer I used to get myself into messes like this all of the time, but as I gained more experience I learned to spot problems shaped like this and avoid the pitfall. The only way to really spot these problems though is to fully understand how your tools work, what their limitations are, side effects, etc. This is something that can't really be handwaved away by a general purpose coding agent.
At the risk of repeating myself, building complex things is still hard.