Links and Fable
Like seemingly everyone else with a Claude subscription, since the start of this month I've been squeezing as much Fable as I can1 out of my little USD$20 subscription. And most of that has gone towards my little side project / informal frontier AI coding test case: a top down railway company simulator game a la OpenTTD. So what have I got out of the big code machine so far?
Well, one thing I'll say is 'it's more complicated than it looks'. For one thing, it contains an signalling system based on OpenTTD's own quite advanced implementation... which I ended up having to explicitly prompt Fable to emulate. Presumably if I'd had broken up the implementation into smaller chunks, or been more precise with my specifications, Fable (or a smaller model, or perhaps even myself writing it the old fashioned way) could've made it's 'own' implementation, drawing off the same ideas but without direct reference to how OpenTTD does it. But without the pre-existing idea to shape its work into something that works beyond its own ability to check its work through tests and harnesses, it's still perfectly capable of giving you the fluently executed, impeccably precise execution of the complete wrong thing.
Not to say that these models are incapable of doing the right thing! Beyond its ability to spit out code given a well-defined task (which you just need to look at one of the 5 billion coding benchmarks to see its uncanny ability at), this latest batch of models have become really good at understanding the environment of the codebase they're in and how to get things done inside it. They now routinely build their own verification tests, sometimes on the fly without explicit prompting, launching and screen-capping the application to check its work. This is a very useful and impressive ability, and does a lot of the hard work of checking whether the idea expressed in the static world of code and text works in the 'real' world of dynamic behaviour. But not all the work, at least for now...
If you want to check out the code for my game it's up on GitHub, though it's still very much a work in progress and nowhere close to a completed game. If you want a much more complete and robust railway company simulator game, I heartily recommend OpenTTD, which is free, open-source and whose developers have done far more impressive things than what I've done without any LLM help at all.
Links
I don't intend for this to be a just a 'yapping about AI' blog... but it does seem to be turning out that way. I'm at least going to try to keep at least every second post about something else completely! For now, here's some links:
- Excellent blogger Dynomight wrote a post detailing his AI usage policy and the thinking behind it, which I pretty much totally endorse and inspired the little disclaimer I've put on my about page. In fact, I'm a little stricter than him in that I don't use it for even proofreading or phrasing feedback.
- A lot of the most interesting writing about AI I've stumbled across recently has come from Robin Sloan. Being most familiar with a corner of the internet whose talk about AI is a lot more focused on predictions, percentages and probabilities (not to say that said talk isn't meaningful and important!), it's a real breath of fresh air to see an insightful outside view that has experimented with and thought long and hard about the intrinsic qualities of these strange new artefacts. I especially enjoyed his look inside the architecture of LLMs to find out where exactly 'the model' is.
- And speaking of looking inside LLMs, I'm still wrapping my head around Anthropic's research into how concepts are represented in a model's 'thoughts'. It seems to fit quite well with Sloan's post above, at least from my very non-expert reading.
Have thoughts? Post them in my feedback form!
While still maintaining a regular sleep schedule / social life etc.↩