I Built a Personal Quiz Engine That Knows My Gaps

Nobody's Talking About What Education Actually Looks Like Now

When people talk about education, it's the same recycled conversation every time. You've got people feeding you a load of garbage to make political gains, and then you've got people who genuinely care but have bought into all the same talking points because what else are you going to do. Very few people are talking about what education actually looks like going forward. Not the policy debates. Not the funding fights. What it looks like when you sit down and actually learn something in 2026.

I had a conversation with Gus today. Gus is my personal AI assistant, the one running all my active automations through Telegram. Once my automations reach a level of maturity and dependability, they go into a stable ops server where I run my VPN, Keycloak, and all the stuff that's in maintenance mode. Gus handles everything that's still in active development or ongoing. The Telegram channel has become something special. Being able to hook up an LLM with a good memory system that actually persists your conversations, wire it into Telegram, and just chat with it throughout the day? It's a game changer. It lets me recapture a lot of missed opportunities for productivity.

Where Did This Idea Come From?

I had two things kicking around in my head, and they clicked together while I was walking the trail with my son. He was on a scooter, getting ahead of me, so I pulled out my phone and started documenting ideas.

The first thing: I've built up a pretty solid reputation over the years. I've had experiences in technology and development, in freelancing with clients on Upwork in particular. And I've been gathering a lot of data on all of it. Blog analytics, traffic from Google Analytics, social media metrics from Facebook, LinkedIn, and X, custom metrics I'm collecting myself, Vercel analytics, plus Upwork data where I analyze market trends, job postings, and client messaging. I have bots that automate all this data collection. I synthesize it, store it, then go back and build reports analyzing everything we've collected. I get powerful insights into market trends, what people are paying for, how much, what technologies are hot, what's trending.

So I'm reaching a point where I have real authority on two fronts. As an engineer, I've got authority from my experience with technology. As a freelancer, I've got authority from my reputation and history. The hot thing right now is AI automation, taking workflows that were historically done with tools like n8n and replacing them with agentic AI orchestration, or combining the two. There's a strong case for combining them.

The Vocabulary Test Problem

I've had these experiences in my career where you know all this stuff, you've got all this experience, you attract attention from potential clients, and then the founder gets an engineer friend to jump on a call and size you up. The good engineers, the ones who are really invested in the company and the project, they'll ask you questions like: here's a problem, how would you solve it? They try to get a feel for your process, how you analyze things, how you come up with solutions.

Then there are the lazy ones. The ones doing a favor for a friend. They shortcut the whole process and basically give you a vocabulary test of terms they learned in college. If you don't know the term, they write you off. "He can't possibly know anything. Anyone who knows anything would know this." And they leave it at that.

That's the extreme case. But there's a more honest version of this problem too. When you're self-taught, you have a lot of practical experience, but there really are certain concepts where you lack depth of knowledge. And there are some pretty fair criticisms to be had there.

Five Minutes Is Five Minutes

How many times a day do you have a few minutes where you can't get immersed in the next thing? You're context switching, you've gotta drive somewhere, run some errands. You can't launch straight into the next task. So what happens? You kill time. Five minutes, scroll through the social feed. Play a round of Polytopia. Stare at a wall.

What a wasted opportunity.

Why not build an education engine where you can be quizzed on exactly the things you need to know? If there's some acronym or phrase or concept that I might not understand, take everything we've been doing, everything we've been talking about, everything we've been experimenting with and running in production, gather it all up and build a curriculum on it. Go out and do web searches, find books, articles, forums, any authoritative source. Synthesize it all down into our own knowledge base, probably in Markdown. A library. A personal university of knowledge.

Then quiz me. When I shoot a command that says "quiz me," ask me a question that closes gaps in my knowledge and advances my understanding. Judge the answer. Save the data. Track my progress. It needs to be a five-minute task I can bang out in no time.

Two Tracks: Pure Tech and Applied Client Scenarios

The conversation with Gus kept going, and we ended up with a two-track solution. The planning was going so well that I said, why don't you just go ahead and generate a quiz question right now based on everything you've got? I'll give you feedback and we'll iterate.

It generated one, and it was remarkable. It was a scenario about building a chatbot powered by Claude where the client was considering using the Anthropic API key directly. It was a combination of a technology problem and a client situation. I highlighted concerns with cost optimization, functionality, a pragmatic perspective on breaking it into phases and the potential upsell for iterating on the initial solution. I gave a lot of great perspective.

But I completely missed a glaring security problem. They were talking about sticking API keys in a React component. The API key would be in the build. Exposed to anyone who opens dev tools. Gus gave me credit for all my opinions that were sound, then blasted me for overlooking the security issue.

That experience made the two-track idea click. One track is pure technical truth: what is the actual best approach assuming ideal circumstances, plenty of time and money, the purely academic best way to do something? The other track is applied client scenarios, where you're dealing with real-world economics, client personalities, and communication challenges. The desired outcome in the applied track is: identify good clients and convert them to an open contract. If you have to lie to get someone to hire you, if you have to agree with something you think is stupid, that's a bad client. We need to identify that.

There's a whole economic factor in the client track. You want to be able to help clients understand why we should do things the way we should do them without making them defensive. And the pure technical track feeds directly into that, because the best way to advise someone is to have a deep, truthful, accurate read on the technology itself. You need to know what ideal even looks like before you can decide whether to push back on a client's bad idea. Otherwise you might just be bullshitting them and not even know it.

Keeping Quiz Data Out of Memory

One disaster I could see happening: if all your quiz data gets introduced into the memory system, all these synthetic scenarios could get read as the history of actual conversations. That would lead to hallucinations and all kinds of negative effects. So we came up with a tagging system. If the education tag is set, we ignore that data in memory. Then we go back once a week and build a report so we can evaluate progress and recalibrate.

There's also an interesting connection to Karpathy's work on automating self-improvement. I need to drill down into that. There might be a viable way to apply it here, though I'm curious what we'll come up with. I hope it's not simply using the quiz data we're collecting. That alone isn't enough data to spot a trend, let alone drive a full self-improvement loop.

Why Separate the Tracks at All?

Beyond the memory isolation problem, I genuinely believe the whole economic component of freelancing is going to get phased out, probably fairly quick, sometime over the next year. We're going to see that if you're an engineer, you don't have to worry so much about money. Just go build stuff. The more novel, the more unconventional, the better.

I think we're going to transition into a world where maximally truth-seeking is all that matters, and having to handle a client with kid gloves goes away. The days of people who have no merit whatsoever holding all the money cards? I think that's going to be gone. This shift is going to expose a lot of frauds, a lot of non-technical founders that don't actually do anything. It's kind of hard to bullshit tech. That's the main reason I transitioned from front-end to back-end. I can show you the science of the impact I'm having. Metric data. Alarms. Alerts. Error logs. I can tell you exactly, scientifically, what's wrong. Front-end design is subjective. Back-end work is not.

This Is What Education Looks Like Now

You're going to have an AI that knows you. It knows everything you've done, everything you're working on, and what your passions are for the future. It's going to help you on your own time with maximum flexibility. A personal curriculum tailored to exactly what you need. It'll quiz you in your spare moments throughout the day. You want to kill some time? Don't play a game. Quiz yourself.

It should be energizing. If you're being honest about what you care about, the things it quizzes you on should be things you're genuinely passionate about, things you crave to engage with as much as a video game or scrolling through a social feed. You can get addicted to this too. It should make you want to stay up all night and keep going. And then you look over those weekly reports and see your progress and it just fuels you more.

It's unreal that I'm going to live to see a day when every kid in every classroom has their own personal AI tutor, tailored to them, teaching them as effectively as possible on any subject, and then helping them figure out where to go next.