Becoming Agent Enabled Software Builder Part II

Note: This is an AI-assisted transcript and is not a verbatim or accessibility-certified transcript. There are some edits in the text to improve readibility. Text in square brackets, [], are in the video but are not needed for reading, so kept in brackets.

Hey, Sumit on this side. Good morning again from my little farm in this tiny village that I am in. We started off creating this video. If you have seen the last one, which was how to become an agent-enabled software builder, and there's a reason why I'm not using the term software engineer, instead I'm using software builder, is because I want to, like I said, I want to be more general in the audience. So I don't want it to seem like this is very engineering focused. Although for sure, building software needs using that kind of a mindset. But we'll come to it slowly, one step at a time. For now, the focus should be more about how to have conversations with LLMs, with large language models, and obviously through agents, particularly coding agents in our case. It's more about the structure, somewhat more about the thought process. Because this is something that I have said a few times before. I'm kind of building more content around it, is that agents are, or rather large language models, are only as good as the training that they have been given. And sure, there is both that they are trained themselves, which is called unsupervised or self-supervised learning, where given all the text that they have from the internet or wherever else, they kind of learn the meaning, underlying semantics by themselves. That definitely helps them understand large amounts of text. But at the end of the day, unlike us in our case, all the concepts that come have of course evolved over many thousands of years. So there's a lot of evolution that has happened from one person to another. There's a lot of gossip that goes around, a lot of talk or discussion, and of course the modern day with the internet, everything is more connected. This is not yet happening with large language models. They are not learning as much cooperatively among themselves. Sure, that will happen sometimes, and we'll see real learning where they are able to come up with long-term thinking and learning in ways that we do. But till that point, there is a lot that we still have to do. And today I'm going to talk about the rest of those things. Please watch that first video if you want to, but also you can jumpstart on this one and then go back to the first video. What is the structure of prompting then? And of course this coming from my background, my experience with structure of prompting. Of course I can only tell you what has worked, and I've been working on this kind of style of prompting that I do, the style of software building that I do, and I've been doing this for more than fourteen, fifteen months now, full time. I do not read or write code, so that's a big shift in my own life, of course coming from an engineer who has written code all my life. How do I start thinking about prompts? So first of all.

If we take a step back, the core ideas are around context. The first key idea is context. By context, I mean how much can you really give to the model, give to the agent. I'm going to use model and agent kind of interchangeably, but of course, agent is very different. Agent and harness is a little more interchangeable, but model is just the language model which is there on the cloud or maybe even on your computer. But the agent is how it reaches out to your files, it does things on your computer, etc. But for now, I'm going to use them kind of interchangeably, but agent and model are quite different. They have different purposes.

Let's say that the agent or the model has to have context of your project. Let's say I come up with a farm management project because I have a farm here. I'm co-invested with one of the local families, and we have a piggery and a chicken farm, along with a lot of food crops plus cash crops. Now the cash crops and the food crops, just for context, I am not really invested in that. I'm invested in the piggery and the chicken farm, and that means that there's a lot of financial back and forth. There's a lot of cost that either I am bearing from my own bank account or my mom is paying because she's also co-invested or my partner is paying, and all of us also have earnings coming from my software consultancy. My partner, she helps me in design, so she gets a little bit of salary from my company. That's my own agency. I am the only other person in the agency. Of course, we don't have any other employees on the software side, but you can imagine this is already a lot of finance. There's a lot of money going here, here, and there. And even though these are not large sums of money, they're pretty small actually. But the thing is that it still needs to be managed.

Which means if I wanted to build software which is around this lifestyle, and this is like I have always said that with agent-led programming, what will happen is that we will build more niche software to suit exactly where we are instead of finding software which is like, oh, hopefully this works. No, I will want to customize. I want to build software which fits my use case. Let's say this as a use case that I have a software consultancy. I earn some money from there, and then of course I'm hoping that these projects, for example, Prompt Together or my other products, they also start earning a little bit of money. That money comes to me personally as, let's say, part of my salary, and then it also goes to my partner. She's also involved.

And let's say my mom, she gets some earning from the farm, but she's also investing some of her savings into the farm. And then I have a business partner with the farm. So the one that manages the farm, as in the piggery, he is also a business partner. He is mostly a passive partner when it comes to money, but he's an active partner when it comes to actual work. So he's mostly taking care of the work, which means he also manages some of the money. So where does the money go? So this was mostly intake. The money goes to—we are buying feed, which is feed for animals. We are buying a lot of equipment. We are probably buying a lot of things to set up the farm. You can imagine this on a monthly basis, there's a lot of ins and outs. And this is very typical. I'm not even talking about a big business. This is a tiny, tiny business. This is like barely—I don't think we have reached more than, I don't know, two, three, four thousand dollars revenue-wise, the whole total rotation of money in a whole year. But from our perspective, this tiny village somewhere in the Eastern Himalayas, there's a lot of money.

Now, the thing is, money to me, whatever the amount is, it does not matter because the amount and the currency that will differ depending on where you are from. But what matters is, am I on top of it? And I want to build software, let's say, to be on top of it. That is what really matters for most businesses. Can I be on top of my finances? Can I be on top of my accounting? Can I actually know where my money is going, what is coming out? And also, there's a lot of projection. For example, because of piggery or because of the other cash crops or even the regular crops, there is a cycle. And I'm sure you can think of these things on your own. There is of course a cycle to either cash crops or to any kind of a farm. You have to invest some money, you have to wait, you have to maintain them, and then you get back your money, right?

Now, if you take a step back and if you think, okay, if I were to explain all of this to somebody, would they really capture all the details with every way, all the combinations possible just in one go? No, no human being can unless they are equally experienced and have been doing this business. They probably have a family farm and all of these things. They will not be able to capture all of these details, which means they will miss out details. And this just happens a lot more with large language models. I hope I can get the concept across is that why the context matters. So context has different

Meanings, particularly when we are talking about large language models, context is how much input, how much textual information, or how much any other kind of digital information they can take in one go. That's context. We typically call it the context window.

But there is also context of just my business. What am I trying to automate? What am I trying to account for? And that context may be too large. Maybe too large even for my own comprehension. Even for my own explanation, it might be too large. Which means that I may not be able to succinctly, clearly, in a compact manner, I will not be able to, or even, it does not have to be compact, but it has to be clear in a clear manner, I may not be able to express it. There goes context. Context is very hard to keep for us as well as for large language models.

Which brings me to this point of you have to manage your context really well, and this is key. How do you manage context? Generally, for me, again, this is all my experience, the way it works is that I break context down into multiple files. I'm going to show you through one of my examples. This is, I'm going to show you through one of my projects actually.

This is one of my projects where I break down the context, but kind of a repeating pattern, I do this everywhere else. What I have is I have a PRD, which is the project's documentation, the very high level. What does this project want to create? What are the requirements of this project? PRD is generally a project requirements documentation or a product requirements documentation, however you might want to call it. But basically, what is the high level overview of this project? Why does this project exist?

In this particular case, Baho — Baho, by the way, it comes from the Hindi word which means to flow. Baho means to flow. It is a workflow tool, it means flow. The concept of this project is that I can throw a CSV file or an Excel file, which is actually my finances, and I could do basic queries on top. Things like how much have I spent in groceries this month? It should not use large language models, or even if it does, very minimally, because what it does internally is it takes the CSV, figures out what the columns are, and applies these very simple rules. Think of it as some kind of a decision tree. Again, I'm trying not to use technical terms here, but think of some way to figure out is this column supposed to be that, is that column supposed to be blah blah blah, etc. And then come to a point where Sumanth wanted to know this. I can compute this because I can calculate what is the total cost in grocery. This column seems to be like the cost column. This column is the notes column. There is a note, and I will figure out from the note if this has anything to do with grocery. Some simple words, maybe check for the word grocery itself, or maybe milk, or whatever else.

And then just sum them, right? This is what I'm trying to build. It's basically a very simple wrangler — typically you call these kind of software wranglers, so a CSV wrangler or some kind of an Excel sheet wrangler.

Coming back to how I would document my thoughts, I would create a very high-level PRD. Some of it is my thoughts, my actual thoughts as I'm prompting with a coding agent. In this case, I think this was created with Codex, but it does not really matter. They will kind of be the same. So I kind of start brain dumping, like, hey, I want to create a software like this, na na na. Okay, here are the details. And then once I'm satisfied with what it is saying back in the replies, I say, okay, go ahead and create a PRD. And then I'll say, go ahead and create an agent.

This file is specifically for other agents, coding agents in particular, to be able to now keep on building this software. Remember, the PRD is only the one-time brain dump. It's not going to be enough. This is what I am saying. It's not going to be enough. It's not going to be enough context. I have to keep on doing. And how do I do that? That's through the epics.

So then what I do is — this is just common concept from project management. You have the concept of epic, which is like a high-level overview of something big, broad that I want to tackle. What is happening with coding agents is something that I've seen versus what it used to be before AI — is that the epics used to be, at least in my experience, epics used to be larger, and then we would break down into many, many small user stories. So from epics we would create small stories or user stories, and then from stories we would create tasks.

In my experience, what has happened is that I generally start with an epic, which is no more than a day or two days worth of things that I want to tackle. What is happening with coding agents is something that I've seen versus what it used to be before AI — is that the epics used to be, at least in my experience, epics used to be larger, and then we would break down into many, many small user stories. So from epics we would create small stories or user stories, and then from stories we would create tasks.

So then, I'm going to put in that the thought process where, how do I want to tell Bahu what to do, what to figure out, what to calculate? Is it an average? Is it a, I don't know, monthly average? Is it running average? Whatever. So these things which are daily on a regular basis needed by myself in a business, I'm going to think through and I'm going to put them into an epic, talk to a coding agent, and then create one epic. Generally, all of this is not written fully manually by me. They usually never are, but it helps me then I can go through this and I can figure out, hmm, this looks like it has captured what I wanted to do.

So each of these epics you'll see is fairly grounded into a very small, compact thing, which is like I said, a day or two days worth of work in my head. So of course, you can make it as small as you want to. Again, when I'm saying worth of work, I'm not saying from a coding agent's point of view. You will get into the habit the more you do is that you will be able to gauge yourself that for a certain amount of work estimation, I need to also review the work. By review, I don't mean reviewing the lines of code, but actually reviewing the output. So you have to run that software and be seeing, hmm, does it do what I want it to do? That reviewing also takes time.

So I'm just taking care of that entire time has to fit like a day or two days at max because beyond that, I kind of lose track of what was I doing. That happens. That happens to all of us. So I don't want it to be so large that I kind of lose track of it. Hopefully that gets the point across. But that's mostly the structure. It has come down to this after many months, it has just come down to this. I have these three key files. I'm trying to get all of my projects into this. I have a PRD. I have an agents, and I have these epics.

So when I will now start prompting again, let's say I want to continue today, I want to build something. I will say, hey, read the agents. Generally, all coding agents read the agents.md. So I'll say, hey, read the agents.md, PRD.md. In fact, I have those copied. I can show you. These are literally the starting lines I copy and I paste it into a coding agent, and I'll say, I want to discuss, or rather, I would give the coding agent context on what has been done. So I'll say, hey, we have already worked on epic zero zero seven or epic zero zero eight, and then I want to do X Y Z new things.

So I'll simply start there and then have a discussion and ask it, what would be your plan? Give me a very high level plan.

And then what I'll do is I'll probably read that very concise plan, and then say, "I don't like that point, or I want to improve that other point," and so on and so forth. But most of the time, if you look at the chat, and I'll show you — I don't know if a window is open right now. Yeah, this is fairly open. You'll see that sometimes it will have references to code, like here, for example, it has references to code in a particular project because I was asking it to do something. It did, and then it gave me references. But I mostly kind of ignore these now. I mean, most of the times I'm just looking at what it is saying. Okay, it just created a new function. Great. Done. Blah blah. So for me, just knowing a new function has been created, and sometimes just the name of it, is good enough. If you're coming from a non-technical background, I want that to be even gone. But we will get there. It'll take some time. We'll get to the point where you don't even need to care about, did it create a new function or a module? What even is a module? You don't need to know. But what you might want to know from a very high level is that yes, the job has been done. This is the summary of what I have done.

Think of it this way. A very good analogy is, let's say I was your hired engineer. So I was the one keeping context of all the engineering things, while you are a business person who does not have a software engineering background. So how would it work for us? You would give me your brain dump, I would compile, I would do all the code, and I'll come back to you with the status. And I'll say, "Hey, go here and check it out." That is exactly how it will work with AI. That's it. It is not going to be different. If you do not want to have the technical context, hopefully that technical context you will not have to keep. We'll get there, like I'm saying. A lot of the tooling is being improved by a lot of people, so I'm very hopeful that this will work out. But for now, this is what I wanted to express — just a little bit of an insight as to how I work on a regular basis, and I don't read or write code. Hopefully this helps. Keep watching. Let me know if you have some comments. Just share them on YouTube, or you can also find me on any of the social networks. Thanks.