Becoming Agent Enabled Software Builder

Note: This is an AI-assisted transcript and is not a verbatim or accessibility-certified transcript. There are some edits in the text to improve readibility. Text in square brackets, [], are in the video but are not needed for reading, so kept in brackets.

Hey, Sumit here. Back again from my little farm in this tiny Himalayan village. Yeah, it's been fun. I think yesterday's video was really nice. For those who have not watched it, please feel free to go ahead and watch it. But yeah, the general idea was why I believe that working with agents, as in agent-driven software engineering or agent-led software engineering, why I believe that this is the future. And what is my background? Where do I come from? Why do I believe in this?

So today, what we're going to talk about is, I want to keep this entire series as low technical, as non-technical as possible, as basically as open for the audience as possible. But there are of course going to be some discussions which are a little more, they'll have a little mix of technology, but it'll be a little open with regards to the variation of the topics. And today is kind of stepping slowly into what would the future of agent-led programming, agent-led software engineering look like. And if you am interested in getting into this, what does it really mean? Does it mean that you don't need to know anything about software? What do I believe? Where do I see this going ahead?

So I am again coming from, like I said in the previous videos, just for that short introduction, again, I'm coming from a software engineering background. I have many years in the industry, have actually coded with my own hands for many, many years, built teams, built entire engineering, the entire, full stack, let's say, everything from front end, back end, database to actually getting it deployed, running the whole machine. Let's put it this way. And today, for the last, I would say twelve, maybe fourteen months, I no longer write code. I don't write code myself. I mostly work at the planning level, but that does not mean that I am very detached with code. I also use my capacity, my existing skills as an engineer all the time. And I'm not sure how all of these would translate to everybody who's coming from a non-technical background. But that's okay. We'll figure this out. I'm happy to go through this journey along with everyone who is following, who is going to be following along.

And the most important thing to understand is that the models are also improving, which means that there is going to be higher and higher level of abstraction when we are talking to LLMs, when we are having the communication with LLMs. There is going to be higher level of abstraction.

By the time this video, of course, I'm not popular on YouTube, or I'm sure that most people are not waiting to watch my videos. Which means that when you're watching this video probably a couple months down the line, the models, the harness — the harness is basically the agent system which you're using mostly in your computer when you're typing into this software like Claude Code or Codex. They are basically harness or you could say agent and harness. I think you can use them interchangeably. But let's say harness.

So the harness is basically how the model, because the model lives somewhere on the cloud, somebody else's computer. You can also have a bunch of these models locally, but they are generally much smaller. Capable models are mostly on the cloud, given directly by Anthropic, provided by like OpenAI or maybe one of the other inference providers. Basically, wherever the model lives, and you're using it from your computer. Where you're using directly on your computer, if you are using Claude Desktop or any other such software you've probably already used, then that is generally the harness.

So the harness is improving. The models are improving, which makes our lives easier. So if you're watching this video probably a couple months down the line, I'm sure this is going to be much easier.

But what is not going to be easy, and I want to start with that. What is not going to be easy, what is going to keep staying hard. I believe so. Of course, if you ask somebody from within the AI research community who is extremely bullish on where AI is going, I'm sure they'll say I'm wrong. But from my perspective, I think we human beings are really bad at planning.

And why I bring that up is because everything that these large language models know, like even the way they respond in our chats, is actually part of their training. They do not do anything that is outside their training. In fact, I don't think there is any evidence to see where they have done something which is absolutely outside their training.

They are great at remixing things, and because the amount of training that goes into building a large language model is so massive, and it's massive for us, massive from a human beings' perspective, that to us, when you remix that content, what comes out, it seems like, wow, this is absolutely genius. This is unseen. No, nothing is unseen. It is all in the training data. It has been combined in a really interesting way. That is for sure.

But nothing that a model replies with, a large language model, nothing that it replies with, is something that we would, as human beings, we would say innovation. We would see, we would say genuinely new. That does not really happen. Even for us, it does not really happen. If you think about it, like even for us, I don't know who it was, but a friend of mine quoted an artist who is well known for saying this, that we also cannot come up with things that are beyond the universe that we are part of.

And yeah, that's true actually. So none of us can. Even the models cannot, which means that if they cannot, now coming back to the point that I made, that we are really bad at planning. I'll give you an example, very specific example. Let's say you want to build an inventory management software. You start off with, okay, I have this, this, this. I have a, I'm selling X, Y, Z things. Maybe I'm.

Maybe I run a large bakery. Let's take a specific example. I run a large bakery, which means I have a bunch of things I buy. They have to come on time. And the order has to be placed. And then I make something out of it. Maybe I have fifteen people in my team. So we are making a ton of things.

And then orders are coming either offline, as in real stores, I have a storefront, and also online, or maybe these food delivery systems where it is those short within a certain radius kind of delivery, cloud kitchens. Typically, that's I think the word for it. So I'm basically selling on multiple fronts.

Now, inventory has to be tracked for what I have, what I have purchased, what is going to come into my kitchen, and all the ancillary things. Maybe they are also part of my inventory. And then, of course, orders have to be tracked, but that's a separate system. Let's say I'm focusing only on the inventory system.

And then suddenly, what happens? And this is just nature, just our nature. When we have something that starts working, we ask more of it, which is where we will say, oh, but I want this to be also part of my inventory system. Oh, I didn't think that the file which was the source of whatever data was in the Excel, or maybe tomorrow, then, or the day after, ah, that file was PDF or something else.

So there will be changes in how we want this software to operate, what we expect out of it, which basically means is even if you can say, but Sumit, maybe we are not bad at planning, that things change. Yeah, sure. Both are true. Things change. That's for sure. But at the same time, we are bad at planning simply because a lot of things which we already know or can know, we simply don't do it.

Which means we are either delayed on when we gather that information, when we structure that information, when we put it onto some kind of a document. That does cause problems downstream. And that's what I'm talking about. All of this to me is a problem with how we operate as human beings.

We are not super binary. We don't have a list of, okay, I have to gather these twenty things for this software to be built. I will do each one of them. I will be as detailed as I can. I will get all the information from all of my team members, and I will get information from all of my vendors, and I'll double check on these. No, never happens. Which means there will be things left out from those details.

So even if I create a great project documentation, if I, even if I create

10x epics and user stories. By the way, these are probably terms you may not have heard of, but that's okay. We are going to go through those in later videos. But let's say some kind of structuring of like what is my thought about this inventory management system. Even if I put in my best, that best will still probably be my best as in the human average best. And we know from hundreds of hours, thousands of hours of software engineering, you can ask anybody in the industry, and they'll say, yeah, we are really bad at planning.

As best as a team can be, we are still really bad at planning. So all of this documentation, etc., they will miss some of the points. And this, I feel like I said, I'm going to start with what they cannot do because we are bad at it. We don't document it, so it's not like the thoughts come out so clearly and we write them out so well.

Which means now, if we take a step back and think, okay, so if language models, if these AI models, they are trained on what we have documented. Well, if we have not really documented good planning or crystal clear thoughts of that whole process, then well, LLMs also cannot reproduce that. They cannot remix on it.

And there lies my first, I would say, point that there will always be this orchestrator role that we have to play simply because we are bad at it. We'll keep chasing the goals. The tools will improve. We will have a better time, and if we have clarity, code will be better. But sometimes we just don't have clarity, or maybe sometimes we have not thought through enough. And sometimes we want to make changes, and sometimes we want to make changes because the other person is asking for it. All of these things will happen, which means that it's going to be that game where we will, one of us, two of us, maybe within your team, or maybe you as the business leader, will keep on refining your thoughts.

Ask the agent, the harness, the model, whatever that whole system. You'll ask it, hey, I want to make further changes, and this will keep going on. That I believe is going to become the future of software, where

I will, as a business owner, have a more direct connection to what I want, but I will not have much better clarity of what I want than it used to be when AI was not building the software. I don't think that will change as much because that is why we needed also layers of hierarchy in software engineering before. There was a product manager and there was a project manager and there was somebody to do a bunch of these internal discussions with the engineering team, and the engineering team would not want to.

Why do you think that has happened? Even if you do not know of all of these systems, maybe hearing this, you're like, wow, were there three levels of people just to get documentation out of actual business owners? Yes, yes, there were, and they still are. Which means we are really bad at explaining what we want. We are really bad at introspecting what we want. That does not change.

So if that is how it is, and that means that we are also bad at putting it into words, LLMs cannot get better at it. So in the end, I think the structure of software engineering — if you want to build agent-driven software, if you want to be an agent-enabled software engineer, if you want to be an agent-enabled builder, then what you have to really get used to is talking to AI. Because the one thing that they do really well is that if you have clarity, if you ask with clarity, if you know a little bit, if you do a little bit of your homework.

If you ask, I want to build this app, I want to build a mobile app. First of all, if I don't specify mobile app or not, I'm sure the agent will start building it in ways that maybe I don't want. So even just having that much specificity — I want a mobile app, or I want a native mobile app, I want a native iOS app, or I want a native Android app, and so on and so forth. These really matter.

And then of course, down the line, there are other such details. Do I want to connect to a server? Blah blah blah. Do I want data to go out? Do I want something to be saved? Do I want other people to interact? All of these details. Again, we are going to go through them in future videos. But these are what we have to keep in our minds. That's going to be the critical part.

The critical part is not going to be to figure out how is the code going to be. No, actually, we will not care about that code as much as we are going to care about the planning, the thinking. What do I really need? What do I need the software to be? How do I want it to look? How do I want myself to log into it, and so on and so forth. Hopefully that clears out my basic idea of where.

100% or agent-driven or agent-enabled software engineering is going to be. For anybody who is interested in this topic, and if you're a younger engineer, I'm saying this only because somebody actually younger asked me this question today. So if you are one, remember that how you will learn in the era of AI, I will not know. Simply because I come from a background where I did not have AI when I was learning engineering. My hands-on engineering was really hands-on like anybody at my age with my experience will tell you. So I am not the best person to guide you how to learn. I can only tell you there are certain things to learn, but there are two key aspects if you want to take home. One, I've already clarified is that we are bad at planning, which means models will keep staying bad at planning for a lot of things. That high-level planning, I don't think it will magically become better. They will not. The second important thing, particularly from an engineering perspective, if you're learning software engineering and if you want to be an agent-enabled software engineer, is that at the end of the day, computers are all binary. All the programming languages, I don't know how many of them there are, like what, 90 popular ones, or whatever. Let's say the top 40 popular ones, and all of their frameworks, so multiplied by like, I don't know, five, six, eight, ten. So that many, because there are so many of them per language, and all the protocols. So frameworks, protocols, libraries, languages, all of these put together, maybe you'll get to like, oh, there are like 3,000 individual things. Whichever, by thing I mean either a protocol or a library or a framework or language, whatever. All of these, most of these are because we have preferences. Us programmers, us engineers, have preferences. The reason why I don't want to work with a programming language, let's say Python, even though I have worked with Python for like 12 years, I'm just saying. Let's say the reason why I don't want to work with Python and instead I want to work with Ruby, mostly comes down to preferences. That's why there are entire language-dependent stacks which exist simply because we have preferences. Do the computers really care? No.

This is something that we will see a lot with AI-enabled engineering is that we might, as we take a step back from reading and writing code, we might go toward languages which are just better at expressing code, closer to the metal, closer to the computer. How do you get lesser abstraction, lesser of that really thick framework that we use today? Let's say Django is a good example. Again, I come from many years working with Django. So Django, I know, has a lot of magic going on behind. Now, if I want AI to build systems, why would I want AI-based systems to really deal with all of that magic when it could—it's actually really great at writing a bunch of simple code?

So I think there's going to be this paradigm shift where we'll see lesser, more explicit, clearer code, but maybe larger in quantity being generated, where they are closer to the metal, which also gives us lots of other advantages, but we'll not go into that in this particular video. But I believe that practicing working, practicing talking with AI agents is really important because the structure of programming is definitely changing. What we expect out of our frameworks, libraries, languages, etc., they are all going to change. The more we step away from reading and writing code, the more the internal system will be coupled, tightly coupled to the metal it is running on.

And a lot of abstractions, by the way, were also created because of either the agency model where an agency has to probably produce, I don't know, 500 projects a year, or because the concepts came from much larger companies, and these concepts may also not exist in future simply because we may not see the scaling of centralized companies, which is something I talked about in the previous video, in the future as much. More people might want to build more customized, highly niche software. I personally feel that is going to be the future, which means that you're going to want to work with less abstractions.

You don't want to deal with 50,000 servers. So the whole orchestration layer for maybe 50,000 servers is not needed. Most small, medium, large businesses, even medium businesses would be happy with like, okay, I have just 20 servers. So let's not overcomplicate this. I think these changes will happen, but we'll see. But I just wanted to lay out where I feel this will go, which means this will shape how I'm creating this video series as well as all the content for the website prompttogether.com. Thank you for watching.