Automation Obsession: Are You Automating the Wrong Things? – Gojko Adzic “Specification by Example”

Here’s what you can learn from this episode of Pragmatic Talks:
Who is Gojko Adzic?
- A developer at heart: Although he is a famous author, consultant, and trainer, Gojko Adzic considers himself a developer first. He sees creating software as a form of magic–turning words and ideas into real products that people can use.
- Creator of practical tools: His influential work, including the book Specification by Example, came from his own need to find better ways to build software.
The origins and core idea of Specification by Example
- A way to disagree quickly: The main purpose of Specification by Example is to bring together people from different roles (developers, testers, product owners) and help them find misunderstandings about requirements early.
- Using concrete examples: It is often hard for people to describe what they want, but it is easy for them to complain about something specific. The method provides concrete examples for the team to discuss, which helps reveal hidden assumptions and disagreements before any code is written.
- An example of an example: Gojko shares a story about a feature to allow downloads if a user has “enough money”. The development team and the sales director had completely different definitions of “enough money”. A simple example uncovered this major misunderstanding, leading to a valuable discussion and a much better solution.
- Reducing rework and saving time: By clarifying requirements upfront, teams can significantly reduce rework. Gojko mentions that many teams using this method have reduced their development cycle time by an order of magnitude. One team at a French bank had no critical bugs for five years while releasing new software every two weeks.
The role of automation
- Automation is a tool, not the goal: Gojko warns that automation is not automatically good or bad–it is just faster. He says, “If you automate garbage, you just get more garbage more quickly.”
- A check on reality: In Specification by Example, automation is used to check that the software still works according to the agreed-upon examples. This turns the specification into “living documentation” that is always synchronised with the code.
- Making knowledge accessible: When these automated checks are human-readable, they become a source of truth for everyone on the team, not just developers. This prevents knowledge from being lost when people leave the team and makes it easier to work on the system in the future.
Specification by Example vs. ATDD and BDD
- Different names for a similar idea: Gojko explains that Specification by Example, Acceptance Test-Driven Development (ATDD), and Behavior-Driven Development (BDD) all appeared in the early 2000s to solve a similar problem: improving collaboration on requirements.
- Terminology matters for engagement: The different names appeal to different groups. “Testing” might not interest business people, but “behavior” or “specification” might. Gojko chose the name “specification” because it sounded important enough to make business stakeholders pay attention.
- Choose what works for you: He advises teams to “figure out the name that sounds dangerous enough for your organization” to ensure the right people get involved in the conversations.
The evolution of Specification by Example and the future with AI
- Consolidation on Given-When-Then: Over the years, the community has largely adopted the “Given-When-Then” format for writing examples, and many tools have been built around it.
- Potential for AI: Gojko is cautious about using AI to write specifications, because LLMs can “hallucinate” and create incorrect information. The main value of the process is the human discussion, not just writing files.
- AI as a helpful team member: However, he sees a potential for AI to act as a supportive team member. It could suggest interesting edge cases and scenarios that the team might not think of, helping to discover potential problems earlier.
When to use Specification by Example
- For complex situations: The method is most valuable in situations where there is a high risk of misunderstanding. This is common in projects with a complex business domain or in large organizations where information can get lost between teams.
- When it is not needed: It is not necessary for very simple tasks, like changing a button color. It is also less useful for requirements that are not deterministic (e.g., making a video game “fun”), where prototyping and user testing are better approaches.
- Key preconditions: For the method to work, there must be a domain expert who can confirm whether an example is correct or incorrect. It is also most beneficial for products that need to be maintained and developed over a long period.
Full Transcript
Introduction
Gojko Adzic: In this episode, people are obsessed with automation, and they’re going to come up with new ways of automating things that don’t need to be automated. Our industry is all about automation, so we have a bias for automating things. We like automating things; we are developers. We consider automation mostly as good, and it took me a very long time to understand that automation itself is not good or bad, it’s just faster. If you automate garbage, you just get more garbage more quickly.
Wiktor Żołnowski: Welcome to Pragmatic Talks, a podcast and video series where we discuss startups, contemporary digital product development, modern technologies, and product management. This episode is brought to you by Pragmatic Coders in collaboration with ACE!, an agile software development and product management conference. We believe that everyone should have equal access to knowledge about product development and entrepreneurship, and also, everyone should have the opportunity to apply it in pursuit of making our world a better place. Through this series, we aim to create an impact on the future world. In today’s episode, we are joined by Gojko Adzic. Gojko is a highly regarded and award-winning author, consultant, and trainer in the software development community. Known for his influential book Specification by Example, Gojko has played a pivotal role in shaping modern development practices. His innovative approaches have helped countless teams improve their collaboration and product quality. In this episode, we dive into the practical aspects of Specification by Example and explore its impact on reducing misunderstandings in complex domains. We also discuss the evolving landscape of software development, the role of automation, and the potential future of AI in supporting development processes. And now, ladies and gentlemen, please welcome Gojko Adzic. Welcome to the next episode of Pragmatic Talks. We are today at the ACE! conference again, and today with us, we have our great guest, Gojko Adzic. Welcome, Gojko.
Gojko Adzic: Thank you for inviting me.
Wiktor Żołnowski: It’s a great pleasure to have you here, especially since I know your work. I’ve known you since 2010, probably, or something like this. And I remember when I read for the first time the Specification by Example book that you wrote, that was something for me that was very new on the market at the time and something that clarified a lot of things around how to talk about requirements, but also how to put the requirements into action and how to turn this requirements specification into tests – automated tests. At that moment in time, that was pretty innovative. Maybe right now, for most people, this is something that everyone is doing, but actually, you were one of the innovators at the time who started this movement, most probably. And most probably, you are the one who propagated it into the market.
The origins and core idea of specification by example
Gojko Adzic: Thanks very much for saying that. I really appreciate it. I mean, those ideas actually come from the mid-80s or late 80s. And if you read Jerry Weinberg and Donald Gause’s books, Exploring Requirements for example, the core ideas are there. Ward Cunningham did a lot of that stuff, and I think he was the true innovator, figuring out how to do these things in the late 80s, early 90s. But in the early 2000s, that was still like magic; nobody really knew it that much. And as you said, today it’s kind of common knowledge, and I’m glad to have played a small part in it. One of the things that really inspired me to do the book is I was at a conference, very much like this, in 2007 or ’08. There was a presenter on the stage talking about how all of the stuff around agile requirements is bullshit because it’s all theoretical; nobody’s actually doing it that way. And I thought to myself, “Well, do I live in some unicorn land? What am I…?” because we just did a really successful project like that. A couple of people I knew from the community were doing really well there. So my goal with the book was not just to explain how to do these things, but actually to document that there are real teams out there that got real benefits out of that. And it was all provoked by this person on the stage saying, “This thing doesn’t happen.” I wanted to show how, yes, it does happen. Maybe it doesn’t happen in your world because you’re not doing it right, but it doesn’t mean that everybody else is stupid. I think a big part of why that book was successful is that it’s actually real stories of real people, not just theoretical knowledge. So yeah, I’m really glad that you liked it and that it made a difference.
Wiktor Żołnowski: Okay, so you’re an author, you’re a consultant, a trainer, but tell us a little bit more about yourself. Who is Gojko Adzic?
Gojko Adzic: I’m a developer. At the core of everything, I never really wanted to do anything else apart from write software. When I was a kid, I started writing software by copying and pasting code from German instruction manuals for a Commodore 64. I had no idea; I still can’t read German. But there was German, German, German, and then some lines of code, and then German, German, German. I would copy and paste those lines of code on a Commodore 64, and the screen would go flashing and things like that. It was magic for me. Since then, I’ve never really wanted to do anything else apart from developing software. And all of this other stuff that came out – Specification by Example, Impact Mapping, and things like that – they’re all like side things I had to learn to build software better. I still, most of the time, code. I build my own software products, and that’s, for me, the most magical thing there is. If you look at those legends from the Middle Ages and movies about medieval times, there are wizards. I was always fascinated by wizards. And wizards come and they say something, and their words turn into reality. You say “abracadabra,” and something happens over there. That’s what we do with software. We type some words in an editor or do something, and those words turn into the real world. People interact with our products, and that’s magic. That’s kind of, for me, I’m completely fascinated by that. I like magic. I like building things out of ideas and words and letting people interact with that and see where it goes.
Wiktor Żołnowski: Gojko, a developer, magician, wizard.
Gojko Adzic: Perfect.
Wiktor Żołnowski: You did not expect this.
Gojko Adzic: I didn’t, I didn’t.
Wiktor Żołnowski: Let’s try to get back to Specification by Example. Some people who are watching this most probably are not familiar with Specification by Example or have never heard about it before. So if you could outline a few things about Specification by Example, what’s most important in this approach?
Gojko Adzic: Spec by Example is a very quick way of getting people from different roles – most usually developers, testers, customers, product people, UX people – to quickly disagree about the key things they need to disagree about, so that they can then reach a shared understanding and an agreement on what needs to be built. The idea at the core is that it’s very difficult for people to say what they want upfront, but they can complain very easily. And if we give people something concrete enough to complain about, then they will get the disagreements out quickly, rather than going and implementing the software, sitting down, and then complaining. As an example of an example, I worked with this company where they were trying to allow people to download digital clips if they have enough money in their account. There was a bunch of things they wanted to solve with that. And so you’d have this function where you put some money into the account; if you have enough money, you’re allowed to download, and then you get the money deducted. There were two examples there somebody came up with: you have €10 in the account, what you’re trying to download costs €5, you’re allowed to download, your account is left with €5. Or you start with €5, the thing you try to download costs €10, you’re not allowed to download, you’re still left with €5. And then what’s really magical about that is you can get these boundaries and edge cases because the misunderstanding usually comes at boundaries. Everybody’s going to agree on a happy day scenario. So we gave two testers five minutes, and they came up with an example of: if you have €9.99 in your account, what you’re trying to download costs €10. And everybody said, “You’re not allowed to download, and you’re left with €9.99,” apart from the sales director, who said, “You’re allowed to download.”
And we looked at him and said, “Why?” He said, “It’s in the spec.” “Where is it in the spec?” I mean, first of all, we don’t have a spec. You just gave us one line. It says, “Allow downloads if you have enough money in the account.” And he said, “Well, there. If you have enough money in the account, look, ‘enough money’.” And then we realized that his definition of “enough money” is completely different from our definition of “enough money.” And all these assumptions came out about profit models, about profitability, and why and why not, and rejection rates on cards. And it was this whole thing that exploded then for half an hour of very good discussions. And then when it kind of consolidated, we understood a lot more about the problem he was trying to solve. And his solution was so insanely complicated for that that it would take like seven man-millennia to do the job. But one of my colleagues said, “Look, in the first iteration, how about we allow people to go like half a euro into credit, and then we do all this complicated stuff later?” And he said, “Okay.” And then we did that, and for years later, they didn’t touch it. And this decision made this company a lot of money because of the whole thing. So those kinds of things usually come out after you develop software, because developed software is the first time people can interact with it, the first time people can really complain. And then all these additional problems come out. But if we can put concrete examples in front of people to get all these arguments out, then we can reduce rework significantly. We can improve the quality of the software. And most of the use cases in the Specification by Example book, people have reduced their cycle time by an order of magnitude because of this whole removing rework that doesn’t need to be done. They were able to align testers and developers and product work into much, much tighter cycles and actually get this whole quality – as in, it does what it needs to do, it’s not broken, there are no surprises there – under much better control. There was a fantastic use case of a team from a big French bank that I interviewed for the book, where they were working on a mission-critical project for a bank, and for five years, they had no Priority One bugs, which is unheard of. And these people were releasing every two weeks. It’s not like they’ve done one version and then for five years they didn’t do another version. That happens in some banks, but at the time when I wrote the book, it was almost impossible to get people to understand that yes, these things exist. But as you said today, luckily, lots of teams do have this as common knowledge.
Wiktor Żołnowski: Yeah, I remember that at that time, in 2010-ish, ’12, ’13, ’14, it was pretty uncommon for people to actually bring all the knowledge into one room. Usually, there was a time that a tester was sitting in a different room, developers or programmers were sitting in another room, some business analyst in another room – not even talking about the stakeholders or domain experts who are usually in a totally different building or city or whatever. So bringing all of the people together and providing them a framework that they can use to talk to each other and to visualize how they are thinking about the product and the requirements, that was something that was really new for many of them. But at the end of the day, right now, as you said, most of the companies, or maybe not most, but many companies work like that.
Gojko Adzic: I wasn’t involved in the software industry in the 50s and 60s; I wasn’t born then. But my understanding was that up until probably the early 80s, stuff was done by people in the same room. And then the industry went through this crazy phase where everything was specialized, and you had… I mean, up until then, the same people were testing and building the software. That was just your job. But then you had testers, you had developers, and everybody had their own roles, and project management got involved and things like that. And it took like 20 years for people to realize that’s a bad idea. So now we’re going back and making it simpler and easier. But then still, you have all these scaled agile frameworks now that are just making things more complicated again.
Wiktor Żołnowski: Yeah, that’s true. Exactly, that’s true.
The role of automation
Wiktor Żołnowski: In terms of Specification by Example, what was also really interesting was that the goal was to put these requirements, the specification, into action and turn it into automated tests. So could you tell us a little bit more about it?
Gojko Adzic: I don’t think that was the goal; that’s a nice side effect of things. And lots of people think about automation as the goal. And because our industry is all about automation, we have a bias for automating things. We like automating things; that’s what we do. We are developers. And we consider automation mostly as good. And it took me a very long time to understand that automation itself is not good or bad; it’s just faster. If you automate garbage, you just get more garbage more quickly. If you automate a good process, you do get something well. So, the idea with automation in Spec by Example is that once you have some kind of common understanding documented through examples or something like that, there needs to be a reality check that the reality did not move. Products evolve. And you have, six months ago, you had this idea that this is the definition of “enough money,” but then things evolve. And especially in complex organizations, things evolve. So six months from now, the definition of “enough money” might change. It might change in the code, but then you no longer have a spec that reflects reality. And the idea was basically to automate the validation of that, so we can keep a check on reality and we can make sure that the documents are in sync with the code. And because that needs to be checked over and over and over and over and over again, automation is a natural solution for that. Now, if we do automation in a way that humans can still understand, then if I change something over here and something over there changed, then we can show a document that explained this to a human and say, “Look, this no longer works like that. Now, is it a bug? Is it something we forgot to talk about? Do you actually want this, it’s just we didn’t talk about it? Is it like, ‘Oh, we really did not understand that this is going to be impacted’?” And having automated tests that are human-readable, through again, examples and things like that, allows us to have that conversation. And again, that saves a ton of time.
And it allows us to then use these documents as a way of starting a discussion next time. So if I’m going to change some kind of payment processing a year after I’ve done it, I don’t need to do the analysis over and over again. I can just say, “Look, this is what we agreed last time. Because this test is green, I can still trust it.” I don’t know how many times I’ve joined a project where people have told me, “This is the spec,” and then they said, “But don’t trust it completely; it was written like two years ago.” And this is where I think, again, a lot of the waste that we have in software processes comes from: knowledge that was lost or knowledge that we don’t know we have. There’s a famous quote by Lew Platt, who was the CEO of Hewlett-Packard for a long time, and he said that if Hewlett-Packard only knew what Hewlett-Packard already knows, we would be three times more profitable. And that’s the problem with this knowledge that’s stuck in people’s heads. If we externalize that knowledge and make sure that it’s valid, then we can spin things up. So automation really is there just as a way of making sure that these things are still valid. And I think where, again, because things move so quickly, especially if you’re developing and releasing software every couple of weeks, if multiple teams are working on the same software, having this set of automated tests that we can use as a source of truth becomes incredibly powerful. Because in lots of cases, code is the only thing you can really trust. And developers can read code, but then they become a bottleneck for information. If a tester needs some information, if a product person needs some information, they have to go to a developer. And if we have this knowledge externalized in the form of automated tests, but human-readable automated tests, then it’s accessible to everybody. And then it does become this institutional memory that survives changes in teams, changes in ways of working, changes in technology, and it becomes really, really powerful. Especially because technology changes all the time, and if you need to migrate from one database to another, your business rules have not changed. And having the business rules there, documented well, speeds up the whole thing.
Wiktor Żołnowski: That’s true. I still meet a bunch of people who are working in projects where they are not automating tests at all; they do not have automated tests. And usually, they also do not have any specification at all, or not even documentation that’s out of date, but just no specification. They’re just happy coding for years. And what I see is that visibly, they are slowing down with the process with every new line of code. It gets slower and slower and slower to add new things, to change things, to modify. So having both specification and automation in one place and doing it once is actually like cutting the cost by 50% of their time.
Gojko Adzic: And as you said, enabling you to keep working at a sustainable pace. Where I think, yes, you can get something live quickly and not care about sustainability, of course. But if this is something that needs to stay with you for the next couple of years or something where the core of your business is going to be built on, then it needs to be sustainable. It’s not just about getting it out now; it’s about preparing for the future as well. And although you can’t really predict what’s going to happen in the future, you can prepare for change. And not doing that is irresponsible.
Specification by example vs. ATDD and BDD
Wiktor Żołnowski: So how does Specification by Example differ from, for example, Acceptance Test-Driven Development or Behavior-Driven Development?
Gojko Adzic: That’s a philosophical question. It’s a terminological question. And I think in the early 2000s, there were lots of concurrent developments happening on this because the early 2000s were a time where the XP community figured out how to do development in a good way, but we were still trying to figure out how to integrate testers into the thing, and business analysts, and things like that. So lots of people were trying to find overlapping solutions for the same problem. And again, these ideas have been around at least from the mid-80s, and people have been doing these things. Craig Larman approached this thing more from a perspective of using acceptance tests to get people to clarify things, and that’s where Acceptance Test-Driven Development is as a phrase. Dan North and Liz Keogh were working with Thoughtworks at the time, and Chris Matts, and they were trying to get business people more integrated into this. And business people don’t care about testing, so they were trying to look at system behaviors and explain this thing more from a behavioral perspective. And there were a bunch of other names thrown around at that time. Some work I was doing around that time, we couldn’t get people to engage in writing examples or writing tests because they didn’t care about that. But the name “specification” sounded dangerous enough so that business people wanted to be involved. And that’s why I started calling this more specification-oriented stuff. And for me, it’s all the same thing, really. But Behavior-Driven Development, again, it evolved into something around Given-When-Then. And that’s a very interesting thing how it evolved. Dan North, who came up with the name, had this presentation in, I think, 2010 where he talked about how Behavior-Driven Development is a whole methodology for developing software because for a while, BDD was whatever Dan North was interested in. And he talks about pull-based, flow-oriented things. So if you look at it from that perspective, if BDD is a methodology, then Spec by Example is the requirements part of it. If BDD is just this conversation piece and collaboration piece that again Liz Keogh talks about, then Spec by Example and BDD are the same thing. Again, we’re getting into different terminologies. And I think for different organizations, different terminology works better. For some people, “specifications” are more dangerous than “tests.” For some, “tests” are more dangerous than “specifications.” Dave Farley talks a lot about continuous compliance because he used to work with organizations where they need to prove regulatory compliance around the requirements process, and doing this allows them to continuously prove your compliance. So I think the message there really is: figure out the name that sounds dangerous enough for your organization and use that.
Wiktor Żołnowski: Dangerous enough so business people will be involved.
Gojko Adzic: And everybody who needs to be involved is involved.
The evolution of specification by example
Wiktor Żołnowski: So it’s 2024. Has anything changed? Has the Specification by Example framework that you described had to change recently, or is it still up to date and still used widely by people?
Gojko Adzic: A couple of things have changed in the last few years. First of all, for whatever reason, the community seems to have consolidated on Given-When-Then as a way of describing these things. And there’s some good stuff about it, there’s some bad stuff about it. But when we were starting out 20 years ago, there was a lot more variety in how you were describing examples. Some people were describing examples with tables, some people were describing examples with flowcharts, some people were describing examples with words and bullet points, and there was a lot of innovation in that space. But today, most people seem to go through this Given-When-Then format, and there’s a whole ecosystem of tools that evolved in that format. That’s kind of interesting. And the other thing that happened, I think, is we’re getting more and more of these decentralized systems. 20 years ago, systems were a lot more monolithic; now they’re more smaller pieces that are talking to each other, and then these things are more broken into smaller bits and pieces. Something that is coming – it’s not yet there – is how all these LLMs will be integrated into that. I get one email a week from somebody who started a startup and thinks that they’re going to change the world by getting ChatGPT to write Given-When-Thens. It’s very difficult to engage with that because for me, the big value of Spec by Example is getting humans to complain quickly. Optimizing how you write 700 Given-When-Then files is not really the bottleneck we need to solve. But there is something there, people are trying to do that. There is something there, and I don’t know what’s going to happen with that. For me, saying, “Oh, we had this hallucinogenic AI robot write our specifications” is insane because who knows whether they’re right or wrong? And if you have somebody generating a ton of bullshit, then you’re creating another bottleneck of somebody else having to review it and understand what’s going on. But I think there might be some kind of collaboration thing going on there where things get suggested or proposed, or some things that might not necessarily be considered get evaluated. One of the things that really helps Spec by Example is having testers in the room to propose these edge cases, like €9.99. The developers are not necessarily good at proposing these things because developers are thinking from a builder mentality, like, “What does it need to do in the core scenario?” and things like that – positive thinking. And business people really care about the end results. But having all these examples is really powerful, and having somebody suggest that is really useful. And when I was still working as a consultant, I met with lots of teams where they did not actually have anybody who’s good at that. And maybe that’s where AI and LLMs can act in this adversarial role and generate lots of these interesting things to consider. That might be an interesting thing where things will evolve. I don’t know. People are obsessed with automation, and they’re going to come up with new ways of automating things they don’t need to be automated and they don’t need to be done at all.
Wiktor Żołnowski: They don’t need to be done at all.
Gojko Adzic: But we’ll see. We’ll see. I think it’s a wonderful time to be alive.
Wiktor Żołnowski: Yeah, that’s true, that’s true. Regarding what you just said, I think that in one of the previous episodes with Joe Justice, we were discussing Mob AI a little, and Mob AI is something that you actually described: that the AI became part of the team, which is supporting the team members in this kind of thing, like discovering the things that they wouldn’t discover otherwise.
Gojko Adzic: And from that perspective, I think there is some interesting potential there to do that. I remember when GPT-2 was out, I was fascinated with that, and I was showing this to a friend of mine who’s a business analyst. We looked at this thing doing a bit of writing business analysis documents. And in England, there’s the gambling sector; I spent a bit of time working there, and this friend of mine works there. There are these horse racing rules, and Rule Number Four is really, really complicated. It gets people to recalculate history, and that’s a good test for a business analyst, effectively, to get them to explain this complicated rule in a simple way. And it’s often used in interviews to do that. And I got GPT-2, and I said, “Look, explain Tattersalls’ Rule 4 with examples.” It did a perfect job. Absolutely perfect job. And this friend of mine was like, “Fuck, I’m going to lose my job now.” And I said, “No, no, but wait, wait, wait a minute.” I said, “Explain rule number five.” Rule number five does not exist. But ChatGPT brilliantly explained rule number five with interesting examples – complete, random garbage. And I think that’s the problem with these things. If you use them as an inspiration, if you use them as something that helps you speed up your process, then I think that could be quite powerful, of course, if you revisit what they are creating and check if it’s true or not. Because as you said, it’s still hallucinating, maybe less than GPT-2, but still, you need to be aware of that.
When to use specification by example
Wiktor Żołnowski: Okay, so two questions combined together. One question is: for whom is Specification by Example? Or rather, to which projects or which part of the requirements does Specification by Example work well? And are there any areas where Specification by Example is something that is too much or maybe too little? And the second question combined with that, that also came from my experience when I was trying to introduce, or actually I was introducing, Specification by Example to various teams. I’ve seen that some teams were struggling with using it, and some were struggling with using it because they see that they cannot figure out the proper examples. And other teams were like, “Why are we using so heavy a framework for such obvious stuff that we just need to ship? It just needs to be done.” So what do you think?
Gojko Adzic: So I think Spec by Example is useful where there’s a big potential for misunderstanding across different roles. Because that’s the problem it solves. It solves the problem of finding out six months later that we missed something important or finding out three weeks later that we missed something important. Usually, that is when the domain is complicated, so there’s a potential for developers and testers not to really understand the business domain, or where the organization is complex enough that there’s an opportunity for information to be lost and not communicated well. So if you have a very small team and they’re all incredibly technical, it’s still probably a good way to clarify things very quickly with examples. But you don’t need the rest. You don’t need writing it down in a human-readable form because everybody can read code. You don’t need this collaboration between testers and developers because everybody’s already collaborating well. In a trivial case, if you have a single-person team and that team has communication problems, Spec by Example is not going to help. But the bigger the team grows, the more likely you will have people at different levels of domain knowledge, people with different levels of knowledge about the system and how things work, how they should work. That’s where Spec by Example fits in really nicely. If you just need to change this thing from blue to yellow, there’s not much point in doing 20 examples of what kind of yellow. As you said, there’s no opportunity for misunderstanding there that much. And another aspect where Spec by Example has a requirement is that examples are deterministic. So there needs to be like, “In these conditions, this thing happens.” And, “In these conditions, we allow the download. In these conditions, we don’t allow the download. In these conditions, you’re left with €7.35 in the account. In this condition, you’re left with €8 in the account.” For things that are not deterministic like that, examples are not a good way to describe things. One illustrative case of that are video games. The key thing for video games is to be fun. It’s not to be correct; it’s to be fun. And “fun” is very difficult to describe with deterministic examples. So Spec by Example is not useful for that. But what they do in video games is they build a prototype of a level, and they play-test it, and they do user research and things like that. Those are much better things. If nobody has the knowledge of what needs to be done, then it’s too early to do Spec by Example. Then you can do market research, you can do customer journey research; there’s lots of other stuff you can do. But if there are no domain experts, then nobody can tell you if this is five or six. Then examples are kind of pointless because you just have a bunch of questions. Again, there are different techniques. So I think the preconditions for when Spec by Example applies are where the domain is complex enough to have a potential for misunderstanding, the organization is complex enough that you have an opportunity for information to be lost or misinterpreted, and that there is somebody actually to say, “This is right,” or “This is wrong.” And there are perfectly valid cases outside of that, and Spec by Example is not good for that. So a two-person team that works on a trivial domain, don’t do this. It’s pointless. Doing something that’s highly technical, like building a new TCP protocol implementation, you probably will have some examples of how this thing needs to work, but getting 20 people in a room to discuss edge cases for that is probably not that useful. But figuring out how to do trade reconciliation in a banking system, probably yes, because there are lots of opportunities for misunderstanding there.
Wiktor Żołnowski: Yeah, and not so many people know how to do it. Like there are maybe just a few domain experts in the bank who actually know how to do this.
Gojko Adzic: Yeah, and maybe sometimes they don’t agree with each other.
Wiktor Żołnowski: Yeah.
Gojko Adzic: And again, one of the things you mentioned is: does this thing need to be long-term sustainable? Are we building something that just needs to be there for the next week? If yes, go and do it. Don’t… just do it. If it’s wrong, do it again. If you’re building something that needs to be there five years from now, you probably want to have some institutional knowledge that will survive all the team members that built it leaving the company, or half of them, or somebody going on maternity leave, or somebody getting sick, or just moving to another project.
Wiktor Żołnowski: Moving to another project, and then we do need this kind of knowledge in some other shape or form, not just in the code.
Learning more
Wiktor Żołnowski: Okay, so if someone would like to learn more about Specification by Example, what should they look for or where should they look for more information?
Gojko Adzic: Well, the Specification by Example book is a good starting point. There are lots and lots of resources online. As you said, it’s more or less not a new thing anymore. I wrote another book with Dave Evans and Tom Roden called 50 Quick Ideas to Improve Your User Stories, that’s quite good. Yeah, I guess just Google for “Spec by Example.” There are lots of resources there.
Wiktor Żołnowski: But you’re also leading trainings and providing courses.
Gojko Adzic: Yes, I still do workshops. And in Poland, we are partnering with ProCognita. So if somebody from Poland is watching this and you want to learn more about it, go to the ProCognita website and look at the Spec by Example courses. Yeah, thanks for letting me plug that.
Wiktor Żołnowski: Yeah, our friends from ProCognita are always welcome to be promoted here as well.
Conclusion
Wiktor Żołnowski: Okay, so thank you, Gojko, very much for the discussion about Specification by Example. So I hope to talk with you soon.
Gojko Adzic: Thank you for inviting me to do this.
Wiktor Żołnowski: Thank you.
Pragmatic Talks is delivered to you by Pragmatic Coders, the first-choice software development partner for startup founders. Be sure to catch all new episodes. Subscribe to our YouTube, Spotify, or Apple Podcast channels. And if you are thinking about building your own startup or struggling with product development, contact us and find out what we can do together.



