Inside an Agile Organization

Here’s what you can learn from this episode of Pragmatic Talks:

The three core pillars for company growth

Wiktor Żołnowski and Jakub Dobosz explain that for a company to grow without major problems, it must focus on three areas at the same time: product, people, and processes. They share the story of how Pragmatic Coders used this approach to scale from around 60 to more than 100 people in two years, building a company that is now largely self-managing.

Developing better products

• Always start with ‘why’: The most important tool is to constantly ask “Why do you need this?” This question is asked to clients and within the team to make sure everyone understands the real user need behind a feature. It helps focus all work on delivering real value.
• Standardized tools build trust: The company uses well-known tools like the Product Vision Statement and Lean Canvas. Using market standards helps new people and clients to understand the process quickly and builds trust, rather than inventing new methods.
• Product discovery is a continuous process: Product discovery is not just a single workshop at the beginning. It is a continuous process of learning. The initial workshop focuses on defining the problem and the offer, which are then quickly validated with potential users. This avoids building the wrong product.
• Team empowerment leads to innovation: When teams deeply understand the business goals and user problems, they become empowered to find better solutions. Wiktor shared an example from his startup where the team, after understanding the users, proposed a faster way to go to production, even pushing back on his initial plans.

Standardizing processes for efficiency

The role of checklists in maintaining standards

Checklists are a simple but powerful tool at Pragmatic Coders to maintain standards and guide the company culture. The goal is not to compare teams but to help each team track its own progress over time.

• Scrum Health Checklist: A list of 67 questions that helps teams evaluate how well they follow the Scrum framework and where they can improve.
• Technical Health Checklist: Ensures that important technical practices, like CI/CD (Continuous Integration/Continuous Delivery), are consistently used.
• Product Checklist: This checklist is structured around four key areas of product development: strategy, discovery, delivery, and leadership.

Ensuring quality without dedicated testers

Pragmatic Coders does not have dedicated software testers. Instead, they build quality into the development process from the very beginning. They believe preventing bugs is more efficient than finding and fixing them later.

• Developer ownership: Developers are responsible for the quality of the code they write. They cannot pass this responsibility to someone else.
• Automated testing: The teams heavily rely on automated tests and CI/CD. This allows them to make changes to the product without fear of breaking something.
• Whole-team approach: The entire team is involved in quality assurance. This includes practices like pair code reviews, where two developers review code together, and exploratory testing, where team members try to “break” the application to find weaknesses.
• Process-driven quality: Good processes like a clear Definition of Done, thorough backlog refinement, and using feature flags help prevent misunderstandings and bugs before any code is written.

Cultivating people and culture

Knowledge sharing and experimentation

• Knowledge guilds: People in similar roles (e.g., Product Owners, UX Designers, Node.js Developers) form “guilds”. These groups meet regularly to share experiences, solve problems together, and establish best practices.
• A culture of experimentation: Teams have the freedom to experiment with new processes. For example, one team tried Large-Scale Scrum (LeSS). Even if an experiment fails, the lessons learned are shared with the whole company, so everyone can benefit.
• Lean Coffee format: A simple and effective meeting format used across different guilds for open discussions. Its standardized structure makes it easy for people to join different meetings and contribute immediately.

Onboarding and continuous learning

• Structured onboarding: Every new person goes through a detailed onboarding process. This includes training on the company’s values, processes like Scrum, and meetings with different departments. This ensures everyone is on the same page from day one.
• Long-term development: Learning does not stop after onboarding. The company offers advanced training, for example, a 13-week advanced Scrum course, to help people deepen their knowledge and share their experiences from working in their teams.

Leadership and evaluation

• People-focused leadership: Every person has a Team Leader who is responsible for supporting their personal and professional development through regular meetings.
• Values-based evaluation: Performance evaluations are based on the company’s values and focus on real-life examples and feedback from colleagues. The goal is to identify areas for growth and align personal interests with the company’s needs.
• Team and individual focus: The company evaluates success at both the team level (using checklists) and the individual level. This combination helps develop strong teams and supports personal growth.

Wiktor Żołnowski: Welcome to Pragmatic Talks, a podcast and video series where we discuss startups, contemporary digital product development, modern technologies, and product management. Pragmatic Talks is brought to you by Pragmatic Coders. We believe that modern methods and approaches to software product development should be widely adopted in the market to minimize the wastage of talent. We also believe that everyone should have equal access to knowledge about product development and entrepreneurship. Everyone should have the opportunity to apply it in the pursuit of making our world a better place. Through this series, we aim to create an impact on the future world. For this episode, we invited Jakub Dobosz, another great at Pragmatic Coders, where he is responsible for the standardization of product and people development processes. Today, we will talk about three dimensions of processes and tools that any successful company should, sooner or later, implement to grow without major problems. This discussion is focused on three areas: products, people, and processes. If you are working on designing your organization’s structure and processes, this talk is definitely for you. And now, please welcome Jakub Dobosz. Welcome, Jakub.

Welcome. In today’s episode, we are going to discuss all of the methods and processes that you may find useful when growing your company. We will try to categorize them into three categories: product, processes, and people. Why those three categories? Because I believe, and I hope that you also confirm, that any growing company needs to focus on those three aspects and grow in these three aspects in parallel. We are going to provide you with some examples from our company, from Pragmatic Coders, and we will tell you the story of how we grew Pragmatic Coders from around 60 people to more than 100 people in the last two years. It wasn’t easy; it was a long journey, but we learned a lot. We made some mistakes, but as a result of it, we ended up with a company that is self-managing, and we have proper processes and some methods implemented, so we avoid a lot of problems, a lot of issues that we faced in the past before those transformations. So Jakub, maybe let’s start with some examples of tools, processes, and methods that we are using in the area of, let’s say, product development and product discovery.

The three core pillars: Product, people, and processes

Jakub Dobosz: Okay, I totally agree with you that we need to pay attention to these three areas because they need to go together. We cannot, for example, start with great product development, let’s say in one product, because okay, it might give us as a company some advantage. We can develop this product very well and have great referrals, but what will we do next? Like, what will happen if we have another product? We need to take care that we have procedures that help us, processes that help us to pass the knowledge, to maintain the same quality. And there is a need in terms of working with various people in order to provide them with knowledge, but also with skills, with competencies, with tools. So definitely, they need to go together.

When it comes to product discovery and product management, what we really aim for is to understand the needs–understand the needs of partners that we work with and just ask the question, “Why do you want to do that? Why do you need it?” And it actually expands towards different areas. We treat our partners, our clients, like this, but we also treat ourselves like this. So for example, when a product owner comes with some idea, talks about some goals, some next steps when it comes to product development, it’s usual for the team–for developers, designers, and so on–to ask the question, “Why? Why do we want to do this? Why do we want to jump into this particular area of the product or fulfill this particular need?”

So what we are doing, we are constantly involving all people in this process. Of course, you might say that a product owner is accountable for the product, product discovery, to work on the roadmap, to work on goals. And at the same time, we invite everybody in the team to take part in it and to be able to provide their feedback on a very early stage. Because we understand that building products is a composition of various aspects–of course, the vision, the need, and so on, but also how it can be built and what we can actually do.

It might sound simple, but the first tool is truly just focusing on the need and making sure that we understand it. The second thing that we do, and it kind of touches also the process part, is we aim to have a standardized approach in terms of tools. And although, for example, we use a Product Vision Statement or we lean towards Lean Canvas just to fill out with our clients, then we have roadmaps that we build based on that. Of course, there is a whole area of UX research that is also standardized, and then we go towards development and delivery practices that help us maintain high quality and actually deliver value to our clients. And last but not least, the third one is also connected with the first one: focusing on value. So when we ask, “Why do you need this?” the second thing that we also think in the back of our heads is, “Okay, when we get the answer, does it bring value?” And also, together inside our teams, we constantly talk about the value and if it brings–currently at this current stage of the product development–if it answers some of our questions or it will provide us with some information that might be helpful in the future. So I think, in general, these are like the three most important tools or, let’s say, items that we care about in terms of product development.

Standardizing the product development process

Wiktor Żołnowski: So maybe before I ask you more about the standards that you mentioned, it’s worth mentioning that at Pragmatic Coders, we are usually working with a few clients in parallel; right now it’s around 13 products, as far as I know. However, sometimes it’s even more. In the past years, we had a lot of chances to actually experiment with different approaches, and those standards that you will be talking about in a moment are mostly based on our experience and are battle-tested on many, many different products in many different contexts. And they are all still being improved all the time. So let’s get back to those standards. You already named a few tools that we have standardized, but can you tell a little bit more about them? Like, what is standard for our, let’s say, product vision statement?

In terms of product management and product development, we identified four areas that we focus on, and we check how we are handling these areas in a particular product, in a particular cooperation with a given partner. So the first one is strategy. In strategy, we have a couple of questions that we ask ourselves, like if we understand it, if we talk about it. And these are examples that will show us that we are on top of the strategic level of product development. The second area is discovery, so if we are properly figuring out the needs of potential customers, if we see what can be done next, and so on. The third one is delivery, so we are checking in terms of the product if we are actually delivering value to our partners and to their customers and how we are actually handling it. And the fourth one is leadership because we see that, in terms of the thing that I mentioned earlier, together as product teams, we are involved in the whole process. So leadership means that we can lead all areas that we are accountable for, we are responsible for, starting from working on the product items like product vision statements and so on and so forth, also UX research and product development itself. So actually developing some solutions is also important for us because some of our clients also expect us–and it’s perfectly great that we can help with that–to work also on processes and to lead them throughout this journey, I may say, when it comes to product development and product management.

When it comes to the standards themselves, what we are aiming for is to use standards that are available on the market. It’s easier to have the same language with people that come and work with us. It’s quicker to understand each other, and it’s actually quicker to build trust. Because it’s all about trust in all areas at the end of the day. Instead of inventing the wheel again and again, we are using things that are available in the market, which is also not easy because there are people who often claim that they know some tools or have heard about some tools, but in practice, they use them in the wrong way. And we are only human. Standards–this is something that we’ll be repeating here like many, many times.

The role of checklists in maintaining standards

I remember that very helpful for us as an organization was creating quite a few checklists. Could you tell us a little bit more about those checklists? Maybe you have some examples of the questions or things on the checklist and how we are using them? How our teams are using them?

Jakub Dobosz: Sure. So, what I already described, these four areas like strategy, discovery, delivery, and leadership–in these areas, we are checking product management and how we are handling them in terms of, for example, looking at now and, for example, in two months, if anything changes or if we are making any progress. The second checklist that we have is the Scrum Health Checklist. Actually, we are in an area where we innovate, so we are trying to be as fast as possible to look, as I mentioned, at the value that we are delivering and to focus on things that actually matter. The Scrum Health Checklist has like 67 questions, and it touches different parts of the Scrum framework, for example, if everybody in the team was aware of the Sprint Goal. This is an example of a Sprint Goal and so on. So the great thing about it, and actually it concerns all checklists, and I think it’s worth mentioning right now, is that what we do with them, we kind of look at the progress of the team. Okay, there might be some comparisons between the teams just to look at the general health of the organization and how we are handling different areas like product, Scrum, and so on. But at the same time, what we aim for is to see the progress for a particular team. So we are comparing the results, usually in two months’ time. It might be more often, but that’s the maximum time we want to check what’s going on.

And it made total sense in terms of the areas that you mentioned. So the product is for the product area, Scrum is for the processes area. We also have a Technical Health Checklist. This is in terms of checking if we keep technical standards, for example, CI/CD. There are things that, if you discuss some items and some items are transparent, then they’re kind of influencing and are embedded in the culture. So this is what we’re also trying to accomplish with checklists like this. Kind of to…

Wiktor Żołnowski: …tools are also driving the culture. Exactly. Starting, following some patterns, some desired behaviors. Like, exactly.

Jakub Dobosz: That’s perfectly correct. This thing also helps us to see if we should take care of or support some of the teams. It might be that there is some technology that they are not familiar with, and we have these competences in the organization, so we can check, based on the score from the checklist, if there are some actions of support needed. And these are three checklists that support the product area and process area. When it comes to products, we have one more thing that we are checking because we see it as important: checking how Sprint Goals were delivered. Because it also can show us some information about the team and how they are progressing, if there are some changes or not. So we are checking the last six Sprint Goals and just count the percentage of how many of them were delivered. You know, so for example, if there is zero, then for sure something’s wrong.

Wiktor Żołnowski: Yes, we need to talk about it.

Jakub Dobosz: On the other hand, if there’s 100%, there’s also some growth, yeah, most probably. So these are indicators that help us, in terms of standards, to dive into teams and just to be sure that we take care of particular items that, you know, need support.

And we also mentioned that there is a third area: people. And it’s a little bit different because this is handled from the perspective of checking how they are satisfied. We have quarterly surveys when it comes to happiness, emotions, and how they perceive the organization. So this is also the standard that we are repeating, that we are checking the data, and we make our decisions. And I think also in terms of standards and people, very important are knowledge guilds, to put it that way, like for example, Product Owners, UX designers, Node.js developers for knowledge sharing.

Wiktor Żołnowski: For knowledge sharing, yeah. This is kind of a standard that provides us the possibility that everybody who is joining the company has this possibility to join the guild of his domain or another.

Jakub Dobosz: And that’s perfectly correct because what we are trying to do is to encourage people to share knowledge, and we have situations where people change their domain.

Wiktor Żołnowski: I have a few questions about it as well, but let’s get back to the product and the things that you already mentioned. There are a few other things, like for example, we have a standard for starting a new product, starting a new team. Tell us a little bit more about it.

Jakub Dobosz: Okay, so from the team perspective, what we try to do at the very beginning is to get everybody on board. We have onboarding meetings in terms of the product. Like, what we are doing usually, some of the people, at least some of the people, take part in initial discussions with our partner, so they are aware of what’s behind the idea, what we are dealing with, and so on. But also, there are people who are joining when we start cooperation. So one of the items, and we might say from the product side now, that we are onboarding people to understand the product, it’s kind of a new thing that we are doing right now, is to build the team actually at the very beginning. So bring them to the office, bring them to some area so they can get to know each other. Because in some cases, people have already worked with each other, but there are some mixes when two projects, for example, finish. Yeah, so there is a need to build a new team. So we are trying to give them a heads-up in terms of how they cooperate, to do some, I don’t know, maybe ice-breaking activities, something simple, or even just meeting in the office to have a pizza. So I can earn some…

Wiktor Żołnowski: Yeah, yeah. Like also a little too much, like those guilds, knowledge-sharing guilds, they are also an opportunity to meet other people, especially in the same technology, where there is a high probability that one day you will be working together on the same project.

Jakub Dobosz: Yeah, yeah, exactly. And when it comes to this starting, we also have this checklist that also helps us to start the product because, for example, the technical checklist tells us what you need to pay attention to at the very beginning. And the same when it comes to the product checklist, yeah. So which areas are good to handle at the beginning of the product development and the cooperation, and that might be helpful for a new initiative.

The last thing that I want to mention that we are doing in Pragmatic is standardizing how we communicate. Because we feel it’s very important, what we aim for is to have as many face-to-face meetings as possible, especially at the very beginning, and also with our partners. But this is extremely important if it’s possible, of course, because we understand that we have our partners from different areas of the world, and that might not be doable in some cases. But if it’s possible, we strongly advise and strongly push forward to meet, to establish communication–how we will communicate, what items will be handled, in what way, how we will meet each other, and so on. Because we think that the beginning of the product development for each successful product, how we start, also plays a vital role in this. So we also intentionally and openly, transparently, say about how we’ll communicate, who will be the point of contact, for example. Some of our clients are used to having one person that is responsible for status reports or something like saying what the team is doing. That’s not how we want to work in Pragmatic. We want everybody to be able to voice their ideas or concerns and so on. So at the very beginning, we mention to our partners, “Okay, this is the person who will be enrolled: a Product Owner, UX Designer, you know, back-end developer, front-end developer, or full-stack developer, and so on.” And we can contact each other because I see that in many situations, there is this misunderstanding that there will be one person. Of course, there are some items that are discussed in particular domains like product ownership and so on, but in general, we want to make sure that everybody has access to everybody.

The product discovery workshop

Wiktor Żołnowski: Okay, so let’s make one step back. Before we start working on the software product, in most cases, we have our product discovery workshop that we run together with our clients, with our partners, or all of the stakeholders, if possible. Could you tell a little bit more about the standards around product discovery workshops?

Jakub Dobosz: It’s great that you’re asking about it because it’s evolving, and I’m very happy to say about it, as we try different approaches and we try if they work and then focus on the ones that are particularly good in product discovery and especially initial workshops. So, what we do for sure is focus on the problem that we want to solve. So we direct our discussion in terms of really clearly defining the problem that you want to solve. The second thing that we are doing is trying to think about the offer. Because there is a problem, so what offer can we provide to our potential customers of the product? Yeah, so it might be kind of connected to a unique value proposition, so what the product will offer, what will be the differentiator of the product. But the main idea behind this is like, what can we offer to potential users?

And then what we focus on is to quickly bounce that idea, the problem and offer, from actual responders, like from actual customers, virtual users. Because in many cases, and we live in this kind of assumptions bubble, I understand that, for example, the people who have an idea, they are convinced that it is great because it’s a driving force of product development, and it’s perfect. At the same time, we are this kind of balance to put it in the right track, to validate it at the very beginning, to check if the problem exists and the offer that we have in our minds will be accepted by potential customers.

Having those two items, we do some initial research, and it differentiates between different products because in a product that is pretty simple, we might jump right into building some prototype and doing some research. In products which are more complex, it might be a good idea to build a landing page to get somebody on the waiting list and so on. So the next items, I would treat it as a decision tree, like what we will do next. For example, build the landing page, go quickly to validate if we’re on the right track. If we validate it, then we can build a prototype, then we can do some additional research if this way of solving the problem, way of providing this offer, is accepted by potential customers.

What is also a misunderstanding in terms of thinking only about discovery workshops or product discovery is that some people think about it as a phase, which is not true. Product discovery is a process. It’s a continuous build. Exactly. What we are trying to show is to validate this problem that exists, this offer that can solve the problem for potential customers, then take next actions, at the same time still continuing product discovery. So we are learning, maybe not on a daily basis, but on, let’s say, a Sprint basis, let’s put it that way in Scrum language. And what is very important is the fact that we might always change our direction or where we are heading, even in some cases 180 degrees. It depends on the product. What we are trying to do at the very beginning is to define the problem, define the offer, and show our partners that this product discovery will continue along our way and we have to be open to changes and to the feedback of people, of customers, potential customers that will eventually use the product.

Wiktor Żołnowski: I would like to add to this that in the past, when we started a new project for clients, usually the client came with some set of requirements, a set of functionalities, or even some mock-ups, some designs to be implemented. What we found out was that usually, even after the first two or three weeks of software development, we were working on something totally different than we had been talking about during this initial conversation with the client. Like, we were working on things that were not designed, were not analyzed before, because we discovered something during the process. So, the product discovery workshop was an answer for this issue, that sometimes we were wasting some time at the very beginning to actually learn from this product discovery things that we should have known at the very beginning.

The second thing that I’d like to mention is that it’s my own example. Some time ago, I started my own startup, and of course, I used a Pragmatic Coders team for that. Because we were in a rush, we had this brilliant idea, I wanted to go to production as fast as possible, we decided we would skip this product discovery workshop. And after four weeks or five weeks, I realized that we wasted most of our time, speaking time, because every time the team delivered a new version, I was like, “No, no, it should be changed, it should work that way.” “No, it works another way.” Then we rework it, okay, perfect. And then we show it to the customers, to the potential users of the app, and they were like, “No, it’s worth… I don’t understand it. It should work in a totally different way.” After four or five weeks, I don’t remember, we decided, okay, let’s stop the development, let’s do the product discovery workshop right now, and let’s try to figure out what our users actually need. Since then, we avoided a lot of problems, a lot of reworking. The work became more smooth in terms of the delivery, in terms of the problems that we faced. But also, what was very important then was that the team started believing in the product idea because they started understanding it, but also they saw that the market’s response was positive. So people we interviewed actually had this problem, had this need that we wanted to solve through the app. That was also very motivating for the team and also improved the work, actually.

Jakub Dobosz: We mentioned two important items. When the team delivered the product, you were providing them with feedback: “Not this way, that way.” Correct me if I’m wrong, but I think this is also due to the fact that you aimed from the very beginning to be the user of the application. So you were actually in the potential group that provides valuable feedback, and at the same time, you’re the only person, so it was kind of biased. It was kind of your point of view. So this is why opening up for other potential users broadened this picture. I had a few other examples of why doing proper UX, user experience research, doing interviews with some cohort of users, makes sense. I mentioned a medical app. It’s for patients and doctors, and we have a doctor in the team. And this doctor had objections on a few aspects of the application. He said that because of the regulations or law restrictions or other things, like the security of the patient data, because of that, no doctors would like to use this app. They will never use this app because of this. We disagreed with him because we ran UX interviews with a number of doctors. At some point, even he himself did this kind of interview with his colleagues, and none of them mentioned that this is the issue. He said, “Okay, it’s only me.” So, he’s like a part of the team, and so he may be considered as our advisor and partner, but he said, “Okay, I’m not the alpha and omega. I don’t know everything, and I’m not the regular user. I’m just one of the users who might have some issues with the app.” But at the end of the day, the entire cohort of users, in this case, doctors, they do not see this problem that he saw. Doing proper UX research all the time is a part of the way we build products, we build software.

And the second thing that I heard from what you said is that, I think it’s a buzzword, but empowerment. You mentioned that the team better believed in the app and better understood what was there and what needs to be solved, actually, what they should do and how they can proceed with that. And this is the same about the product discovery that we run. As you probably noticed, I didn’t mention any features or some functionalities of the product. Focusing on the problem, focusing on the offer gives the team incredible space to innovate, to think how it can be solved, like what can be done there, how to handle it, how to check a given item, a given offer. Having the teams…

Wiktor Żołnowski: …that are business-oriented, focused on business, focused on value, they can use their full potential. You can, I don’t know, use some great tool to dig a small hole, that’s for sure, and it will work, yeah, but what’s the point? It’s better to use the tool to build something… it’s like they’re great.

Jakub Dobosz: Yeah, these two items, I think, were very important in terms of product discovery and product development at the very first stage. So to be focused on feedback from the potential users and to really hear it, and the other thing is to leave space for experimentation, documentation, and for searching for better solutions for problems. Yeah, like always searching for a private solution.

Wiktor Żołnowski: One other thing that I just recalled is that the team that I was working with, at some point, they decided to go to production faster than we planned it because they figured out a better way to do stuff, like things that they don’t need to do before going to production to actually go to production. I was like, I’m the guy who is always “launch it, go to production as fast as possible,” etc. And I was actually holding them back, and they decided on their own that it’s better to go to production. Of course, they informed me; it’s not that they went to production and didn’t tell anyone, but they informed me, and they convinced me pretty easily that we don’t need it at the very beginning. We found a better way to satisfy the client’s need and go to production much earlier than we thought. So, this is actually the result of the product discovery process that we have in place, the UX designers who are also interviewing users, the team that is aware of the real user needs, and they can be empowered to actually make this kind of decision or actually make this kind of ideas to build a better product, to be building it in a better way. This is something really, really amazing, even for me, when I felt it on my own skin when I tried to build the product with them.

If you’re watching this, we are talking here about product management, we are talking here about Scrum. You might also like to watch the two other episodes, or actually three other episodes, that we recorded previously. One is about product management, and there with Michał Kania, we discuss how our product managers, product owners, are working. There are way more details and tools about product management that we are using and standards that we are using as well. There is another episode about Scrum with whom we discuss Scrum for startups, but actually, it’s a universal Scrum description, the way the process works, and also the way we are using Scrum. And we are very focused on using Scrum in the right way and using it as leverage to improve our performance, our effectiveness, but also to improve our product management and make it more efficient. We are also focused not only on Agile but also on the Lean part of product development. There is a third episode especially about product discovery and this continuous learning about the product with Kate Smolarek and Jaszka, our UX designers, with whom we talk about the UX process at Pragmatic, but also about UX, user experience in general.

Cultivating people and processes

Let’s get back to our discussion about the processes and tools. We have those standards for this checklist for products, for product development, product management. You mentioned the technical checklist, you mentioned also the Scrum Health checklist. We also have a few other smaller checklists, like for example, team onboarding, etc. As I mentioned before, the creation of these checklists helped us a lot, and thanks to them, we avoid a lot of problems that usually are not visible at the beginning, but their results are visible later on, and the consequences are usually pretty, or could be pretty, painful. So maybe let’s talk about the things that you already mentioned. Let’s talk about people. I think it’s on the verge of processes and people.

Jakub Dobosz: I would say that we have standards like, for example, for knowledge sharing, for getting involved in some parts of the product development itself, or in a given team, or in general, in the whole organization. What is great about the composition of these two items is that we have some frames, like how we work, but we do not define inside of these frames. Like, teams and guilds, these knowledge guilds, have their own position, their own ways to, for example, exchange knowledge, to motivate each other, to check each other. Because what we are aiming for is to be sure that we as a team, as a company, we are doing good. We care about each other. Yeah, like for example, we have a Guild of Product Owners, so they meet on a weekly basis to talk about their products in terms of being a little bit grilled. It’s called Product Drills.

Wiktor Żołnowski: Yeah, it’s called Product Drills, not officially.

Jakub Dobosz: That’s correct, but the intention is very good there. So we are trying to get better and better. And in terms of people and this composition of process, this frame and people gives the possibility to experiment in teams. And I mean teams in terms of these domain teams, like for example, Node.js developers and, for example, Java developers, and so on. And also in terms of product teams, like for example, we are building a given product so they can also somehow experiment. And an example is as such: we had a product that more people were involved in, where there were more than 10 people in the team. And then the team saw that there is a place to try LeSS (Large-Scale Scrum). So they just tried it, if it works, how it can be implemented, what can be done. And then we have this kind of also a standard, it’s called Monday Talks, that we can share knowledge between teams. So somebody raised their hand and said, “Okay, I want to share our experience with LeSS and the company.”

Wiktor Żołnowski: Why it’s a meeting where people can present on their lessons learned, even some things that are not related to work. Exactly.

Jakub Dobosz: On the other hand, we have the team that also was big, and they decided to split their, let’s say, streams into smaller ones, and they also tried different processes inside. And it was cool also to see how they’re organized, focusing at the end of the day on the value that they deliver and the cooperation and so on. So that gives great space, these frames. And what’s also extremely interesting from one point of view is how the teams might inspire each other.

Some time ago, I by myself tried this kind of “Product Owner Partners” initiative, linked to combine pairs of product owners that will meet, for example, on a daily basis and talk about challenges, some things that they want to discuss. So there will be continuity in a given pair. That was an idea. There will be support for each other on a regular basis. And for various, various reasons, this experiment didn’t work. We may go deeper into that, but that’s not the case. What I’m trying to say here, after like four months, I had a discussion with Kate. She’s, as you mentioned, leads the UX team in Pragmatic. And she mentioned that they started using this, and this is basically the same idea: two people meeting and just exchanging, supporting, and so on. And in their case, it works. So you know, I might have said to myself, “Oh, that was a bad experiment, and we wasted some time,” and so on. And looking just locally in this area, it might have been right. But taking care of a global perspective and seeing different teams, I see the results of it. So that we can inspire each other, and that’s perfectly fine that in some place, this practice will not work, but our team, our group, can benefit from that, just to somehow take the idea and tweak it for their needs. So this is a great example of this space for experiments and space to be able to choose own ways to handle, for example, knowledge sharing, support, and yeah, at the end of the day, delivering value.

Knowledge sharing and experimentation

Wiktor Żołnowski: Yeah, I also remember the story of those checklists that we mentioned before. Like, the very first checklist was a Scrum Health Checklist, which we developed during our internal Scrum training that we are running. Also as a part of, let’s say, every of our employees has to go through this extended Scrum training, which is like a three-month duration, one time a week for one hour. We work with people, do some workshops about Scrum and particular parts of Scrum, and teach them how to use Scrum. Part of the first training that we had was our creation of the Scrum Health Checklist. And when people started using it, like every two months, filling it with their own team’s results, etc., they noticed that it actually helped them. So they started another checklist, and this is how the Product Health Checklist appeared and was created. And then when people saw that, okay, it was for product, it was for Scrum, let’s do the technical checklist to also avoid some mistakes and some problems and not to actually even monitor the way how good we are in this area or what we will improve as well.

Jakub Dobosz: I just connected the dots and your previous question about product, like when we start the product, and actually there is a big checklist in there, absolutely. Yeah, yeah, it’s based on the experience from the various implementations. And yeah, it was actually inspired by the other checklists, and so we put there information from our point of view that is important to smoothly run it, to be sure that we took care of everything at the very beginning because usually the beginnings are a little bit messy, so some items may slip. So this kind of checklist is a great help.

Wiktor Żołnowski: I also remember that the same happened with the Lean Coffee. Like the first one was the Scrum Lean Coffee that was a part of this training process that we have. But then it appeared that there is a separate Product Lean Coffee, then another guilds like a Node.js guild started their own meetings in the same format. When people started learning one tool for facilitating knowledge exchange, etc., they are starting to be used in a different context, and that’s great. Also, it’s very good for those people because they don’t need to learn a new method, a new facilitation or meeting format every time because each of them looks pretty the same, and they can move between these meetings smoothly.

Jakub Dobosz: You mentioned a very important thing, and it was one of the reasons to introduce Scrum in the whole organization. The standardization helps to move inside the organization itself. So as you mentioned, even Lean Coffee, for example, I am a Node.js developer, and I’m attending my Lean Coffee, and I’m thinking, “Okay, I’m interested in blockchain. I would like to check what’s going on there.” So when I’m joining the blockchain Lean Coffee meeting, I already know how it is run. So I don’t have to ask about, “Okay, how you discuss it?” and so on. So that’s great.

And the other thing also connected with experimentation is we have this Scrum Lean Coffee, and then the idea pops out to bring product. It’s completely separate, and we started these two initiatives, and then suddenly somebody got an idea, “Like, we are talking about Scrum, product things, so let’s combine these two Lean Coffees. It will be great.” Actually, at the end of the day, this experiment was not very good because product owners took over the meeting, and we saw that, okay, it would be better to separate it. And actually, it was, I think it was Michał that just asked, “Okay, maybe let’s do Product Lean Coffee,” and it was started by him again. So we came back to this practice, and it was like, I treat it as a product lying on the shelf. It’s available, so you know the standard, you just can grab it, use it, check if it works. Because we had a kind of a cycle. For a time being, it worked, and then it didn’t work again. So it’s good to have the standards for people also to be aware of what can be used and how they can configure it, how they can play with it.

Wiktor Żołnowski: For those of you who don’t know, the Lean Coffee format is also a standard that you can Google and read about. But it’s a very simple tool where you have a limited time, like five minutes, to write down all the questions that you would like to ask or the topics that you would like to discuss with people who are on the meeting. Then we are doing a quick dot voting to decide what is the order of the topics that we would like to discuss. And then we have like seven minutes to discuss the first topic. If we finish the discussion before, it’s perfect, we can end it. But after those seven minutes, if there is still a discussion, we do a quick thumbs-up, thumbs-down voting if we want to continue for another three or, depending on the format, five minutes. And then every time after the timebox is finished, we decide again. So it could happen that for the entire hour, you will be discussing only one topic, or it could be like plenty of topics that you will cover during these small time periods, these time slots. So it’s a very effective way to discuss some topics, some issues, some problems that you may face. I honestly recommend it.

Jakub Dobosz: Actually, I don’t know if you knew, there is a third option right now what we are testing, and some meetings are using. As you mentioned, thumbs up is continuing, thumbs down is we will not continue, and thumbs to the side is, “Let’s stop, but we will make an action point.” Okay, so this is kind of a differential between, yeah, and between work, between, “Like, we don’t want to talk about it,” and that’s fine, and “We covered it,” and “We feel that we don’t want to talk about it, but there is space to do something about it in the near future.” Great.

Wiktor Żołnowski: Wow, that’s cool. I didn’t know that. I need to attend these meetings more often.

Onboarding and continuous learning

So if we are talking about knowledge sharing, knowledge exchange, it’s very important to also mention our onboarding process when a new person joins Pragmatic. Could you tell us a little bit more about how this process looks like?

Jakub Dobosz: Sure. So we want to make sure that everybody has basic knowledge at the very beginning. We make sure that we are on the same page. Naturally, it’s nearly impossible to get all knowledge, like this living knowledge that is in the company, in this small period. So there are some actions planned later on, so I will talk about them. But at the very beginning, we aim to have short meetings that will just be like an injection. We want to focus on the very important items. So of course, there are some admin registration meetings, so it means like how to fill up the time off, yeah, exactly, this kind of stuff. There are some meetings in terms of different departments, so how we are working. So it’s delivery, it’s sales, marketing. And also what we are aiming for in the very beginning is the meeting about our values and also how we work. So we have this, I may say, short six-hour training. So we want to provide basic knowledge to everybody.

Actually, everybody, and I’m pretty proud of it, that I truly feel that we practice what we preach, and our departments, marketing, HR, also work in a Scrum-like way and work in Scrum. Two or three months ago, I was having this onboarding meeting from Scrum with everybody from marketing because that was the time that people were joining our company, and it was amazing. It’s another point of view from people that weren’t included in this way of working before. This is the very initial part of onboarding.

Of course, besides that, especially if somebody is joining a particular team, they get onboarding from the product owner, the team, about what’s going on there, they meet our partner, and so on. So this is like a regular thing depending on the team that they are joining. The last thing is everybody is told about initiatives that we have in Pragmatic that are, let’s say, cross-functional. So these guilds, Monday Talks, some meetings. They are invited for our monthly company updates, our demo days where we show our products. They are aware of what’s going on in the company because we aim to build this relationship, which we think is very important from the very beginning. And later on, when we move towards, like after these two months, we aim to have it till one year’s time from the starting point of a particular person. We invite them, for example, for an advanced Scrum training, which takes around 13-14 weeks, where every week we meet for one hour and we have long discussions about a particular topic in Scrum. So thanks to that, everybody can understand better, and also, what is more important, they can start sharing their experiences and their feedback from the teams that they are working with right now. Because it’s also very important to get feedback from people that just joined our organization because they can have a fresh look on the ways we work.

What we also consider very important is to have some trainings around communication, around technological stuff, around, for example, domain knowledge like product ownership. We have this training of seven meetings, for one hour and a half, that we cover each step of the product management process. Participants get homework, so it’s a benefit for them to develop and to discuss the outputs and outcomes of what they achieved from the work. Along the way, we, of course, invite people that joined us to be on guilds, to be present in the company space, in company events, meetings, and so on, because it is the best way to onboard, like the fastest way to onboard, to be aware of what’s going on and how we operate. Some of the trainings are mandatory, some of them are optional.

Wiktor Żołnowski: We’ve also created an app, a very simple no-code solution, to actually monitor who is attending which training, who has already completed all of the mandatory trainings and finished the onboarding process, and who is signing up for the optional ones. I believe that is very important, and this is the best way to actually bring people up to speed when they are joining a company. And also, it’s a great tool to work on the company culture. Of course, aside of that, it’s very important to hire the right people who are a good culture fit, or fit the culture that you want to build with them. Especially if you want to change your company culture, you need to start hiring people who fit your new company culture that you want to build. So it’s also very important. In terms of hiring, we also have standards for hiring as well, but I believe that hiring developers or people for software companies is a topic for a different discussion and a different meeting. And I believe that will be one of our very next episodes.

Leadership and evaluation

Let’s talk a little bit about how we evaluate people and how this process looks like, how the leadership in the company looks like.

Jakub Dobosz: Sure. We focus on the everyday development, let’s put it that way. If they follow the Scrum framework, which is kind of mandatory, they are encouraged to do retrospectives, they are encouraged to learn, to develop, and to look for new ways of working, like in terms of process, in terms of competences, and so on. So this is one thing. The other thing that helps us evaluate are the particular meetings. For example, as we mentioned, we have this Product Review where product owners are presenting their products, and we also have action points from this meeting, like product owners have action points from this meeting. So we also check how the action points were done or what happened with them. So it’s kind of also an evaluation in a way that it motivates you to go further, I will try new things and go for the next level.

In terms of individuals, maybe I will focus on that first, because I started with product owners and so on. For individuals, we have a role of Team Leader that is working with up to five people, and this Team Leader is responsible for the development of these people that he or she is working with. This is the frame, and that’s beautiful, this is the frame. And how they organize this evaluation and development, it’s their idea. They can meet, for example, on a weekly basis, they can meet on a monthly basis. At the end of the day, we have definitely at least one checkpoint, like a yearly check. We aim to have half-year checks, but this is also the frame. And during this yearly check, it’s up to the team leader and the person themselves to decide how they will run it. For sure, they need to gather feedback, so we have these surveys or we can talk face-to-face with the person that you want to get feedback from, and that corresponds to our values. So we are checking how the person is fulfilling the values or how they feel with the values, and what are actual actions, actual things that the person did that show the full use of the value or the lack of the value. Because we aim to work on actual examples and actual things that can be fixed or changed or can be cherished, let’s put it that way. So if somebody truly, that person truly takes ownership, we want to, I know, have a high-five and ask how that person can expand the approach that they are presenting to others, just as an example.

There is this discussion with the team leader on what to focus on next in terms of further development. It can concern the values themselves, like particular behaviors, particular things that were done or were not. Also, it can consider the various areas or places that we work. And what I mean is, for example, I am a Node.js developer. There might be some items from the technical point of view that I might focus on during the next month. Also, if I am a Node.js developer, there might be room for me, and it might be interesting for me to focus on Scrum. I can jump into the role of Scrum Master in my team, yeah, and that’s interesting for me. Also, there are examples of people transitioning from developer to product owner. So it depends on the person. We try to adjust, especially when it comes to senior developers and senior roles, what areas, what things of interest are for this particular person, and try to find the common ground, like how we can help each other. This is on the individual level.

What we also evaluate on the team level are the checklists. We are checking how the teams are doing, and also we have a dashboard that shows all the teams. But what is the most important thing is the number that is next to the percentage of checked items from the list that shows the change. So if it’s with a plus sign, it means that the team developed. If it’s minus, that might be a thing to check, like why. It might be reasonable in some cases, but it’s definitely worth checking. For the first example, if there is a rise in the percentage, it might be good to talk with the team, to do something, to share how they managed to, for example, get from, I don’t know, 78% to 86%, yeah. Because that’s kind of cool to share, especially as some teams might be on a different level and might use it.

Wiktor Żołnowski: I may say that both of these perspectives are very important because from the organization’s perspective, we look at both. I believe that teams might actually produce more value than individuals. So looking and… I don’t know that, yeah, I don’t believe it. I’ve said it so many times that I know that. Exactly. So looking at how they develop, how they are evaluated is a great indicator for how much value we’ll bring. At the same time, looking at the individual level helps us develop people, which of course influences the team level, and also gives, in some cases, motivation for personal development, being seen, and being heard. Also, combining these two perspectives is a great recipe for success. Perfect.

Ensuring quality without dedicated testers

Okay, so let’s talk a little bit about the processes, especially the quality assurance processes. The thing that is distinguishing our company from others is that we do not have, we do not hire, software testers. It doesn’t mean that we are not testing the software, but having someone who will be looking for bugs and some issues in the software instead of assuring that those issues will never appear is actually a waste. It’s obviously a waste. So this is why we don’t have testers, because there are other, better ways to assure the quality of the software than just testing and controlling the quality and then doing the rework of fixing the bugs, fixing issues, and redoing it over and over again. Could you tell us a little bit more about the processes, the processes and tools and methods that help us assure the quality of the software?

Jakub Dobosz: Sure. Actually, one of our values is ownership. So I cannot imagine passing the responsibility, passing the ownership of the part of the software that I developed to somebody else. Because actually, testing it is just like, “Okay, I did it, and somebody else will check it.” And so it might result in that. It’s not certain. What we are aiming for is to actually solve the problem before it happens. So we definitely take time to automate the tests, like the tests that can be automated. For sure, we are implementing continuous integration, continuous delivery in our products because we believe that it will help us to grow the product without fear. Actually, the manual tests might also result in a lot of fear of changing some items. If you have automated tests, which I know takes like, for example, five, ten minutes to run for the whole solution, it’s simple. You do some change, just run the tests, you see that everything’s working, you are fine, you are secure. There might be some items that are besides, like this, for example, unit testing or functional testing. And we also check it.

Wiktor Żołnowski: We focus on, let’s say, exploratory testing, and we do it inside the team, the product team. For example, other developers are testing, we are cross-testing. We pay attention that somebody who developed it will not test it. It’s not that we don’t have testers means that we are not testing. We have these testing competences inside of the teams, but we don’t need a separate role for that, especially that those roles are costly and, as I mentioned before, it’s very hard to plan the way in an agile environment, in Scrum, in a way that that person will be efficiently providing value to the product development. Of course, I’m not saying that testers are not providing value to the team, that’s not true, but there are better, much easier, and cheaper ways to assure the quality of the software than just testing it.

Jakub Dobosz: Exactly. And this kind of preparation for longer development, because if we start from the very beginning to automate testing, testing the most-used cases, I won’t say all because there might be some items that need manual testing in some cases, but we focus on automating it. It helps us and gives us the possibility to develop the product in later phases without the fear that I mentioned. What was also cool in one of our teams that they worked with, it’s not about testing, it’s about code review. What they did, they decided to have a code review in pairs. When somebody finished the part of the code they wanted to be reviewed, they talked to each other face-to-face or through some remote tool. And the person that wrote the code went through it, and the code reviewer was looking at the code and was listening to that person. And what was extremely interesting is that the author of the code by themselves, they fixed it. It’s like a, you know, a yellow duck, rubber duck in this scenario.

When the code review is done and the code reviewer understands how it works from the technical point of view, she might be a good person to test it. That person is already inside this business case, use case that the code was representing. It also might influence the speed of how we are checking our product. I’m talking about this exploratory testing, like if we need to break it, we try to do some, I don’t know, weird stuff just to try to break the product, stress it, exactly. So thanks to that, people learn and also they look for efficient ways to ensure the quality because code review is also part of it.

Wiktor Żołnowski: It’s also very important that aside of that, there are plenty of other things that we implemented that support quality assurance. Like for example, the fact that we are working in Scrum, that there is a Definition of Done in Scrum, so another checklist. And the fact that we have great product managers who are also using some standards on how to describe the requirements or how to write user stories, etc. That there should be some acceptance criteria. And again, using Scrum assures that we have an extensive backlog refinement process where people who will be doing the work are also talking and discussing and researching the requirements. It’s also a way to avoid a lot of errors, a lot of bugs that usually happen from not understanding the requirements. It’s hard to not understand a requirement that I was writing down, as I was discussing before. So if it won’t be done in the next five months but in the next Sprint, I still remember what we discussed and what were the edge cases and other stuff, and especially if we make notes, most of the problems are usually avoided. Also, trunk-based development, continuous delivery that you mentioned, single branch flow development, those are methods that allow us to release very, very often to production. And it’s also a method that allows us to revert the changes if we find something that is wrong, that there is some issue, and that the impact of this issue is high. So even if we made a mistake, we can easily revert the situation. It’s all about the fear of making changes. If we can easily revert, in most cases, there is no fear. We can release anything to production. Of course, it may be annoying to some users if they see that something is there, doesn’t work, and then disappears, but it doesn’t happen very often. I think the best proof to confirm that our processes work is our clients’ opinion, that actually no one is complaining about the quality and there’s not so many discussions with our clients about the quality issues or something like this. So if they are there, they are usually easily fixed by some adjustment in the process, not by adding testers to the team.

Jakub Dbosz: Exactly. And also, I think it’s cool to share a practice. When we had a little bit bigger team, like actually it was around eight developers in the team, we figured out that during refinements, there is no need for everybody to be in the refinement meeting. We agreed that three to four people is enough to have an effective discussion and to figure out what needs to be done in this particular user story. We refined this user story, having the team engaged, like three to four people. And then when we were done with the refinement, when we wrote down all acceptance criteria, when we defined all subtasks for a given user story, we invited the others to just read the acceptance criteria, maybe hear two or three sentences about what it was about, and then we started estimation. And it really helped us to double-check if we truly understood the user story. Because in most cases, the estimates were close to each other, we did a great job. But there were some cases that, for example, the estimate from the team estimating was like five using planning poker, and somebody from the invited group gave 13. He’s like, “Why?” And then we had an additional discussion. And this is actually like planning poker, it’s another tool to increase the quality. I would say that this is the main goal of using estimation: to have the same knowledge about what has to be done in the particular product backlog item. And the side effect of this process are the numbers, are the estimates, not the opposite way around. If you are doing estimation and you are focused only on the numbers, not on this knowledge sharing, then most probably you are not doing it correctly.

What is very important also from our perspective is to get our partner on board in terms of our process, how we check quality, and how we ensure it. And I remember products that we were developing that our partners just, you know, went through it very quickly just to have a track to see if from the business perspective it’s working. But they definitely didn’t do any, let’s say, checking of all acceptance criteria because they were trusting our process, and actually, they were right. Everything was okay.

Wiktor Żołnowski: When we tested it a lot more, it also helped us to understand what is the way for our partner in terms of the product that we are developing in a bigger picture. Because some of the items might go to production instantly, like when we develop it. Some of them might be connected with actions that our partner is doing on their side, so the manual marketing, onboarding new clients, and so on and so forth. So understanding the process, how it flows, also from the quality perspective and the time of deployment, let’s put it in that way, it’s very important from my perspective. Also, like the deployment doesn’t equal release. Blue-green deployment, feature flags, etc., those are the practices that we use. That’s another way of controlling quality control. Feature flags, you mentioned reversing changes, yeah, switching off a feature flag is even simpler. It’s really simple, and we can provide the admin for a client who can do it whenever they want. Yeah, so those are like quite a few: innovation, test automation, where people use test-driven development for their work, and others by writing tests after. I am a TDD guy, so I always believe that the best way, the only way you should do this is TDD, but I won’t be discussing it with all of our developers. I’m just interested in that they are writing the tests. That’s the basic, that’s also a basic requirement when we’re hiring people, hiring developers. So if you want to apply to us and you don’t write tests, please start. Don’t waste your time.

Jakub Dobosz: This is about the standards for recruitment. And I remember one recruitment interview that I was running, it was about a culture fit. We also pay close attention to that. And I had one person that I figured out from what that gentleman was saying that he is not willing to test. And I asked him, “Okay, what if I said to you that we do not have testers?” And he looked at me and said, “I wouldn’t like it.” I said, “That’s fine, but we are not a fit. We will not work.” So yeah, that’s perfectly fine. It’s like everybody can have their own ways of working.

Wiktor Żołnowski: Yeah, exactly. That’s, I think that this is something again for another episode about hiring. But yes, the cultural fit, these kinds of topics during the recruitment, the way we work, etc., and asking or checking if the person, the candidate, fits to this kind of process is very, very important.

Conclusion

Okay, so all of that, like all of these processes and tools, actually made us way more efficient and effective. The best way to showcase how efficient we are is by saying what is the average size of our team, which is around three or four developers. And with such a small team, we are usually able to deliver pretty fast, usually in less than three months, the first version of the product to production. It’s also based on the product management. Again, I invite you to watch the episode about the product manager and how we do that. What is also the result of the way we teach people how they should work, the processes that we have… and the best examples are, for example, a banking app that was delivered by the team of up to eight, nine people in a year and a half or something like this. And they were ready to deliver even before that time period, but because of the regulations, because of some licensing and other stuff, it took longer. Comparing that, eight people, one and a half years, releasing a bank, comparing that to other companies that are building banking apps or banks that are working in a different way, where they need like a hundred or 200 people to develop an app and to ship something to production, I think that shows the difference between our approach and the approach of other companies. So is there anything else that we should discuss today?

Jakub Dobosz: I think adding to what you have said, and I think it will be also a great summary of what we just said, is two things. It’s understanding the needs, I’m saying about everybody in the team, everybody understanding the need, and focusing on the value. Because, you know, building the bank in a year and a half or even earlier, it’s about what has to be delivered for the bank to be operating. Of course, there are also items to be added if the product is running. It might be another story for some time, but when we focus on value, when we think about value and we listen to users of the product, to our partners that also have valid insights that bring us to realize their goal and bring what actually people need, which is our goal.

Wiktor Żołnowski: So, thank you very much, Jakub. I hope that you enjoyed the discussion. I hope that you found something useful for you. Maybe you can copy some of our tools and methods. Just be careful and first of all, always try to assess your needs and your organization and what are your challenges and problems, and try to choose the solutions that fit your challenges and problems. But if you have any questions about how we work or if you have any questions regarding the tools or methods that we mentioned here, do not hesitate to contact us, me personally, or contact Pragmatic Coders. Of course, if you also would like to experience this kind of working and join us, you can always apply. We are hiring mostly people from Poland, but of course, we will consider any candidate. And also, if you would like to experience it from the other way and start working with us as our client, or as Jakub used this word many times, as our partner, because this is how we try, or not try, how we actually treat our clients, also contact us, and we will always be happy to work on your product together with you.

Don’t forget to subscribe to our YouTube channel or to follow us on Spotify and Apple Podcasts, and please give us a thumbs up to make this episode more available for a broader audience. Thank you very much.

Pragmatic Talks is delivered to you by Pragmatic Coders, the first-choice software development partner for startup founders. Be sure to catch all new episodes. Subscribe to our YouTube, Spotify, or Apple Podcast channels. And if you are thinking about building your own startup or struggling with product development, contact us and find out what we can do together.

[/accordion-item][/pc-accordion]