This article is a how-to for people who design, build, and run software-based systems. You’ll learn how to establish a simple tool—design principles—in a way that promotes innovation within guardrails and helps you avoid a version 2.0 failure.
Here Comes 2.0!
One of the benefits of consulting is that if you do it long enough, you see a lot of systems. Sometimes you come in on the ground floor to get a new system launched. Other times you are cleaning up a mess that got off the rails early on, but nobody knew (or cared) to fix things.
In my career, I’ve seen a clear pattern of what I’d call “2.0 Systems” — the name for those software systems whose primary goal is to fix everything that was wrong with the original 1.0 system. Organizations often pin their hopes and dreams on Version 2.0 because 1.0 is so loaded with technical debt that it—and the people working on it—have no option left but to re-write it.
The Version 2.0 storyline is familiar
First, the organization assembles their dream team, who promptly give themselves a cool sounding name and Slack channel (#ThunderCatz). The team starts full of optimism as all the problems of the old system will be magically fixed with Version 2.0. There’s a long list of technologies this dream team has been reading about and now is the time to use them.
Second, they draw impressive diagrams full of colored boxes with arrows all pointed in the right direction. Both solid and dotted lines are used. Buzz words are tossed about the dream team. Kubernetes is a given. And why not Service Meshes? Serverless?
Finally, executive leadership is impressed, and Version 2.0 development is green lit. Everyone is happy, the future is bright.
And then it happens. Architecture meets engineering. Things start to go south.
Sure, the designs look beautiful, but when the development team begins, things don’t work out quite like everyone envisioned. It can start with small things: a new requirement here, a new integration platform there. But, over time, the same thing that did in 1.0 will do in 2.0 too.
Breaking the Cycle
Today, if you’re building Version 2.0 (or 3.0+), there are thousands of things to think through. And there’s no shortage of opinions on the best ways to design, build, and run systems. Design thinking? Check. Agile and TDD? Yep. Do we need SAFe? Not sure. How about CI/CD? Of course. Did I mention the cloud-native architecture that will leverage our data lake for doing machine learning? Sweeeet. On Kubernetes. Hell yeah! And then we sprinkle that magic-pixie-dust we all love, DevOps, and it’ll be nirvana.
Sound familiar? How many Version 2.0s have you seen in your career?
With anticipation and pressure building, delivering Version 2.0 is quite daunting even for an experienced technical leader. So, where do you begin?
I begin with establishing a set of design principles.
But first, I’d like to share a personal story. In my career as a software architect, I’ve had quite a few lessons seared into my consciousness. I learned one of the most important ones in 2004, while early in development of a Version 2.0 product in the banking industry. We were in Sprint 6 and integrating an early version of Hibernate as our ORM solution. It had started well, but we were hitting snags that were jeopardizing all the stories for the Sprint. Without a working persistence tier, you can’t save and retrieve data. And without the ability to save and retrieve data…well…you don’t have much of a product!
A veteran engineer had recently joined the architecture team and was leading up the persistence tier. He came to my desk the day before turnover and said we were hitting Hibernate issues. The demo with the CEO was the next day and I was feeling the pressure. Instead of collaborating with him so we both could solve it, I said, “I’ll fix this,” and proceeded to spend the next 12 hours trying to wrangle Hibernate. After all, I thought, it was my decision to use Hibernate and if someone needed to fix it, it needed to be me.
Yeah. I didn’t get the code fixed. And our demo was a disaster. Which created more pressure. It was not my finest day and on top of that, the engineer had now lost some respect for me as a technical leader. He later said, “if we’d have worked together, I think we could have solved it, but you took over and I couldn’t help.” I learned an important lesson that day that has stayed with me ever since:
As a technical leader, it’s not about what you can do, it’s about what you can get a team to do.
This was a hard lesson for me to learn, but immensely important for the growth of my career.
Shortly after my meltdown, I discovered the importance of principles to guide the development of systems. The word principle has many different definitions, but I’m referring to: “an adopted rule or method for application in action.” Over time I learned that principles are what allows an architect to retain influence on the integrity of the whole system without having to be there at every design decision. I’ve been inspired by folks like Bob Martin, Martin Fowler, Kent Beck, and many others in our industry who care about building quality systems. Rozanski and Woods work on Viewpoints and Perspectives was particularly enlightening for how to think about the concerns of a system.
My colleague Don Mills introduced me to the term “innovation within guardrails” and it occurred to me that design principles are one way of establishing the guardrails. They set the boundaries for what the system will (or will not) do. There are thousands of design decisions that happen when software is being built. Some are innocuous, some could have far-ranging impacts (both positive and negative). Principles, once adopted, enable engineers to confidently make decisions to meet delivery commitments while in-line with the overarching technical vision of the system.
As a technical leader, you must learn that you can’t be with the development team all the time. So, early on you must establish a system that provides concrete guidance to them without being overly prescriptive. Focusing on the design guardrails enables the team to innovate within certain constraints of the system. It empowers teams to make local design decisions, so long as they don’t run counter to the shared principles.
While design principles often come from technical leaders, that’s not always the case. Some of the most famous design principles were laid out by Jeff Bezos in a 2002 email that helped set the amazing trajectory for Amazon. “All teams will henceforth expose their data and functionality through service interfaces.”
So, how do you get started? Whether you’re starting greenfield or brownfield project, the process is the same:
- Organize your system into a set of topic areas or concerns.
- For each, identify the guard rails that must not be broken.
- Socialize and refine.
In terms of the principles themselves, my advice is:
- Start with a simple list and avoid long narrative. Make each individually, the group collectively, easy to remember and apply.
- Keep it to a single page (or in Bezos’ case an email). Any more than one page and no one will read it.
- Write it with language everyone—not just techies—can understand. Be clear and concise. Write then edit.
- Share them prominently so everyone can see them. Link to them from your README file. Publicize them on Slack. Reference them during code reviews. Keep them top of mind for the team.
As teams navigate the inevitable challenges of development, where things don’t always go to plan, the design principles serve as a compass to guide the team. They provide latitude to designers and engineers to innovate, while also keeping them within the constraints required for the overall system to perform.
For our Team Insights product, we defined our Design Principles that spanned three areas: Cloud Architecture, App Architecture, and Dev & Ops processes. Once socialized, the message to the team was, “you are empowered to make any design decisions you want, just don’t violate the principles in doing so.”
In doing this, we learned an important lesson too. Initially, we thought the product needed many fine-grained services, so we wrote a principle about using microservices. After about a month into development, we realized this didn’t work for productivity and collaboration in the team. We already had 7 code repos with different parts of the system that were all independent, but it took a lot of work to stitch together all the code into a single application to develop, test, and deploy. Not so good.
So, we changed the design to be a monolith to start and updated the principle. Decision made, principle refined, team informed. In retrospect, this was a principle we put in place too early before we needed it. Lesson learned: don’t just put in principles to use the latest methods before seeing if they’re actually needed.
Introducing Design Principles to Your Team
Let’s jump into action. In my experience there’s three basic steps to introducing design principles to a team:
Create – Draft a set of principles (think MVP to start).
Adopt – Socialize and rollout to your team.
Refine – Update based on feedback and changing requirements.
Do this process then rinse and repeat. Now, let’s dive into the details.
The lead technical person normally takes responsibility for creating the design principles. How they create them can be as simple as a solo exercise in writing a bulleted list and sending an email (Bezos’ method). A more collaborative way such as a Design Principles workshop is another method for drafting a set of design principles.
Once you have design principles, it’s time to share with others. This can be done through your normal communication channels. The first time I share them I like to send the principles in advance and have people review them, then setup a meeting for feedback and discussion. When we meet, we spend most of our time on the ones people find confusing (or discouraging) rather than reviewing one by one.
Depending on your culture, design principles may come across as top-down and stifling. As you share them it’s important to focus on the benefits for the team—primarily that agreed-upon principles reduce micro-management on every last design challenge. It also encourages (but doesn’t guarantee) that the sum of the system parts will “work together as a whole.”
Without agreement from your teams to abide by the principles, they are a worthless exercise. So, ensure you spend as much energy (likely more) in promoting adoption as defining them. Be an active listener and seek feedback from your team. Be willing to make suggested changes if they make sense. Remind folks that the goal is innovation within guardrails and the design principles serve as those guardrails. They are a living agreement, not something to be hung on the wall to collect dust.
While some principles easily stand the test of time, others must evolve as time evolves. I think of design principles as a living set of statements that should be refined over time. Systems change. Requirements change. Context changes. So too may your design principles.
How often they change will depend on where you are in the development lifecycle. Early on they may change more often, but over time they tend to stabilize and change less frequently. As your system goes live and your team works through incidents and defects, gaps may emerge in your design principles that should be addressed by updates. Alternatively, things you may have feared would happen never materialize, creating an opportunity to remove principles—which is even better.
When updating the principles be sure to communicate to the team the “why” behind the change. Just remember to keep it to a single page and use clear and concise language.
Technical leaders have many tools in their toolkits for solving complex problems, and design principles are just one. But they are a simple tool that can promote innovation within guardrails and connect architecture with engineering.
So, the next time your organization decides to build Version 2.0 and you get nominated for team #ThunderCatz just remember: it’s not about what you can do, it’s about what you can get a team to do. Give design principles a try.