So what should an ‘agile architect’ be doing?

“So many architects walk around with the world on their shoulders. The responsibility of figuring out all this ‘stuff’ is supposedly taxing. Screw that… You are a creator… Go create…”

Agile Architect in his natural habitat.jpg

“An agile architect in his natural habitat dancing in a field of complexity…”

I recently wrote a blog on whether software architecture matters. It was a quick stream of consciousness hacked out in record time like most of my blog entries (probably riddled with grammatical errors and spelling mistakes). And like many of my blog entries it got a few muted grumbles from my extended professional network, especially from those that are ‘architects’ by trade.

The one thing that all of the grumbles had in common was the question of what I think a so called ‘agile architect’ should  be doing? This because I suggested that most of the design decisions should sit with the team. Not wanting to disappoint my somewhat less than usually chirpy ‘architect’ counterparts here is my list of things of things I believe we should be doing collectively:

1. Be part of the the development team

You are there to inform, motivate and influence the team without expending social capital by forcing your decisions. Being removed from the actual development effort and simply flinging architectural documents (that probably don’t reflect actual system code) at devs is not good enough anymore.

2. Adapt to the team’s way of working, not the other way around

In order to build high performance teams you need to create space for autonomy, mastery and purpose. The autonomy boundary can be constrained, but each team’s sweet spot in terms of their way of working will differ slightly. Standardizing your approach will simply lack the flexibility and variability that modern teams need and require.

3. Your job is to accelerate the project not to slow it down

Change priorities, trash requirements, architecture spike yourself to death. Whatever you do don’t stop the flow of value creation. There are many things that need to happen in projects. Many of those don’t need to stand still while perfect understanding (or consensus) is sought. Too many ‘architects’ plant their heels into the ground demanding big analysis when uncertainty is encountered. By nature they are top down thinkers that want to completely disseminate and understand a problem before making decisions. This however introduces a ton of friction into the development process. Defer as much decision making for as late as possible, outsource as much risk to others, focus on the really scary parts that can blow your design but don’t become another blocker.

4. Your job is preliminary about communication, do it often and early

It is really hard to create a shared understanding in the minds of everyone involved in a project (business, tech and design). Add different personalities, skills and backgrounds and it gets even harder. Heck even involving too many people or the wrong people too early is enough to explode the number of communication paths you have to contend with. So be transparent and open at all times. Trash complicated diagram standards and do everything in your power to drive context and understanding. Simplify and re-frame things over-and-over. Repeat this relentlessly until a semi coherent common understanding is reached and people start taking initiative on their own. Once that happens simply curate and make sure it is correct being careful not to trample and people’s attempt at contextualization.

5. Focus on the essence of the problem, the simpler the better

Stop creating artifacts, documents and stuff that nobody can contextualize. Even if you have to explain things over-and-over again this will refine the concept in your head and provide extra insights. Look at initiatives like C4 diagrams to get your drawings more in line with the actual thing being built.

6. Test your architecture

POCs or Architecture spikes are single-handedly the best way to validate parts of your architecture. But in too many organizations this is seen as a waste of time and money. Change the way you fund projects and cater for this activity. While you are at it always add slack for testing the non-functionals of your architecture (scalability, performance and robustness).

7. Question standards

Corporations are rife with cargo-cults that never question why they do the things that they do. “It is just the way it is done, or the standard”, is a common refrain. This is such a bullshit attitude that guarantees sub-optimal solutions. Be ruthless in questioning ordained “standards”. Especially if a “standard” is merely written down glorified opinion piece that has never been tested and validated.

8. Promote agile practices in other areas of the business

Unfortunately most organizations only embrace agility in their delivery teams. It is your job to promote lean and agile practices in the supporting functions that support the team.

9. The most important thing: Don’t take yourself so seriously

So many architects walk around with the world on their shoulders. The responsibility of figuring out all this “stuff” is supposedly taxing. Screw that… You are a creator… Go create great things with your teams, have fun with them and enjoy it. Stop taking yourself so seriously.

 

Advertisements

Does software architecture matter?

“This means that no distinction should exist between architecture and the act of coding. It is an activity that should be a shared concern of an autonomous team building a system collaboratively.”

fba7038527ae2d58a26a984c58e6e95d.jpg

The strangest version of this functional role is when someone that does not come from a technical background starts making proclamations about architecture…

One of the strangest functional roles that exist in corporate software development environments is that of the “software architect”. The very term evokes images of a senior engineer that has by virtue of his or her experience mastered his or her craft so well that they need not partake in trivialities such as writing code anymore.

Just like the architect in the movie ‘The Matrix’, these individuals can simply sit and make high brow proclamations about how software should be built. Afterwards they can marvel at the genius of what they have supposedly conceptualized while skeptically bemoaning the apparent failures on part of the people that had to execute on their vision.

The biggest problem with having these ivory tower styled architects is that if you are going to be conceiving of software design (and be a “technical person”), you simply have to be comfortable with the actual act of coding. That is after all the activity that makes the design real and has it own set of nuances and intricacies.

This means that no distinction should exist between architecture and the act of coding. It is an activity that should be a shared concern of an autonomous team building a system collaboratively.

Of course it helps being able to articulate architecture and highlight impacts of technical decision making especially in ways that non-technical people can understand. But that is a skill that every technologist should strive to perfect, not just the select few that carry the title of an architect.

The strangest version of this functional role is when someone that does not come from a technical background starts making proclamations about architecture and gets labelled as one. The subtleties of technology frameworks and practicalities of actually building coherent software systems would be completely lost on such an individual. Yet modern corporate environments are rife with these types of individuals who “build” systems every day.

So what is architecture then if anything at all?

Personally I like Martin Fowler’s view that if there is such as thing as architecture, it is essentially about two things. One is about creating a shared understanding in your teams and the other is actually investigating and answering the things in the projects that will be hard to change in the future.

The rest you can leave to your very capable and trusted development team.

The silent devastation of ideas-people

“The reality is that there is nothing quite as useless in this world as an ‘ideas person’. Not only is it excruciating to deal with their egos and misplaced protectionism around their precious ideas, but the reality is simply that anyone can dream and do what they do.”

In the software industry you often come across so called ‘ideas-people’

Behold the unicorn ideas-person. Full of ideas and low on execution.

In the software industry you often come across so called ‘ideas-people’. You usually spot them talking loftily about some idea that they are ‘working’ on. They might even verbally insinuate how big their idea is, and how it is going to change everything. Typically they can flip lyrically between the hundreds (if not thousands) of ideas that they have had. When someone else in industry has had a success that vaguely resembles one of their previous ideas they will publicly lament the fact of how they had that idea years ago, blaming circumstance for not pursuing it. They can go for years carefully nurturing their previous big idea, waiting for life to align just right for them to finally nab that sucker.

The worst part is that their fanaticism around their ideas frequently gets mistaken for passion. Sometimes if the person has the gift of the gab some might be fooled into thinking that such a person is a visionary of sorts. They might get dumb investor money thrown at them. When they fail – which is most of the time – they typically blame external factors for it. The market may not have been ready for their product or the investor funds dried up just before the big breakthrough was made. If they are not technical their darn developers were either too slow, too expensive or too inexperienced. It is very rare that they will blame themselves or their godlike ideas for the (not their) failure. Admitting failure would erode the one thing they hold most dear, their ego.

The reality is that there is nothing quite as useless in this world as an ‘ideas person’. Not only is it excruciating to deal with their egos and misplaced protectionism around their precious ideas, but the reality is simply that anyone can dream and do what they do. Coming up with ideas is not some mystical perch that is reserved for an elite category of human beings that have had some rare grace bestowed upon them giving them raven like vision into the future. In fact if you disseminate most of the things ideas-people come up with you will soon realize that it is nothing more than imitated innovation with concepts simply being copied from other successes they so diligently study and follow.

True visionaries are dreamers but more importantly they are doers. Make no mistake dreaming is a big part of the package of a true visionary but you also have to be able to follow through and actually make it happen. Ideas without execution is absolutely meaningless and wastes everyone’s time. Execution is king.

The devastation that ideas-people wreak is especially magnified in the corporate environment where ideas-people can plow their trade with minimal risk to their livelihood and reputations. If half-baked ideas fail in corporate environments it is virtually impossible to attribute causation since there are simply too many moving parts that may have contributed to that failure. As such a non-execution ideas guru can almost always abscond their precious idea from being the true reason for failure.

In contrast – even though it is not perfect – the startup world does do a fairly good job of weeding out the doers from just the ideas people. Unlike the corporate environment execution is everything in startups. With all the fat taken out of the process there is simply no place to hide. If you cannot build something that is useful to humanity and that people want to use your dead. It is a brutal game that rewards the true visionaries that can actually follow through on their ideas and destroys the pretenders that actually just like the sound of their own voice.

The really tricky bit is that you actually want to encourage innovation and experimentation in corporate environments since this is the only way to evolve and grow. But the people typically tasked with coming up with these strategies and product ideas are usually a little bit light in credentials for following through on those ideas on their own accord.

Project Delays: Why corporate companies are terrible at estimation

“‘The creation of genuinely new software has far more in common with developing a new theory of physics than it does with producing cars or watches on an assembly line.” – Bollinger, T

Why corporate companies are terrible at estimation

At the heart of this problem lies the phenomenon of discovered complexity.

One of the biggest problems in software projects is the inherent difficulty of planning and estimation. It is a problem as old as the industry and almost everybody in the life of a project regardless of their role will have a strong opinion on the matter.

Similarly every company will have its collection of war stories about projects that have slipped spectacularly or seemed to perennially overrun on their budgets and targets. Jobs may have been lost, careers ended and most importantly relationships destroyed.

For some companies this is such a common occurrence that the devastation wreaked by bad estimation may have become the norm instead of an exception. It could have become part of their culture to operate in a state of constant project flux. In environments such as these the trust between business and technology may be so eroded that any real attempt at even coming up with reasonable project estimates get abandoned regularly.

Instead it gets replaced with a constant barrage of unrealistic business demands that is framed as ‘critical-delivery’, typically based on no quantitative reality of effort but exclusively on what business would want to happen by a certain date for some particular reason. These ‘immovable’ business targets generally are manufactured to squeeze out as much delivery in a world that is hard to quantify. Where it gets ugly is that these targets are often arbitrarily tied to people’s KPIs even before the feasibility of the piece of work is explored. Typically the lucky recipients on these KPIs are people that have no experience in how to do proper software estimation, and who have no understanding of the differences between an estimate, target and goal (go read Software Estimation: Demystifying the Black Art by Steve McConnell).

At the heart of this problem lies the phenomenon of discovered complexity. For some reason people view software as a mechanical task similar to that of an assembly line in a factory. But as research has shown (even mathematically), ‘the creation of genuinely new software has far more in common with developing a new theory of physics than it does with producing cars or watches on an assembly line.’ [Bollinger, T]. It is a creative process that requires a lot of thinking about as of yet unknown problems that constantly present themselves. These accidentally discovered problems may in turn actually lead to more subtle and indirect problems that also require solving. This is called accidental discovered complexity.

Put simply at a particular point in time, you don’t know what you don’t know yet and there is no way to tell how long it will take you to figure it out when you discover it. In a world such as this how can you definitively estimate how long something is going to take? At best you can give a reasonable guess using the information you have at your disposal at that point in time. But even this is a problem since what you know about the problem domain develops and changes as the project progresses.

Given that it seems virtually impossible to estimate these creative tasks with reasonable accuracy. It therefore becomes imperative that you follow an iterative development methodology where there is a constant feedback loop on estimates. If you are not continually re-estimating how long a piece of work is going to take based on all the new information constantly learned, you are putting your project and more importantly your organizational goals and commitments at risk. Do that often enough and watch the carnage unfold.

The learned helplessness of corporate companies

“I don’t necessarily want to elaborate on all the details of the experiment but in a nutshell it pretty much involved electrocuting dogs in controlled experimental chambers where their only option was to take the shock.”

Learned helplessness in corporate companies

It is really hard not to become despondent in bureaucratic organizations

One of the strangest things that I have witnessed in many bureaucratic organizations is the phenomenon of learned helplessness.

In 1965 Martin Seligman and his colleagues were doing research on classic conditioning. I don’t necessarily want to elaborate on all the details of the experiment but in a nutshell it pretty much involved electrocuting dogs in controlled experimental chambers where their only option was to take the shock. The negative conditioning (zapping fluffy) when done enough induced a sort of learned helplessness in the dogs leading them to avoid seeking potential escape. Even when the configuration of the chamber was changed with an obvious exit in plain sight, the poor creatures still settled for the familiar negative reality of being shocked.

I believe that most corporate environments are like these chambers. There are simply too many people in what is more than likely a heavily regulated industry working in archaic ineffective ways with teams structured in Orwellian like configurations. In such a space if you put one foot wrong or deviate one iota from the ordained norm there is a generous amount of individuals that will stand in line to tell you how what you are trying to achieve is simply not possible and contrary to what is expected. These self-anointed guardians of all things process will then place as many obstacles in your way to prevent you from deviating from the learned norm.

The go-to argument most of the time is governance. The sad truth is that in these toxic environments the norm has become for people to use governance and regulation to build intricate systems of excuses whereby they can absolve themselves from ever having to take any accountability for real change or progress. This self-induced learned helplessness gets passed on from culture to culture, team to team until the vast majority of organization live in an eternal state of feeling hamstrung and dis-empowered, never venturing beyond the safety of their constant suffering.

To be fair most of these individuals have been smacked so many times by the same forces of self-induced learned helplessness in others that you can’t blame them for losing hope. It is really hard not to become despondent when facing the illogical reasoning of some of the hurdles that get placed in front of change agents.

There is however one even bigger evil that lurks in the shadows of learned helplessness. It comes in the form of the few that have learned how to exploit the learned helplessness for their own political benefit. The ones that dramatize and use the hyped up implications of these self-imposed governance policies to manufacture artificial urgency in teams and people. This fear based approach forces people to continually go above and beyond to get things done frequently for their own narrow need. In the process employee well-being frequently gets destroyed. This type of behavior almost always obliterates any hope of having a caring psychologically safe environment and can so easily become the norm for a learned helplessness infused environment.

So in closing, all leaders face constraints. I don’t for one second suggest that there is not real constraints placed on many industries. But true leaders differentiate between constraints that must be accepted and those that can be challenged. True leaders show their people that it is not only okay to go against the grain, but that they will be supported by the leader in that moment when they want to make a stand against the machine.