On Agile: Generalists vs Specialists

Let's imagine you're a Program/Product Manager, SDM, or Lead Engineer/Architect.  You're starting a program to develop tech thingie 'X'.  You've read all the books.  You've looked into Agile, from the brevity of the Agile Manifesto, to the what-are-you-selling nonsense that is the Scaled Agile Framework.

In all that, you come to the same decision that people have had since Amenhotep designed the first pyramid:  How do you organize yourself?  That is, how do you set-up your group of people to accomplish the task?

Digression: On the Spotify Model

At this point, many of you will yell: "Self organizing teams!!!!" like it's some sort of talisman, obviating any need of further thought.  "Smart people will organize themselves optimally."

Let's take this head-on: No, they won't.   To be precise: At scale, this does not work. For our purposes, "scale" is about 10-20 people or more.   They'll do their best, but results will waste everyone's time and money versus someone who laid out an organization structure with clear lines of ownership.

But, please, leave a comment on how Spotify does that and they're successful.   Suffice it to say, there are examples where that works, but they're usually:

1. Open Source projects without a schedule or budget (e.g. they're "not real" from a business perspective),  or 
2. Companies at a micro-scale with super-smart people, in which case I argue (at length) ANYTHING WILL SUCCEED.  For awhile.

Moving on.....

What I'd like to treat today is the question that naturally arises in Agile:  Do you have generalists, or specialists?  I'll attempt to consider both options and provide a pragmatic answer that combines both approaches.

Generalists and Specialists Defined

Let's first define our terms:

• Generalists implies that pretty much anyone in your (team, group, squad, crossfit box) can do every task.   Bob and Sue are "Software Development Engineers" and hence can do front end, coding, database work, infrastructure, design, etc.  They're also intimately familiar with the target domain of your product--insurance, personal finance, selling things, businesses.  As will see, that's the  real trick.  (Following the axiom: Most things you do aren't technical problems, they're people problems)
• Specialists turns that on its head.  Instead of organizing by general talent and capability, you organize around job function:  Bob is a DBA and Sue is a back-end developer.   Bob has no idea how to write Groovy/PHP/etc.  Sue couldn't write a query-optimized stored procedure if her life depended on it.  Carrying this further, you need Business Analysts (BAs) to know the domain, feeding requirements and specs/mocks/post-it notes to the people who do the actual implementation.

Why Neither Works

If the second description sounds pejorative, forgive me.  I've grown-up in the world where "specialists" were decried and we were supposed to all be generalists.  The benefits are obvious, right?  You have 50 people you can apply to whatever problem in your business domain, versus 10 DBAs, 20 coders, and 20 front-end people with a Business Analyst.  Your job as a Manager/PM/Architect is so much easier.  Just throw people at the problem.

If you're still reading, you understand:  It's not that simple.  In that model, people hyper-define their role and hold up their job description at you when you ask them to do something and scream, "That's not my job."  It's a management nightmare.  You'll end-up with 50 people who have the productivity of 10 people.

Generalists are broad and shallow, by definition.  They probably know a little about everything, but they're not tossing down the super-tuned Stored Procedure or whiz-bang front-end thing.   At best they're overweighted in certain areas, and they get-by in others.   When you're looking for that competitive edge, you need some people with true domain mastery. If you're running a 15-30% attrition rate, your generalists will never get there.

So, what to do?  

Proposed Way Forward

I'd like to propose a simple solution:

• Make coehesive teams  Make your "unit of responsibility" a 4-to-8 person team.  Work doesn't go to 'Bob' it goes to the web experience team.  This lowers your risk and bus factor, and eliminates chaos.  Things have an owner, and that owner is a team.
• Teams get to specialize That team has a well-defined, domain-specialized responsibility within your thingie 'X'.  This serves individuals' need for mastery. People naturally become good at anticipating customer needs and technical issues within that domain.  This is much better than dropping in-and-out of disparate domains iteration to iteration.  You'll notice your employee satisfaction and retention increase, too.  People want to grow and get good at things.  Work with that, not against it.
• Within that team, everyone is a generalist.  They have knowledge, procedures, and automation to do anything that team needs to do.  Your technical risk will spread.  Its individuals' responsibility to come up to the bar of the team's capability and raise that, mentoring others.  
• Rotate people through the teams. Make sure people have growth opportunity and that they don't become stale and over-specialized.

This has several immediate advantages.  At the program level, you can deal with (say) 5 teams much easier than 40 individual developers.  You'll have an innate sense of "where things go." This is classic hierarchy in the Persian/Roman army sense, with units of people that "live" daily (~8 people), and units that go to battle (up to 80 people).  Leadership can maneuver the teams in a way that maps to the human wetware 7 +/- 2 rule.   Let human biology work for you.

Some things be aware of:

• The teams' culture will diverge.  One team may be more process-focused.  Another adjacent one may be more slash-and-hack get-things-done.  That's natural, inasmuch as individual personalities diverge.  As leadership, encourage team identity, but discourage contempt or outright win/lose thinking.  We're all trying to get there, wherever "there" is.
• People will grow stale if you let them.  Insist people rotate through the teams or through other groups in your company.  People managing services need a rotation through front-end or mobile, and vice-versa.
• People will try to specialize within the team and avoid certain tasks that are team responsibilities.  You'll need to be diligent in not allowing this to happen, or your right back to 

In summary, organizing people is one of the key challenges in humanity, in whatever goal you're trying to achieve.   The "Agile Revolution" brought considerable energy, but it hasn't fundamentally changed the underlying drives of human beings.  Above, I've argued that successful, sustainable technology efforts should organize around domain-specialized teams populated by generalists within that domain.  This lowers your risk profile while making your organization of 40 as nimble as one of 5 reports when you're planning and executing.

After 18 years, that's just what makes sense to me.