For years, Agile strategies have favored self-organizing, collaborative teams, yet without clear strategic direction from team leaders, many Agile transformations fail, write Philippa Fewell and Cliff Berg, co-founders of Agile 2.

You’ve decided to create an innovative new product, or perhaps you just need to respond to market changes more rapidly, and you believe an Agile strategy will get you there. You hire the requisite Agile coaches and engage them to implement a combination of various frameworks and “best practices” across the organization. A year later, you are in no better place than when you started, possibly even worse. What happened?

Unfortunately, this situation is all too common. According to the Scrum Alliance, 47% of Agile transformations fail, with the number one cause being the use of what John Kotter calls “waterfall leadership.”

Marketing Technology News: How To Make Connections In An Online World

The Agile community has long dismissed the importance of leadership, advocating a single form of leadership known as servant leadership, or promoted self-organization and autonomy for teams as absolutes and the singular models to consider.

Agile philosophy dismissed the importance of leadership in reaction to the dysfunction of autocratic and theory X management styles. According to Mark Schwartz, former CIO of the US Citizenship and Immigration Service and a well-known Agile and DevOps evangelist, “the Agile world, ever suspicious of management, proceeds as if it can manage without the involvement of IT leaders.”

Yet leadership is the most essential element in undertaking any Agile initiative, whether it be innovative products, faster releases, or just simply building better quality software. Furthermore, leadership is a complex topic. Servant leadership is a very useful model, but it is just a screwdriver when one actually needs an entire toolbox. This is why leadership is a first-class element of Agile 2 – indeed, it is a central element.

The biggest mistake most leaders make is to think of Agile as a process change; simply swap out the old process and techniques and put in the new Agile framework. But Agile is much more than that. An agile transformation is a learning journey: one that requires a different way of thinking as well as a different way of working. It is about creatively designing and building something through collaboration and iterative inspect and adapt cycles. It is influenced by an organization’s culture and how decisions are made, which all starts with leadership.

With good leadership, all else will follow. With bad leadership, no framework, methodology, or process will succeed.

There is a need to balance the desire for autonomy within teams with the right level of management and guidance to achieve the expected outcomes. Teams and individuals should have the freedom of creativity on how to solve complex problems. They are the ones closest to the problem, and so have the best ideas of how to solve them. Yet teams, in general, do not self-organize all that well unless they have a long-standing relationship with one another and are very experienced.

A leader is someone who has influence, and a manager is a leader who has authority. Authority is necessary for certain situations but it should be used sparingly. One important use of authority is to intervene in a helpful way when teams struggle and help them to learn to self-organize, the paradox is that to do that well requires authority.

Marketing Technology News: MarTech Interview with Steven Kellam, SVP Marketing and Alliances at 360insights

Leadership can often take many different directions and many forms. Leadership is necessary within a team, across teams, and with groups outside the team. On a team that does not have a leader, generally one will emerge, which may not be the best outcome. Sometimes the best leaders are the ones who would not emerge as the leader: these might be the quiet helper or the natural organizer, or the person who can calmly talk through any issue with anyone else. Such people often operate in the background and do not emerge as “the leader” when in fact they would be the best suited.

Groups of teams need direction on collaboration and how to make decisions that are optimized for the whole and not just their team. Leadership is needed to create the vision and set the direction for the teams to follow. It is also needed to assess the outcomes and to decide whether they are worth the investment.

Leadership can also come in many styles, such as Servant leadership, Socratic leadership, Theory X and Theory Y, and Mission Command. There is no one right way or style; it is highly dependent on the situation, and leaders need to use judgment in deciding which will work best in the given context. Servant leadership is the style most commonly associated with Agile, usually in reference to a Scrum Master who is a leader with experience and influence, and who supports the team by removing impediments but does not necessarily have authority. Mission Command is another whereby the leader has authority but sets direction and grants autonomy to the team based on their competency and skill levels.  The amount of autonomy may increase as the experience and judgment of the team increases under the tutelage of the leader. Good leaders will immerse themselves in understanding and monitoring what is the right balance.

Agile has grown up. Agile 2 articulates a broad set of updated ideas, most importantly pertaining to leadership, that are needed to make the heart of Agile become a reality.