Why Agile is Here to Stay

Why Agile? Throw the question out to programmers, consultants, or even Google, and you’ll get tributes or tirades, with little in between. It’s not just software developers that use “Agile” ways of working. Now there are even agile marketing and agile HR teams. In fact, 90% of organizations surveyed for the State of Agile Report say they practice agile with most of their teams.

So why is Agile so popular among teams, companies, and stakeholders?

This resource seeks to explain why Agile – an adaptive approach for project management – is loved by some, loathed by others, but destined to stick around.

Agile is a project management approach that prioritises short work iterations over long-term project deadlines. Agile is built around a few core principles, such as working iteratively, putting the customer first, only committing to building small increments at a time, continuous improvement, and working in a lean way.

Over the two decades since Agile emerged, it has exploded in popularity, with many companies attempting to “go Agile” or launching “Agile transformations” to help them build better products faster.

The story of Agile begins in 2001, when 17 developers met in Utah and crafted The Agile Manifesto.

Its principles inspired people everywhere to create new practices for their day-to-day work.

The Agile Manifesto is only a few lines long but has had a huge impact on the world:

Alongside the Agile Manifesto are 12 Agile principles to guide would-be Agilists in their implementations.

When you read the manifesto and its principles, two things stand out:

The Agile manifesto doesn’t give any practical guidance or agile process for implementing its ideas. That’s why many agile frameworks have emerged, like Scrum, Extreme Programming (XP), and SAFe that offer options for putting Agile principles into practice.

Whereas the Agile Manifesto only provides the why and what, these methods provide the how of agile working. They describe things like which roles to create on your teams, the type of meetings to plan, and how to run them.

💡 Fun fact: Did you know that the Scrum framework pre-dates the Agile Manifesto, even though it is now the most popular “agile framework”. 81% of agile teams using some version of Scrum, including Scrumban or a hybrid Scrum model.

Agile offers companies a way to deal with the fast-changing world around us – including shifting market conditions, customer preferences, and technology. Today’s teams also expect more autonomy in their work. Agile fits these requirements and improves customer satisfaction, too.

Project managers, teams, and organizations often want to “go Agile” because it:

💡 Fun fact: 93% of companies in one study agreed their agile business units outperformed others on customer satisfaction during the COVID-19 pandemic.

Before agile software development, few people were happy with the way software projects were executed.

Programmers spent much of their days writing documentation instead of code or were stuck in meetings. While product owners hyperventilated in a corner somewhere as launch deadlines arrived without much evidence that the product would actually work.

Agile changed all that by inspiring new, iterative ways of working that lead to high-quality outcomes.

Agile advocates often discuss it as a mindset and a way of living – one that cuts through the bureaucracy of traditional teaming and makes work more meaningful, efficient, and empowering for everyone involved.

Before Agile, teams would manage the software development process based on a waterfall model – the knowledge-work equivalent of a conveyor belt. In such a traditional project model, every phase and task moves sequentially. Nothing can happen in parallel. Think: Gantt charts.

As with the factory conveyor belt, one problem can hold up the entire project, and the customer only gets to see the finished product at the end. By that time, they might not even want that product anymore.

The waterfall model doesn’t leave any room for changes or additions once development is underway. Sales reps and customers would bicker over the scope of the contract, agreeing on what would and would not be built. The result: bloated project budgets and over-runs.

As Lean manufacturing transformed factories, Agile transformed software projects and other knowledge work. One of the best real-world examples of Agile’s effectiveness compared to waterfall is the turnaround of the FBI’s Sentinel project.

Sentinel started in 2005 to merge the FBI’s internal data and make it shareable across the organization. By 2010, using traditional software development methodologies, the project was a year behind schedule, only halfway finished, and close to blowing past its $451 million budget.

Then Chad Fulgham, the FBI’s chief information officer (CIO), brought in Jeff Johnson, a Scrum expert. He reduced the project’s team size by 80%, spent just $20 million, and finished the project within 18 months.

Two of the most popular Agile frameworks are Scrum and Kanban. While mastery requires longer – and help from an Agile Coach is recommended – teams can be “doing Agile” within hours with these approaches, which is why they’ve become so widespread. The easiest starting point is simply running regular retrospective meetings to reflect and improve on work.

Scrum, for example, originates from the official Scrum Guide, a short PDF you can read in less than an hour. It describes a few roles and meetings your team can adopt immediately, without further cost or training. Understanding and setting up a Scrum board or starting retrospectives takes less than an hour. You can even do it without disrupting your team’s existing methods and practices.

Kanban’s central element is the Kanban Board, a board in which columns reflect the different phases of a team’s working process. Cards representing work items move through these columns, from idea to completion. This visualization of work helps reduce bottlenecks and gives a “status update” to anyone who looks at the board.

Despite its soaring popularity, many managers dislike Agile’s mantra of decentralization and self-management.

Others fear it will cause organizational anarchy or will make their roles redundant.

When Agile is practiced incorrectly it can also become miserable.

Daily Scrum meetings can turn into endless ordeals every day of the week if they’re not properly time-boxed. It’s also all too common for teams to find they’re still practicing a half Agile half waterfall medley that causes a lot of stress and insecurity.

Many companies also use Agile as an excuse to introduce more metrics, processes, and control from the top down rather than empowering teams to arrange these for themselves.

Some Agile Coaches and consultants from the so-called “Agile-Industrial Complex” alienate teams with rigid rules presented as Agile laws.

They proclaim right and wrong ways of doing Agile instead of teaching its fundamentals, so teams can adapt it to their specific needs.

In its most innocent form, a well-meaning but inexperienced Scrum Master doesn’t allow any deviation from the Scrum Guide because it’s meant to be “immutable” and they’re insecure about what happens when they stray from the official path.

In darker scenarios, some companies purposely dictate a complex, rigid framework labeled as Agile to demand a hefty licensing bill and hourly fees to “make it work.”

The words agility and rigidity are essentially opposites. There’s nothing wrong with using a particular method, but it should be a starting point and guide, not dogma.

Agile’s aim is continuous improvement — to constantly experiment and adapt in service of the customer, team, and organization. Not to blindly follow one framework to the letter because “the guide said so”.

When organizations misunderstand Agile, teams do daily standup meetings, write user stories, and call projects sprints without knowing why. Worse, managers let them go through these motions on top of their existing practices, causing burnout.

The result? “Conformance Scrum” (or Zombie Scrum for horror movie fans) and “waterfall development with additional overheads like sprints and iterations.”

Working in an Agile way is meant to reduce bureaucracy by making space for deep work. When misapplied, it creates more bureaucracy that reduces productivity and increases frustration – the opposite of what Agile should achieve.

Max Andaker, co-founder and chief product officer (CPO) at CoScreen (now Datadog), told us:

Confusion around Agile terminology also doesn’t help Agile’s case: Agile, agile, Scrum, daily scrums, and daily standups, to name a few confusing agile terms. When even experienced practitioners use different words for the same thing or the same word for different things, you can be sure everyone else gets confused, too.

A simple internet search doesn’t always fix things either. As Monica Georgieff, Agile coach and trainer at Agile Sherpas, told us:

This confusion creates divisions among developers and between development teams and business folks. They no longer speak the same language and misunderstand each other.

Agile is not a magic solution that solves every organization’s problems. Yet, ignorance of Agile’s strengths and weaknesses, its soaring popularity, and commercial interests from consultants and coaches, means Agile often gets applied in cases where it shouldn’t.

Here’s why Agile doesn’t work for some teams:

Maarten Dalmijn, Serious Scrum ambassador, told us he sometimes doesn’t pick an agile approach for projects he’s involved in:

Agile teams thrive when they understand its principles and apply them in the proper context. It works particularly well amid rapidly changing circumstances and unpredictability – precisely what the future holds for most companies doing complex knowledge work.

Agile frameworks like Scrum create a sustainable work rhythm. Instead of thinking, “I can sit back because the deadline is in six months,” or being overwhelmed by the enormous task ahead, you work toward smaller, shorter-term goals that prevent project stagnation and all-nighters at the end.

You also have more time for deep work because meetings are fixed at scheduled times and places, reducing ad hoc distractions.

These shorter, focused cycles also allow agile teams to iterate faster. They constantly ship working features, which keeps them close to the customer.

As Jeff Sutherland, co-creator of Scrum and a signatory of the Agile Manifesto, says, Scrum helps teams “do twice the work in half the time.” He regularly sees Scrum teams improve their productivity by 300%, sometimes even 800%.

Now, Scrum’s creator might be biased about his own method. But research by management consultancy JCURV found that 70% of agile organizations think that agile helps them achieve a faster time to market. Such performance improvements mean more companies will adopt agile or keep it around for the long term, regardless of who loves or loathes it.

Agile is… agile. Its principles and frameworks are flexible rather than rigid, resilient instead of fragile. These qualities give organizations and teams the ability to adapt and evolve their processes along with projects, customers, and the larger world.

For example, during Sprint Retrospectives – a meeting type in the Scrum framework – teams evolve by scrutinizing their own performance. Jordan Husney, CEO of Agile meeting platform Parabol, says:

Agile’s evolution is a testimony to this ability to change. There are now dozens of frameworks based on agile principles, and new ones appear every year. Some, like Scrum or Kanban, are widely adopted. Others, like the so-called “Spotify Model” for doing Agile, are tailored to specific organizations. Fredrik Carleson, an IT professional, says:

Agile has spread far beyond its software origins. National Public Radio uses it to create new programs, John Deere to develop machines, and Saab to make fighter jets. It’s also used across disciplines like marketing, HR, and executive management. For example, in recent research by Agile Sherpas, 51% of marketing teams considered themselves agile.

Agile isn’t a silver bullet that solves every challenge under any circumstance. But it might be a good fit if you:

Do one or more of these points apply to your situation? Consider diving deeper into agile with the guides and resources listed below.

Agile Frameworks: A Complete Overview

So you want to “go Agile?” But which framework should you choose? Here’s an overview of all essential agile frameworks. We spoke with several Agile coaches to help you decide which framework is right for your team or organization. And we organized the most relevant frameworks according to team size and organization type.

Agile Meetings: 9 Formats from Different Frameworks

An agile meeting is typically considered any event that’s part of an Agile framework, method, or theory. This resource catalogs them all.

How to make your Agile implementation a success

A step-by-step overview of what to consider before embarking on an agile journey and how to implement one of its frameworks when you do.

Scrum 101

Scrum is the most popular agile framework. It’s easy to get started with and the most recommended take-off point for teams new to agile. This guide covers everything you need to know.

The 5 Scrum Ceremonies Explained for Remote Teams

An in-depth overview of Scrum’s five events: Sprint Planning, Daily Scrum, Sprint Review, Sprint Retrospective, and Backlog Refinement. The article includes guidance for remote and cross-functional teams.

The 9 Most Confusing Pairs of Scrum Terms Explained

A conversation about Scrum and Agile can quickly get confusing, even for the initiated. This resource covers and clarifies the nine most confusing terms.

Agile Glossary: Essential Terminology for Agile Teams

Want to get up to speed with all the Agile lingo? This comprehensive glossary is worth the whole team bookmarking so you can stay on the same page.

300+ Agile and Scrum Statistics for 2022

Perhaps you’re trying to figure out whether Agile is right for you, or justify it to your team. This resource is the most comprehensive list of Agile and Scrum statistics on the internet. You’re sure to find something useful.