“After all these years, of hearing people talking about you, I defended you.  I tried to appease them … maybe as they said you were egotistical, ungrateful, tyrannical, closed-minded, anti-social, destructive and selfish, maybe they had a reason to say all of this.”

 The more I thought about this, the more I realized that my characteristics that are so obnoxious to so many people are the traits that have helped Scrum succeed (along with all of your efforts). If I were conciliatory and all those other things, Scrum might have become EssUpScrum, ScrumFall, Murcs, Scrum-Z, ScrumBan and all the other watered down adaptations that have been been suggested and pushed.

 So I decided that I will follow my New Year’s resolution for today only. This is your chance. After this, I will keep pursuing my mission of improving the software profession in character, the only way that I really know how.

 Thanks in advance for putting up with me during 2012.

 Best,

Ken Schwaber

From the foreword:

This book is for leaders within organizations that depend on software for their survival and competitiveness. It is for leaders within organizations that can benefit from developing software rapidly, incrementally, and with the best return on investment possible. It is for leaders who face business and technological complexity that has made the delivery of software difficult. We have written this book so that these leaders can help their organizations achieve these goals, enhance their internal capabilities, improve their product offerings, and more.

This book is for CEOs, executives, and senior managers who need their organizations to deliver better software in less time, with lower cost, with greater predictability, and with lower risk. For this audience, we have a message: You may have had negative experiences with software development in the past, but the industry has turned a corner. The software profession has radically improved its methods and its results. The uncertainty, risk, and waste to which you are accustomed are no longer par for the course. We have worked with many software organizations that have already turned the corner; we want to help you do so, too.

In this book, we show you how to create business value using a process that delivers complete pieces of software functionality at least every thirty days. This book will show you how you can prioritize the functionality you want and have it delivered á la carte. It will show you how to gain transparency not only into business value, by tracking functionality delivered against functionality desired, but also into the health of the software development process and your organization as a whole. The tools in this book will help you work with your software organization to get up to speed with modern practices and begin to deliver the results you’ve been expecting all along.

 This is software in 30 days.

Happy Holidays,

Ken and Jeff

December 25, 2011

I thoroughly appreciate all of the work everyone in the Scrum community and the wider Agile community has made to improve our profession. The insights that have moved us forward from the failures of the past are a good starting point, but only the courage, good will, and hard work of the last ten years have moved us forward. Look back, and take heart at what we have accomplished, together. Our insights and determination have been greater than our differences.

For the New Year, I propose a toast to the next ten years, as we rise to the challenges of increasingly complex and critical products for our increasingly complex societies.

Bottoms up!!

Ken Schwaber

December 24, 2011

* The Chaos Report (2011), The Standish Group International

The latest Scrum Guide was just released by me and Jeff, and is posted on Scrum.org. Some things like release planning, sprint tasks, and burndowns were removed from the formal definition of Scrum. They were removed because they weren’t Scrum. Are they useful? Absolutely! But it became apparent that these weren’t Scrum when people proposed other techniques that were equally effective. We certainly don’t want people to feel restricted or constrained from other effective practices if they use Scrum.

You should feel free to continue to use burndowns (Sprint and Release), to do release planning, and to commit. Do anything that works within the Scrum framework and aids you in doing your work, building complex products.

You even feel free to do things that aren’t coherent and consistent with Scrum. You are free to do continuous flow, but if you choose to do it without increments and iterative time-boxes, you aren’t doing Scrum. You are free to assign tasks to resources based on their capacity, but then you aren’t doing the self-organization required of Scrum.

We give you permission to do anything that you want. The Scrum Guide will help you understand whether that is Scrum or not. The results will help you decide whether that is a wise practice or technique, or not.

However, in his blog he has misinterpreted or misunderstood some key Scrum and Agile concepts. Many people do the same, but Brian’s influence is widespread and seen in the implementation of Microsoft products. I feel a need to address these conceptual flaws.

I’ve been nervous for a while about how well people would understand self-organizing Scrum Teams and the development teams within. Our tendency and tooling from waterfall and predictive processes is to view people as assignable, parsed, optimized resources. This works great if you are running a factory line and people are doing simple work. It really sucks if you are trying to do creative, complex work where there are many competing ideas and solutions emerge from interactions. The Agile Manifesto reflects this in “Individuals and Interactions over Processes and Tools.”

The new VS measures a person’s available capacity. You have to wonder. People don’t have measurable capacities when they are performing intellectual, creative work. Things move back and forth between different parts of the brain and some of the best ideas come at 2:00am in bed.  Scrum and XP are sneaky. A team of people sign up to solve problems within a Sprint. They are engaged in group problem-solving for the duration, and the problem never leaves some part of their brain. They are engaged. Brian’s comments and the consequent tooling reminds me of ideal days: a person is only working when his or her fingers are on the keyboard.

Last, but I’m sure I’ve missed others, is the idea of “assigned” work. This is a common smell of a development team that is not self-managing. Who “assigns” work if a development team is self-organizing? Development teams select work, figure out how to do it, and go do it. Assignments are a dysfunction.

In my measurements, self-organization is a prerequisite for Scrum and Agile productivity. This is where the 100%-plus productivity occurs not to mention the creative ideas, enjoyment of working, and quality.

Dr Jeffery Liker, author of “Toyota Way,” estimates that only a very small percentage of US Lean adoptions include its core principle, respecting, valuing and engaging the workers. There is a very tricky understanding between Scrum and non-Scrum. If we continue to apply the principles of directed, supervised work we are doomed to Lean and Scrum being another fad, another cover for a continuing use of Scientific Management and its management practices.

Many organizations have not adopted the self-organizing, team-based aspects of Agile. They still are predictive, top-down organizations. Tools that don’t support this function are hard to sell. However, form follows function. If we continue the same predictive manufacturing model, wrapped in Scrum tools, we as software professionals will have a very hard time rising to the increasing demands of our world for creative, sophisticated, quality products.

Scrum without self-organization and empowerment is a death march, just like waterfall, but an iterative, incremental death march without slack.

Also, the blog extols continuous flow. Now, I’m all for continuous delivery. It allows time for competing ideas, exploration, and emergent ideas. However, continuous flow is a Lean manufacturing technique, best formalized by Kanban. It is great for complicated work, such as service tickets, customer complaints, and some bug fixes. Kanban  is a very specific set of techniques from Lean that is applicable when the work is inherently simple. The work also may have been simplified, through standardized techniques, requirements description, or assigned roles. This is complicated work (where more things are known than unknown), not the complex work Scrum is designed for (where more things are unknown than known). Only the most sophisticated Scrum organizations have gotten to this type of work normalization and flow of work on products.

I’ve seen continuous flow touted as a simpler Scrum. It isn’t. It is another process for different types of problems. Waterfall was a simpler process for complex work: its failure rate indicated that it was inappropriate. The same failure rate will occur with Kanban as applied to the complex work of software development.

Jeff Sutherland, myself, and the Scrum community are updating Scrum with “Scrum Guide 2011″ on July 21. It contains some significant and subtle changes. These changes build on the core principle of people being the value in organizations. The thought path Microsoft is following continues the old path of people being micro-managed, interchangeable resources.

I’ve often wondered why Microsoft is building tooling around Scrum and doesn’t have either of the authors of Scrum, Jeff or myself, as an adviser. We could help it avoid these mistakes and keep its tools consistent with Scrum. The right process produces the right result (from Lean). Similarly, the right implementation of the right process produces the right result.

Only build tools for things in which you have expertise, not just the experience.

Best,

Ken

But, I was less than convinced. If we were a profession, how could I tell if one person was acting professionally, and another was not? What was an unprofessional act? Something that was not transparent? Cutting quality? Not refactoring design when enhancements were added? I struggled with the question.

I link to some discussion of professions from Wikipedia below. If these criteria were applied to people selling their services as software developers, then I would have a minimum expectation. I could expect that they were educated in a certain curricula, had apprenticed with an accepted professional somehow, and had passed an initial set of tests. I also would know that there was a formal licensing body that established and maintained these criteria. Finally, I would have confidence that the person had to periodically reassert his or her qualifications to continue his or her participation in the profession. I would know this because the professional would have a license from a professional body.

I again raise the question: Are we professionals? If yes, how do we fit the below descriptions, or are they wrong? If not, what are we?

——————————————————–

Formation of a profession

A profession arises when any trade or occupation transforms itself through “the development of formal qualifications based upon education, apprenticeship, and examinations, the emergence of regulatory bodies with powers to admit and discipline members, and some degree of monopoly rights.”

Regulation

Main article: Professional body

Professions are typically regulated by statute, with the responsibilities of enforcement delegated to respective professional bodies, whose function is to define, promote, oversee, support and regulate the affairs of its members. These bodies are responsible for the licensure of professionals, and may additionally set examinations of competence and enforce adherence to an ethical code of practice. However, they all require that the individual hold at least a first professional degree before licensure. There may be several such bodies for one profession in a single country, an example being the accountancy bodies (ACCA, ICAEW, ICAI, ICAS, CIPFA, AAPA, CIMA, IFA, CPA) of the United Kingdom, all of which have been given a Royal Charter although not necessarily considered to hold equivalent-level qualifications.

Typically, individuals are required by law to be qualified by a local professional body before they are permitted to practice in that profession. However, in some countries, individuals may not be required by law to be qualified by such a professional body in order to practice, as is the case for accountancy in the United Kingdom (except for auditing and insolvency work which legally require qualification by a professional body). In such cases, qualification by the professional bodies is effectively still considered a prerequisite to practice as most employers and clients stipulate that the individual hold such qualifications before hiring their services.

Source:http://en.wikipedia.org/wiki/Profession

A Product Owner plans a release based on all current information. He or she lays out the goals, the features and capabilities that will deliver those goals, and the probable cost and date of delivery. From that point on, the Product Owner’s job is to assess what is possible given the Team’s capabilities, and to make the best decisions to reach the desired goal. Given the nature of technology, markets, requirements, and people, trade-offs are made. Sometimes the goal cannot be reached for a reasonable price. Sometimes the goal will be reached, but in a way different from what the Product Owner initially intended. This is empiricism in action.

The Team (of developers) on the Scrum Team does the same. It meets with the Product Owner and assesses what the Product Owner views as the most important things to do next. If done, these will move the emergent product in the best direction toward the desired goal. The Team selects as much as it believes it can do over the upcoming Sprint. The cost of the team and Sprint length are fixed. Only the amount of Product Backlog selected can vary.  The Product Owner and Team often define a goal for the Sprint. This is a subset of the release goal.

When the Team selects Product Backlog Items in a Sprint Planning meeting, it commits to do it during the Sprint.  A definition of “commit” is:

To bind or obligate, as by pledge or assurance; pledge: to commit oneself to a promise; to be committed to a course of action. (www.dictionary.com)

This conforms to my intentions of the word commit, which is a pledge, a commitment to a course of action.

However, many Scrum Teams use the word commit as if it were a “guarantee.” This is a residue of waterfall, where an estimate was a contract. However, it still rings in the heads of product owners and developers. I have found team after team that feels they have to do anything to deliver their commitment. The usual victim is quality.

I wonder if we should change the word from “commit” to “forecast?” That might elicit the impression of the weather forecaster attempting to provide us with the best possible information. She works with what is known and the science of meteorology. She doesn’t provide a guarantee, but something that we can work with to make decisions. We find forecasts used by sales organizations as well. Perhaps this clarification will help us understand that a “commitment” in Scrum is a pledge to do our best with what we have.

PMI in the past has embraced the predictive, mechanistic approach to project management. This was first espoused by Frederick Taylor in “Principles of Scientific Management,” which was the basis of the Ford Model T assembly line. It is an approach for predictable, high volume, low cost manufacturing. Its benefits accrue from squeezing all of the unpredictability from the problem space through standardization and repetition. Perfect planning, training, and repeatability are the hallmarks. Plan and then do over and over again. The measurement of success is the yield rate, which is often very near 100%. Productivity is optimized through perfect processes, invariant workflow, and optimized use of resources (people or machines).

Agile processes, by way of contrast, work for development of complex products, where there is some repeatability but there is more new development than old. Products have to be devised anew each time as the requirements, the technologies, and the capabilities and creativity of the people change. In these situations, we’ve found that just-in-time planning, with frequent inspection and adaptation is required. Risks are managed and predictability created by limiting the time span between these planning events, and by ensuring the transparency of all artifacts. We have also found productivity, quality, and creativity is greatly enhanced if the people doing the work also plan their own work. They are not managed as resources, but as people that can do their best when they figure out how to do the work themselves. Self-organization and cross-functionality are vital to the success of these teams.

We have found that the role of the project manager is counterproductive in complex, creative work. The project manager’s thinking, as represented by the project plan, constrains the creativity and intelligence of everyone else on the project to that of the plan, rather than engaging everyone’s intelligence to best solve the problems.

In Scrum, we have removed the project manager. The Product Owner, or customer, provides just-in-time planning by telling the development team what is needed, as often as every month. The development team manages itself, turning as much of what the product owner wants into usable product as possible. The result is high productivity, creativity, and engaged customers.

We have replaced the project manager with the Scrum Master, who manages the process and helps the project and organization transition to agile practices.

How PMI bridges the profound difference in philosophy, thinking, leadership and management between its traditional predictive approach and the new agile empirical approach will be fascinating to watch. How it refashions the role of project manager will be an exercise in agility itself. I for one wish the people at PMI the best. They certainly need a better success rate than 50% to regain the trust of their customers.

———————————————————————-

1. Derived from “The Rise and Fall of the Chaos Report Figures”, January/February 2010 IEEE Software, J. Laurenz Eveleens and Chris Verhoef, Vrije Universiteit Amsterdam.

2. 2009/2010 Standish Chaos Report, Standish Group, indicating:

• 32%   Successful (On Time, On Budget, Fully Functional)
• 44%   Challenged (Late, Over Budget, And/Or Less than Promised Functionality)
• 24%   Failed (Canceled or never used)

3. In addition to a low yield rate (rate of successful delivery), statistics also point out that over 60% of the functionality delivered is rarely or never used, a tremendous waste of development and support funding. Resulting from not prioritizing requirements. http://en.wikipedia.org/wiki/Software_bloat, “Software Bloat.”

I pondered this. I didn’t remember having said this. Perhaps this was an evolution of my having said that only 30% of all teams or organizations that use Scrum will become excellent development organizations. That fits with my memory.

Scrum is like chess. You either play it as its rules state, or you don’t. Scrum and chess do not fail or succeed. They are either played, or not. Those who play both games and keep practicing may become very good at playing the games. In the case of chess, they may become Grand Masters. In the case of Scrum, they may become outstanding development organizations, cherished by their customers, loved by their users, and feared by their competitors.

Scrum appears to have “crossed the chasm,” as Geoffrey Moore described. Scrum is now more mainstream than radical. Scrum is sometimes more a fad than a serious endeavor. When it is adopted, some of its practices are inconsistent with the culture of the team or organization. In response, the team or organization changes Scrum so it is consistent and fits in. For instance, some managers like their predictions of how much work will be done to be true, regardless. The teams in these organizations change the quality of a Sprint’s increment so the predictions become true. Some managers like to believe that a team or organization will only succeed through the application of their own and only their own intelligence and insights. Self-organization of teams does not occur then.

There are many adaptations or modifications of Scrum by organizations. I’ve called them “Scrum buts.” I ask someone if they use Scrum. They respond, “Yes, we use Scrum – but, our manager believes that we will fail if he doesn’t tell us what to do, so we are told what to do and how to do it.”

So feel safe. Scrum will not fail you.

Good journeys.

Two Product Owner assessments are available. The first, PSPO I, is multiple-choice. It tests if the person taking the assessment knows Scrum and the role of the Product Owner. The second, PSPO II, is essay and multiple-choice. It tests if the person thinks like a Product Owner and knows how to create business agility.

The results are starting to flow in. Most people do well on the PSPO I assessment. However, only one person has passed the PSPO II assessment. His origins were business consulting. Everyone else answered the questions from the point of view of the development organizations. These people are some of the best Scrum Masters that I know, but they are entrenched in development thinking.

We have to pull our Scrum Masters back so they help the developers on the Scrum team use Scrum, and to help the Product Owners on the Scrum Team bring agility to the organization.

I am commonly asked, “How do I sell Scrum to senior management.” A good way to do so is to show them how they will benefit from agility, and the ability to take advantage of opportunities. If we simply tell them what they have to do for the development organization, they have much more pressing issues.