Can Software Developers Meet the Need?

Marc Andreessen provided some insights into the importance of software and the software profession’s ability to meet the need at “Why Software Is Eating The World” , http://goo.gl/ob2Cvx

Scrum facilitates control through frequent, regular inspection and adaptation of transparent software functionality. Transparency means the software is ready. It can either be immediately deployed or built upon without regression. It has no technical debt. Transparency mandates modern engineering practices and tools, and application of enlightened value-driven management.

I’ve found that most software developers do not have these skills. For example, the concept of building code from requirements and specifications that are then used as tests is incomprehensible to many.

Our  shortcomings were surprising to me. When I rolled out Scrum, I thought that the excellent developers that had been stifled by waterfall processes would emerge, and we would again do great work and build great software. Much to my surprise, many never had those skills or had lost any skills they had. The pressure of “get it all out now, cut quality to do so” had reduced most developers to unskilled, unprofessional, angry drones. The larger the organization, the worse the situation as “mediocrity scaled.” Organizations whose software is only five years old are already crippled by technical debt.

Our society, infrastructure, organizations, and all products rely heavily on software. It is the last scalable resource, and everyone is flocking to use and make money from software and software enabled products. My experience helping organizations improve the value they can generate from software leads me to conclude:

A software profession governing body is needed. We need to formalize and regulate the skills, techniques, and practices needed to build different types of software capabilities. On one side, there is the danger of squeezing the creativity out of software development  by unknowledgeable bureaucrats. On the other side is the danger of the increasingly vital software our society relies on failing critically.

We can either create such a governance capability, or the governments will legislate it after a particularly disastrous failure. I am not happy proposing this, hoping that skills would emerge faster than need. However, the gap is going the other way, with needs outpacing emerging skills.

If I were on a team, I wish that I knew the level of the developer that I was bringing on board, instead relying on personal references and “the usual suspects” to avoid getting swamped by unfounded braggarts.

My prediction is that if we don’t do something, the government will. The impetus will be the disaster that could have been avoided by better software quality. A government software-regulation bureaucracy would be an equal disaster.

32 thoughts on “Can Software Developers Meet the Need?

  1. Pingback: Can Software Developers Meet the Need? | M Anthony Librizzi

  2. Not sure that the government will, but rather the people will, or the market will. Taking the scary news about the Heartbleed bug, are we getting any closer to people realizing how much of their lives is ‘riding’ on software and how many developers are ‘forced’ to build crappy software that just barely works? And imho mostly due to a totally inadequate view of business and management folk of how to build software?

    I can only hope people will wake up sooner rather than later.

  3. Pingback: Professional Software Development – Can We Mandate What We Can’t Define? | Form Follows Function

  4. I’ve long lamented the inability of programmers to interact with their users. There are a small minority that can do it. They have been discouraged by management to do this but programmers have not done themselves any favours by not developing the people side of their jobs. This is the people side. I’ve also seen lots of crappy code that developers are happy to leave as is – no craftmanship ethic. Managers are incapable of discerning quality and allow this to carry on. Maybe this governing body could be good. The better programmers can deliver faster and better then a horde of incapable or unwilling to grow programmers.

  5. Oh dear. I can’t quite put my finger on it, but this post makes me sad.

    So you’ve concluded that its all the fault of poor developers and that the solution is they need governance and regulation.

    You’re painting this bizarre view of the world where everyone else is business is ‘doing it right’ whereas developers are weak and feeble creatures, somehow deskilled and impotent, lacking the ability or energy to fight for and produce quality work.

    As a generalist, developer (and I DO have the skills you talk about) and in more recent years a Product Manager I would suggest you look a little closer to home.

    You’re a Product Manager right? You are there to represent the interests of the customer right? Then who’s letting this sub-standard work out of the door? Who’s producing the business cases that lead to the corner-cutting you describe? Who is creating the product expectations and conditions which lead to poor software? Who cares so little about their customers that mediocrity is the standard you accept?

    Blaming it on developers is a very poorly thought out argument and of course is guilty of sub-optimization.

    Regulate Product Managers, governance for marketeers.. why aren’t these equally commendable?

    Of course producing better software should be the aspiration. But how do you make the leap to regulation of one aspect of product development as the solution?

  6. The old form of QA was that QAS was responsible for ensuring that only quality work got out the door. Professionals, craftsmen, do great work that can be relied on. With software, it has to be relied on for years, not only in its singularity but in conjunction with all the interfacing facilities.
    As developers, we shouldn’t need to have our work checked. We should and can do great work.

  7. Who wouldn’t agree with that.. however your comments on governance and regulation imply that ‘we can’t and we don’t’.. and worse still it implies there is some development complicity in declining standards!

    Ignoring the fact there will always be good and less good craftsmen in any trade, ‘good’ or ‘quality’ can only be defined in relation the economics driving it. Modern business is primarily focused on the short term, is highly subject to the influence of the markets and is driven by a disposable-oriented society.

    The realities of product development initiatives are that they have to deliver into this environment.

    ‘As developers we shouldn’t need to have our work checked’…

    But you’re in favour of regulation and governance of software development.. Which by any definition is going to require ‘checking’ our work.. just how and by who and by what criteria?…

    Surely there’s a commercial/social argument for good quality software (although perhaps not fully realised yet). And if there is then demand will instigate supply.

  8. Pingback: Can Software Developers Meet the Need? - The Agile Product Report

  9. The problem won’t be solved by government regulation or oversight, it will be solved by the market. Government involvement will only ensure that every software project turns into healthcare.gov. Companies are starting to figure it out, but it takes time… time to change attitudes. Time to recognize that the conventional wisdom is wrong, and that what your organization is failing with now will not get it to where it needs to be in the future.

    The problem isn’t isolated to the software developers, it’s endemic to the culture that tolerates low quality because low quality is the best the organization thinks it can get. The problem isn’t a lack of knowledge of engineering practices, it’s a lack of management and leadership. When management has no idea of the capacity for delivering work over time from its engineering organization and yet insists on setting arbitrary goals that have no basis in reality, when management views engineers as fungible so outsourcing is an attractive way to pare down costs and yet looks only at the per-hour labor rate instead of the per-delivered functionality labor rate, when employees are judged by how many hours they work on something instead of how many deliverables they completed over time, when engineering managers and engineers get unrealistic commitments yet do not speak up… this is where the problem lies.

    The solution is simple… but not easy. We have to understand that EVERY software team has the ability to deliver quality software. The unknown is, how quickly can they deliver quality software… or how much quality software can they deliver over time? Deming showed us the way, Scrum gives us an easy way to apply the Deming Cycle… yet how many so-called ‘Scrum’ teams actually use the retrospective? Too many people focus on improving the product via feedback, too few focus on improving the process via feedback, and very few understand how Scrum is instrumented to deliver feedback on the process.

    Four simple steps: get a process in place, and measure the rate of delivery using that process, identify and eliminate impediments to flow and experiment with process changes to prevent similar impediments in the future, use the rate of delivery as a gauge to tell if you’re improving by determining if the rate of delivery is increasing, rinse and repeat.

    And, leave the government out.

      • I wouldn’t worry too much about govn/central control.. they’d have to actually define exactly what and how etc…never going to happen.. the /real/ issue is that there are probably certain types of *products* that require some regulation, and therefore qualification to develop them.. but not people per-se.. just imagine how ridiculously unmanageable that would be. And imagine which country is going to be brave enough to put those controls in first.. ha ha.. as if.

  10. How about we empower the team to “push back” against the demands of the business to emphasize quality over speed? As team members, a ScrumMaster, or a Product Owner the onus is on all of us to not compromise on quality. Without the emphasis on quality, processes such as Scrum will fail.

  11. Teams are made of individuals, individuals are humans. humans are not perfect.
    In every profession, you can find skilled or poor-skilled people, even in surgery-hospitals or in airplane cockpit. We have to deal with it, we can’t expected everybody to be highly skilled professional.

  12. In the UK, a Profession (capitalised) is a trade with a professional membership organisation. I can’t call myself an Architect (though I can design buildings) unless I satisfy RIBA’s membership criteria, which in turn call for a certain level of academic and professional development. Likewise Doctors, Engineers, Accountants, Surveyors and so on.

    Generally professional training and certification takes about 7 years, a remnant of the old Guilds. If it takes about 10 years to become expert in a field, 7 years to become an independent Professional seems reasonable.

    Without such a governing body, professional programmers will remain tradesmen/craftsmen/semi-skilled labour. And just like plumbers, it’s hard for clients to know if they’re hiring the real deal or a cowboy.

    Professional status might enable entry-level coders to practice (just like entry-level building-designers) while affording Chartered Programmers a degree of clout that we’ve never enjoyed.

  13. Pingback: Professional/Certified Scrum Developer – Lamentable Takeup | Nikola Zdunić

  14. Pingback: Expectations of Software Engineering | Mind & Logic

  15. I would like to share my experience regardless to this great post. I Go to the office earlier than I’m supposed to , thrilled by the challenge and the task that I have at hand, with the honest intention and a great energy to build the best peace of software at my best intellectual capacities by applying best engineering practices and theories which I have learned at the engineering school such as UML, TDD, Refcatoring, Design Patterns etc. and most importantly to work with my peers to get this masterpiece DONE, until I hit the reality. Peers are not welling to collaborate ( lack of people skills ) or does not have the skills to challenge that masterpiece and improve it. Technical leads for job security reasons will disqualify ideas that you can come up with just to protect their job in the hierarchy, or because their huge ego is hurt therefore they abuse their position power, or they don’t show appreciation to your work. Clients does not appreciate the value of the hard work and the intellectual effort put in place to build that master peace, Managers will scream at you or talk on your back because of your poor performance ( we hired a highly graduated developer but you know what ? it took him 4 days to create that whereas X , who is less graduated did it in 2 hours and my response was of course . he cut corners ..) not matter how positive you are, energetic with strong leadership abilities, down the road you will be depressed if nobody in your team share the same believes as you and you end up either by doing crappy job or in my case leave the company. Thus, I totally agree with Jhon post. The problem is the culture. We need to change the culture, teach managers about quality, developers about modern engineering practices, open their vision to consider long term goals ( product life cycle) rather than short term ones( go live and move on ) , the impact of bad technical choices on the cost of the product and the ability to innovate. Put in place the right technical leadership to drive technical excellence and teach managers about the importance of delivering less with high quality versus delivering more with crappy code. I strongly believe that if we drive that change from the top ( technical lead, architects and up ), developers no matter how not skilled they are they will follow the lead, improve and became more productive.
    Looking forward to hearing your thoughts.
    The Best
    Bassem

    • IIRC the original post was claiming the necessity of some form of regulation/governance/qualification of developers to counter poor practice.

      What you have just described is a dis-functional organization.

      Whilst I agree with your points, I don’t see the two as fundamentally related.

      Crap organizations atrophy and fail to make commercial sense, and ultimately lose out to the competition. The law of the jungle weeds them out.. and even if it doesn’t who cares? Just leave and go and work for a better organization that shares your values.

      There will always be good companies and crap companies. Regulatory bodies (and especially profit oriented ones) aren’t going to solve that problem ad more than a two day Scrum course is.

  16. Scrum is great way to develop product roadmap. Software Developers must have transparency with value driven management.
    Agile is one of the best method for to manage ALM, SDLC.

  17. Pingback: Who is the Professional Scrum Developer? - Scrum.org Community Blog

  18. You really make it seem so easy with your presentation but I find this topic to be actually something which I think I would never understand. It seems too complex and extremely broad for me. I’m looking forward for your next post, I’ll try to get the hang of it!

  19. Pingback: Who are the Professional Scrum Developers? – Ryu Amy

Leave a reply to kenschwaber Cancel reply