Product Development: Divided we fall?

On the way home from ProductCamp Vancouver, we got discussing the increasing division of product development into smaller and smaller cubicles – each with a hyper specialized individual in it.

Back in 1979, I had my first professional employee - then consulting gig. Technically I reported to the head of IT. My customer was a department head managing a 30+ employee group. I meet directly with the department head, his reports and the clerks that would use the application (an interactive system using CICS running on a big IBM 4331 –2 megabytes of memory!!). I did UX mockups and reviewed them. Shadowed the clerks. Refine the requirements. Then I proceed to write the specifications. After that I proceed to design the database and implement (create a relational database using ISAM – not a trivial feat). Then coded up the application in COBOL and tested it (did QA too!). Then it was passed over to user acceptance. Finally, writing up documentation for users as well as design documentation. Everything was done interactively, often with a new component delivered every 2 weeks.

The process I followed would be known as agile today. That system kept in production until Y2K and had to be replaced because the hardware was not Y2K compliant. A year before that happened, I got a call from one of the folks there and found out that there has been almost no changes in the twenty years of operation and the existing staff was not happy with the replacement – the existing system was still completely satisfactory and had features that the new one lacked.

That pattern was my preferred one and I used it again and again for almost a decade (with nothing but very satisfied customers and robust systems) until I found myself contracting to Microsoft around 1990. At that point I found myself cutoff entirely from contact with the end user who was hidden behind a dev lead. Over the last 20 years I have seen the number of people involved grow and grow) and grow. With each new people-layer, communication issues deepen.

Today a project may well contain the following collection of people:

• Product Manager
• User Experience Specialist
• Graphic Designer
• CSS Specialist
• Accessibility Specialist
• Architect
• JavaScript Developer Engineer
• Web Site Developer Engineer
• Web Services Developer (for JSON etc)
• Middle Tier Developer Engineer
• Database Modeler
• Database Administrator
• Database Developer Engineer
• Software Developer Engineer / Test
• Testers (or Quality Assurance Specialist)
• Governance Specialist
• Security Specialist
• User Documentation Specialist
• UML Documentation Expert – developing maintenance documentation
• And of course, a Development Manager and a Test Manager

Each of the above can end up sitting in their own cubical. Worst yet, if something goes wrong because of a lack of holistic analysis (like poor scalability) – finding out the cause or who is accountable becomes a challenge. Personally, I have kept my finger in all of the above specialties and can swing into whatever role is needing assistance – which makes me an oddity. I read C.J.Date and I read Jakob Nielsen (and everything between).

My concern is whether the degree of “divide and conquer” happening is becoming a house of cards – I suspect that I may be the little boy in H.C.Anderson’s Kejserens nye Klæder (1837) (Emperor’s new Clothes – note Denmark did not have an Emperor, but a King) which was a reference to a certain emperor in Europe. His tale was a retelling of Libro de los ejemplos (1335) – illustrating Solomon sayin, there’s nothing new under the sun.

Although Agile is not new, I am concerned that the use of Sprints in SCRUM is encouraging/locking developers into their cubicles often. If development is constantly doing sprints, how can beneficial cross-training in all of the specialties above happen? Are developers being reduced to factory components? Is this hyper specialization a good thing or a bad thing in the long term? Locking developers (and other specialists) in their cubicles often produces short run efficiency – but does it sabotage long term competitive competencies?