Deming’s System of Profound Knowledge is a “manifesto” (he didn’t call it that), for using systems principles to understand organisations. His work mainly focussed on manufacturing organisations, and whilst his book Out of the Crisis does contain a section on Service Organisations, I wish there was more material from him on the subject.
Deming’s System of Profound Knowledge (SoPK) in a Nutshell
Deming advocated that Managers in an organisation should possess the following qualities
- Appreciation of the organisation as a system
- Knowledge of variation (the elimination of which is key to quality)
- Theory of knowledge (tools, techniques and principles to separate knowledge from information)
- Knowledge of psychology (human factors)
We will come back to these shortly.
Given the relative rarity of his work being applied to Service Organisations, and given the era in which most of his work was written, it comes as no surprise to me that I have never found any material by Deming on one pervasive aspect of modern organisation design, the IT Department. If you know different please let me know, I would be fascinated by what he had to say!
So in the absence of, or my ignorance of anything from the man himself, I have decided to take Deming’s System of Profound Knowledge, and apply it to my experiences of the IT Departments that I have worked within. Dr Deming, meet the IT Crowd!
A Look at a “Typical” IT Department
First of all, whilst this is a generalisation, in my mental model, an IT Department typically falls into one of two types.
- The one in the basement, portrayed in the C4 sitcom series the IT Crowd, staffed by misfits and largely invisible to the rest of the organisation
- The monolithic corporate entity
The former is an increasingly rare phenomenon, so I am going to focus on the latter, but this in turn is going to point us back in the direction of the former.
SoPK #1 – An Appreciation of the Organisation as a System - What Does a “Typical” Corporate IT Function Look Like?
It may not even call itself IT. It may have a name that alludes to Change, Technology, Solutions, or it may just call itself IT. Whatever it’s name there will predictably be some common properties;
- It will be big, in fact, very big
- It will be the area of the organisation that carries the responsibility of delivering IT led projects – despite the benefits case belonging to the consumers of the IT
- It will be designed as a top down functional hierarchy
- It may comprise outsourced functions
- It will be process driven
- It will be standards driven
- It will contain a wide variety of functional disciplines including but not limited to
- Portfolio, Programme and Project Management
- BAU Service Management
- Network, Infrastructure and Hardware Engineering
- Software Engineering, Testing and Design (hopefully not in that order!)
- Strategic Planning, both Corporate and Technical
- Rightly or wrongly, it will be perceived by the rest of the organisation to possess a combination of any or all of the following qualities;
A highly simplified and incomplete systems view of an IT Department would look something like this;
However, most IT Departments see themselves as something like this;
SoPK#1 Score: Nil Pois!
SoPK #2 – Knowledge of Variation – How Well Does a Typical IT Department Understand Variation Over Time?
IT Departments are systems that contain enormous levels of variation. They have to respond to things going wrong, things needing to be changed – in response to internal or external drivers, information needing to be provided, the list is endless. Just one type, of one type, of the things than an IT Department needs to do is to make changes to software and its associated infrastructure, otherwise known as Enterprise Applications.
An Enterprise Application is something that an organisation uses to run it’s business. Examples are Word Processors, Spreadsheets, CRM and Call Handling systems etc. The distinction exists to differentiate software engineering for organisations, from software engineering for other purposes, for example embedded systems that are used to control washing machines, or fly-by-wire aircraft.
What does a typical Enterprise Application Project Look like in a monolithic IT Department?
Walk the offices of a large corporate IT department delivering Enterprise Application Projects and you will see people sat at desks working on computers. You may peek into a meeting room and see a few people in suits pointing at slides being projected onto a wall, or writing on a whiteboard. Largely however, that is the extent of the physical activity that you will see.
So what is actually going on? Amongst all the desks, computers, meeting rooms, slide projectors there is certainly a heck of a lot of activity going on;
- Project/Programme Management Offices (PMOs) exchanging information about Programmes and Projects
- Project Managers planning, updating their logs, chasing things up and escalating issues to other Project Managers or to the PMO
- Programme Managers making sure Project Managers are doing the above
- Architects keeping the whole thing together
- Designers designing, reviewing, re-designing, clarifying, consulting
- Development Managers liaising with Project Managers and Test Managers, managing resources
- Test Managers managing Testers, reviewing Test Plans and Test Cases, and updating Project Managers on test progress
- Testers writing Test Plans, reviewing Test Plans, testing software
- Configuration Managers managing source code and binaries
- Scum Masters managing daily scrums and liaising with Project Managers, Test Managers and Product Managers
- Software Engineers working with source code to develop the enterprise application
The above is an oversimplification, but it shows just how much scope for variation there is in the system. How much of this variation is understood by the organisation and it’s management? How many statistical control charts v.s. balanced score cards, dashboards et al did you see when we walked the floor a moment ago?
SoPK#2 Score: Nil Pois!
SoPK #3 – Theory of Knowledge – Let’s Study the System and Find the Value Work
In our Enterprise Application Development Project, where is the value work (i.e the stuff that actually delivers the service)?
- Designers Designing?
- Software Engineers working with source code to develop the enterprise application?
They are doing the value work, and you can’t see it because in the main it is happening in their heads.
I am struggling to find anything else in the list from our floor walk that is truly value work.
All of the activity we saw may be necessary for the the organisation to facilitate the software engineering activity, but none of it apart from design and coding actually contribute directly to the finished product. It is all waste. For example, just as Deming stated that you cannot inspect quality into a product, you cannot test quality into software (the use of modern test driven tools and techniques constitute part of the build process). In both cases, the quality is either there or it is not. It may be necessary waste but it is still waste.
How well do most people who’s job falls into this category respond to statements like the above? Understandably not very well.
How well is waste recognised, and how well do organisations act to identify the causes of its necessity and to remove those causes? Not well at all.
SoPK#3 Score: Nil Pois!
SoPK#4 Understanding of Psychology
If you have not heard Dan Pink’s talk on Intrinsic Motivation I would urge you to have a listen, however, this is by no means new thinking – despite outward appearances and a highly scientific approach to his work, Deming had a huge affinity with human beings in the workplace, and his work showed great similarities to Pink’s words decades ago;
All anyone asks for is a chance to work with pride. W Edwards Deming
So who are the people we should really pay attention to in our Enterprise Application development organisation? The people who do the value work, Software Engineers (including designers).
Software Engineering is a process of problem solving. You cant solve a problem by breaking it into pieces, giving each piece to a different person, then expecting the subsequent assembled parts to be cohesive.
So why do we reduce the role of the Software Engineer to be the recipient of a work package, to “code” some changes and to package them up and pass them up the assembly line?
What do we understand about what motivates Software Engineers ? What do we understand about what they find fulfilling? What do we understand about their decision making processes?
RAD, XP, Agile et al have shown the huge gains in productivity, quality and engagement that can be made from allowing talented people to use their talents, but the large corporate IT department thinks this is something you can just install.
SoPK #4 Score: Nil Pois!
Software Engineers have for too long been considered the to be the “IT Crowd” – they have been turned into a commodity that you can offshore, onshore or nearshore, distanced from the work, and placed at the bottom of the organisational hierarchy. In the meantime, decisions about how to manage the work and how to do the work have been taken by those even further from the work, based on the advice of consultants, vendors, analysts, beliefs, assumptions and superstition.
Maybe the IT Crowd Know Better?
I think there is something to be said for allowing the people who actually do the value work to decide the best way to do it. Is it too much to ask to let a Software Engineer see the product of their work in use? They should have a crack at Deming’s System of Profound Knowledge, after all, traditional attempts to make the IT Department effective seem to have yielded a resounding nil pois. I reckon they would pick it up in no time, if they don’t know it already.
There is still a role for the rest of us; work to improve the system, help unpick the causes of variation, or step out of the way.
Innovation comes from people who take joy in their work. W Edwards Deming.