I finally met up with Dave Snowden this week and, aside from his sins for being a Cardiff Blues fan (even thought leaders have flaws in their thinking… there is only one team to follow in wales, the Ospreys), we share a few common interests, one of which is 360 degree feedback (other, non-professional, interests being Wales, Welsh rugby, Liverpool FC and a general Welsh penchant for speaking our mind)
Dave demonstrated how his SenseMaker software is being deployed as a sensory network for 360 degree feedback, where key individual/organisational traits are explored via story-telling narratives (provided by a broad cross-section of people who know the subject, based on varying degrees of professional intimacy, or coupling) and the positioning of ‘stones’ within a series of ‘triad’ descriptors. The output provides quantitative data on the subject(s) (the output of the positioning of stones within the various triads – see picture), which is then supported by the narrative provided by the stories (which are then subject to what I would describe as a form of meta-analysis). I need to preface this by saying that, in my opinion, what Dave is doing is extraordinary in the world of 360 degree feedback. The size of the sample pool (20+ in one of the examples Dave gave relating to individual feedback, which attempts to mitigate the risk associated with the law of small numbers), the intimacy diversity of the sample and the immediacy of the feedback (Dave is exploring continuous feedback over a 30 day period, as opposed to a single hit every 12 months) has the potential to reshape the HR performance assessment landscape and I for one am looking forward to being able to watch, and hopefully participate in/promote, the journey.
This said, my support is tempered by a generally healthy scepticism of software driven solutions for processes steeped in social biases. Feedback of the 360 degree type is biased by the complexity of context, in terms of subject, respondent and collective (organisational) worldviews. When analysing 360 feedback, taking a lesson from historians (see my blog on asking, why), we need to be aware of the context of the actions that are eliciting a response from the feedback provider. Raw numbers just don’t cut it at times, which is where the SenseMaker story telling (or narrative) element is interesting. However, stories, even collective ones, can be inherently biased and hindsight provides potentially toxic links to cause and effect (biases, such as the halo effect and availability bias). My argument would be that to get the true context there will be a need to go native (ethnography), a version of which Dave alluded to in our all too brief discussion. Without this ‘native’ context, we could misinterpret the output of the data. However, with that comes the problem of second order cybernetics and the impact of the observer upon their environment and vice-cersa, which, it could be argued that Sensemaker, and its quantitative measuring, attempts to overcome – almost reducing the level of agency afforded to the person within the system.
Using a European football example of what I mean by context (don’t ask me why, it just came to mind when writing this blog), a manager dealing with an under-performing team (high capability, not getting expected results and awaiting their regression to the mean (in a rut, losing games and reinforcing the adage that the hardest thing to break is a losing streak)) might employ a period of ballistic input to motivate that team/coaches/staff, whereas his/her ‘normal’ approach is calmly analytical. I would argue that any analysis that happens at this time, that doesn’t take account of the context, would provide feedback of questionable value. Dave pointed out that SenseMaker allows for the removal of periods of extreme disruption from the feedback, but I would argue that the bias, according to the impact of the incident/period of time upon the person giving feedback, would resonate and disrupt respondent feedback beyond the specific incident/time-frame in question.
In another example of the potential toxicity of data, what about the mix of styles that allows a team/organisation to become dynamic, agile and adaptive? Do we run the risk of providing feedback, altering behaviour, that actually serves to disrupt a fragile ecosystem, one that is often unrecognised by the members of that system? I wrote a blog recently on the need for ‘predators’ in organisations in order for progress to occur. I would argue that feedback, incorrectly interpreted by the organisation, not the SenseMaker software itself, could unbalance the organisational ecosystem, causing more damage than good. My argument here is that it doesn’t matter how good the software is, it will always come down to a person ‘correctly’ applying the outputs (data/information) to make the best decision for the individual/organisation. We, as consultants/advisors/software developers, can filter toxic noise, but in the end it is going to be down to people, and their inherent biases, to correctly interpret the data presented and make a decision (whether as an analyst or the end decision-maker).
Then there is the question of whether 360 degree feedback is truly 360 degree at all. Years ago, I read a book by Charles Handy where he spoke about his wife’s (Elizabeth) passion for creating portraits of individuals/spaces (see example), applying multiple snapshots of that individual/space, taken in contexts that the individual being photographed believed to best describe their whole person, or use of the space, to produce a picture that provides the gazer a broader insight into the person/space than the traditional individual mono-dimensional portrait. I would argue that, in order to understand the individual/organisation, there is a need to solicit similar information that is multidimensional, that informs us whether that individual/collective is capable of displaying the traits/competencies that we are looking for as an organisation/society. To do this we need to look at a 360 degree process that provides decision-makers with more holistic data/information.
Handy linked his wife’s portrait technique to the Johari Window concept, where it is believed that nobody can know the whole person, not even the person him/herself. The thinking behind the window being that a reflective mind will seek feedback from many sources to open the window and increase understand the person as a whole. I would argue that true 360 feedback needs to take a similar approach, in that way we, as analysts/decision-makers, can better interpret what is being presented to us and better support the individual/organisation/society that is sending us the signals – for example, when analysing a manager, it might be that s/he is not displaying traits that we desire to see in their work environment, but s/he is within her social environment; in which case it might be that we need to look at what is inhibiting that trait within the workplace (a tightly coupled relationship in a hierarchical structure, perhaps).
My other concerns are that feedback is ultimately based on questions asked (the output from the 360 process will only be as good as the questions asked), the warning that comes with taxonomies used to inform the feedback and the discourse analysis that is used to triangulate the quantitative outputs.
In fairness to Dave, and his SenseMaker tool, he takes a rigorous scientific approach to his methods, one unmatched in my experience in this field, and had theoretical grounding for every challenge that initially sprung to mind. Considering what is at stake here, I wonder if the same could be said for other software providers who profess to have developed solutions for 360 degree feedback and competency development? My experience tells me not.
I still believe there to be grey areas here and I am still not sure how this approach can help in managing natural human bias, but it is hard to argue against the fact that SenseMaker does seem to offer one giant step for HR decision-making.