Ideas for the measurement of Enterprise 2.0 effects

Juni 2, 2009 - Allgemein - 55 comments

In an internal discussion of our Advisory Board regarding the topics of the E20 SUMMIT program Joachim Niemeier posed the question whether classical economic measures are suitable to measure and verify Enterprise 2.0 effects. Though cases like the TransUnion project (cited by in his blog post on determining the ROI [cross-referencing to the post of Ross Mayfield]) give a proof of evidence for measuring the E20 effects with the classical ROI formula – many projects have difficulties in measuring it. As from the discussions about this problem at E20FORUM the main difficulties in measuring are perceived as the difficulties in identifying and quantifying the “net profit” from Enterprise 2.0 activities – as the nature of effects is mainly soft and intangible. As this is planned to be a major discussion at E20SUMMIT I would like to sort my ideas on this to stir up the discussions upfront.

In regards to a distinguished discussion I’d like to differentiate the notion of Enterprise 2.0 into two dimensions of impacts. In a recent German article for the T3N Magazin and I have discussed (as already a lot of other people before) the characteristics of Web 2.0 apps and social software in comparison to classical “information management systems”. Besides mentioning the leveraging characteristics of being mainly open-source concepts with light-weighted architecture and a “simplified user experience for the masses” we focussed on Tim O’Reilly’s main point: the supplement of the social dimension and the network effects. This lead us to the point that the usage of Web 2.0 tools within the enterprise (as the notion of Enterprise 2.0) results into a more transparent and outward focussed information gathering that itself implies again a more transparent and effective collaboration. We quoted at this point who is talking always about the change from a world of “content objects & processes” towards world of “feeds & flows”. So in regards to the ROI discussion we have therefore to distinguish the impact on information management & distribution within the enterprise from the impacts on collaboration.

At this point I would like to focus on the part of “information management & distribution” as this is a precondition for the impacts on collaboration and also seen as the more difficult part to measure.

Measuring the value of information management & distribution

For the further discussion I’d like to equate the term of “information distribution” to the technical dimension of “communications” – leaving out the social, emotional and intential aspects of communications. Why am I doing this? Because the “communications” discipline within the enterprise has a long time existant valuation problem which is recently addressed by “value based” approaches (see a explanation on “value based management”) that might be very suitable towards the valuation of Enterprise 2.0 effects.

At least within the German speaking PR community there is a controlling (or better KPI) model – based on the ideas of Walter K. Lindenmann and Norton/Kaplan. The latter have discussed a “strategy map“, a concept that illustrates the causes and effects on the building of business values. “The strategy map links the long-term game plan or competitive strategy of a business with its operational activities.”

In referring to this strategy map and its different “perspectives” Walter K. Lindenmann has developped a three-level KPI model for the valuation of PR effects, that constitutes the foundation of valuation for the IPR toolkit (IPR = Institute of Public Relations) as well as the German PR association (GPRA / more German background papers at CommunicationControlling.de). The model consists of three interdependent levels of impact:

  1. output level: e.g. questions whether the message has reached its target group
  2. outgrowth level: e.g. questions whether the message has been perceived and understood
  3. outcome level: e.g. questions whether the message has changed the knowledge, opinion or behavior of the target group

As this model is not including any reference to the financial dimension of the strategy map Lothar Rolke added in article in 1997 the term “outflow” as a forth level that questions the business effects. While the GPRA is nowadays not differentiating between level 2 & 3 and therefore proclaims only a three-level model (output / outgrowth & outcome / outflow), for a further discussion on how Enterprise 2.0 is effecting the business value a differentiated four-level model would be more suitable.

The connecting pieces of these levels are the “value links” that constitute a causes and effects diagram like this example for the “value links” in communications by :

I won’t translate this model as it is only a fictional examples of a value link system. Actually the causes and effects differ from industry to industry and from company to company – as the value generation differs in all kinds.

I first heard about the practice of this model in a non-PR context from the people of aexea at our Swiss Intranet Management FORUM about Intranet governance & controlling. They are using it to evaluate the effects of an Intranet project which in return resemble the targets of Enterprise 2.0 projects but mainly driven from a centralized viewpoint of organizing it.

As from the common practice of working with this model they have added a input level to the diagram that discribes the denominator of the classical ROI formula in terms of costs for the Intranet management. “Output” describes the produced content by the input – in quantity, frequency, reach & actuality, comprehensability & usability. “Outgrowth” explains the perceived messages from Intranet output – measured for example by the knowledge about the contents of the distributed messages/information. “Outcome” indicates the effects from the “outgrowth” in regards to the changed behavior in terms of participation in any Intranet services. At the top of the model “outflow” pictures the business effects of the changes in behavior.

Evaluation approach towards Enterprise 2.0 activities

Returning to the starting point I think this model is a very good approach in describing the effects of the impacts of E20 activities on the corporate “information managenet & distribution”. In contrast towards the classical ROI formula the model takes into account the “soft effects” of the activities.

In the following I put together some potential key indicators on the (for me) four main E20 activities: tagging, blogging, “wiki”-ing and “social network”-ing:

I think this little practice shows quite nicely that using a more differentiated approach to the valuation of Enterprise 2.0 is very useful as it illustrates the achievements made in an earlier stage.