Categories
M&E

Game-changer Strategic Mapping Tool for Networks?

A proto-type of a new mapping tool called The Strategy Landscape™ (SL) is available for viewing.  “It has the potential to be a real game-changer for the field,” explains John Branam, director of programs at Grantmakers for Education.

SL promises to help networks working in “issue arenas” (corruption, forestry, water, health, etc.) develop more effective strategies and efficient application of resources.  It visually presents information about organizations’ strategies, and financial resources being applied to support them.  By gathering these two types of information from organizations working in an issue arena, along with other information like geography they are working in, relationships between numerous organizations can be better understood.  This will help identify duplication, important gaps, opportunities for working together and potential synergies.

SL is an online interactive visualization tool, developed by Monitor Institute with support from the Rockefeller Foundation as part of their initiative to accelerate innovative practices in the social sector.  Its development has been piloted with a group of climate change funders, and you can play with the tool to see how easily it generates pictures about relationships.

Monitor’s Gabriel Kasper says that the tool was originally developed for private funders.  But they recognize that it can be helpful for many others…and they are looking for pilots that are not categorically in the philanthropy space as well as with fundes.

I remember talking with bi- and multi-lateral funders in Madagascar about their strategies and was astonished to learn there was so little collaboration or even knowledge about what each other was doing.  With SL, that could be addressed.

Multi-stakeholder networks could find the tool useful by gathering similar data from their members and others working in their issue arena.  Perhaps the best pilot for them would be in a specific geographic area, such as a country or collection of them.

Gabriel’s team first collected financial data and created a list of strategic categories.  Of course different funders used different ways to describe their program strategies, so part of the value of the exercise…and also some of the touchy part…is to create a collective understanding of the set of strategies in an issue arena.  For the climate change funders, the 16 strategies include:

1.     Smart growth and transportation

2.     Agriculture

3.     Coal-related emissions

4.     Energy efficiency

5.     Renewable energy

6.     Adaptation and resilience

7.     Policy change

“We don’t think it’s perfect,” says Gabriel about the maps the SL generates for the climate change fundrs.  “But we don’t want the perfect to be the enemy of the good.  And if we get it 80-90 percent right, that’s more than they’ve got now.  And we will tweek it as we go.”

Gathering the data to generate the initial maps was quite a task.  Gabriel estimates the cost was about $150,000.  But keeping the data-base up-to-date will be a relatively simple task, now the categories and reporting frameworks are defined.

Of course one of the weaknesses is that financial expenditures are only one indicator of energy applied to a specific strategy, and this could be further refined by collecting other data like person days spend on a strategy.  But financial is probably a pretty good proxy for amount of effort.

Personally I find one of the biggest attractions of this approach is that it helps people think systemically.  And if the data-base is kept up-to-date, it will be such an attractive source of information that organizations operating in an issue arena would undoubtedly reference it when doing their own strategic planning.  This in turn would create a much more coherent collective strategic planning space, hopefully creating a virtuous cycle of ever-improving information as this “strategy commons” becomes better understood and organizations become more willing to provide information for its development.

Categories
M&E

A New Approach to Multi-Stakeholder Network Assessment

Assessing the effectiveness of networks is a notoriously under-developed field.  I recently took my hand at it with the co-leadership of Horacio Trujillo and iScale.  We applied our thinking to the International Land Coalition, a multi-stakeholder network aiming to promote secure and equitable access to, and control over, land through advocacy, dialogue, knowledge sharing and capacity building.  The major ILC participants are NGOs and intergovernmental organizations (IGOs) like the World Bank and the Food and Agriculture Organization (FAO).

We emphasized that our work was an “assessment” of the last strategic plan period (2007-2011), rather than an “evaluation”.  The latter term is associated with baselines and quantitative analysis.  (Usually evaluators draw from organization-based methodologies that are highly problematic and even undermine the work of multi-stakeholder change networks.) 

We didn’t have baseline data, and ILC wanted us to focus on questions about how it is doing as a whole network.  The product is being shared with funders as a report, and integrated into ILC’s new strategic plan. 

We did some web crawl mapping, interviewed a lot of people, looked at lots of documents, did a focus group in Nairobi and with the ILC Council in Rome, and developed four case studies – one case each was at the local, national, continental and global levels.

Gathering data is, of course, lots of work.  But the really hard part of doing an assessment is to make sure you are asking the right questions, using the right lens to look at the data, and summarizing everything in a comprehensible and comprehensive actionable way.  We used two frameworks to organize the data, raise questions, and create a synthesis. 

The GAN Framework

We used Global Action Networks (GANs) as a framework because we thought the ILC was a GAN, as did some people at ILC.  So the big GAN question was:  as a GAN, how is ILC doing?  We used two tools to answer this, presented in Tables 1 and 2.  For each we asked:  What has ILC achieved?  What does it need to do better?

For example, with “Shared Visioning” of Table 2, achievements of ILC included:

  • Broad agreement about the major roles of ILC: visioning, system organizing, learning, advocacy (reassuring some who feared other participants viewed “financing” as an ILC role)
  • ILC’s vision is now reflected in two major international documents recently adopted by other organizations. 

On the other hand, in terms of doing better the data indicated that:

  • ILC should shift from a focus on “visioning” to “implementing”. 

The Theory of Change Framework

Theory

of

Change

Framework

The second framework was “theory of change” (ToC).  Horacio looked at ILC’s strategic objectives (SOs) and mission, and reframed them as a ToC represented in Diagram 1. This Diagram reflects that ILC’s mission around access to land is, in fact, a strategy to reduce hunger.  The preceding SOs explain how ILC says it is going to realize these outcomes. 

We then took the data from the GAN framework and applied it to the ToC to “rate” ILC’s performance using a “stoplight” approach where:

  • Green represents those items that we assess as having been realized to a significant extent,
  • Yellow represents those we assess as realized to a lesser, but still meaningful degree,
  • Orange represents those we assess as realized to some, but less meaningful extent, and
  • Red represents those objectives we assess as not having been realized or having been realized to an unsatisfactory degree.

From all of this, we then proposed a refinement of the ILC’s ToC, represented in Figure 2. In this refined version there are the following important changes:

  1. SO1:  The dotted lines aim to emphasize that this is a utilitarian, organizational development objective of different status.
  2. SO2:  This civil society-focused objective is greyed-out and integrated into the next SO, reflecting our suggestion to increase focus on developing the multi-stakeholder aspect of ILC and supporting more specifically CSOs’ (and others’) capacity to engage in multi-stakeholder processes.
  3. SO3: We replace “members” with “participants”, reflecting the need to broaden the ILC’s stakeholder focus, but without getting into formal governance issues.
  4. SO4:  We explicitly add “other stakeholders” to reflect our recommendation that the ILC broaden its stakeholder focus.
  5. SO5:  We:
  • replace the terms “coherent and coordinated” with “catalyzing of coherent” to better reflect our view that (1) an active verb is preferable, (2) a broader range of actions than coordination are necessary (for example, developing), (3) the core function of ILC is to create “coherence” amongst stakeholders.
  • identify the path of influence that global commitments can have on catalyzing national commitments, by breaking these apart.

All of this is contained in two reports.  The first report contained the full GAN analysis and a couple of dozen recommendations.  The second report, for broader consumption, omitted the GAN analysis and focused on the two Figures and four Summary Recommendations that ILC should:

  1. Focus more rigorously on strengthening the multi-stakeholder objective of its strategy.
  2. Recognize in its priorities an opportunity to shift from creating frameworks to the implementation of such networks.
  3. Be more disciplined and strategic about prioritizing, setting targets and allocating resources. 
  4. Consider reframing its strategic objectives as a more concise theory of change.

For Horacio and me this project was, therefore, about producing a useful assessment for ILC and also about producing a methodology for conducting such assessments.  We believe that it would definitely be useful for other GANs, and adaptable to other networks as well to provide useful strategic direction.

Thanks to others on the iScale scaling for impact team who contributed to this work:  Randy Kemp, Sanjeev Khagram and Saira Abbasey!

ILC Director’s Comment

I asked International Land Coalition Director Madiodio Niasse if he’d like to comment on our assessment, and he shared the following:

The review of ILC’s 2007-2011 by iScale has generated results far beyond our expectations. We started by anticipating an ordinary evaluation in which projected results and actual achievements would be compared and analysed. We ended up engaging in a radical collective reflection on who we are, what brings together, what is our comparative advantage, and how to position ILC in the future. A constructive tension between continuity and innovation has emerged from the internal consultations. And I am sure that ILC is today more ready for change than ever before, although it will take us time to clearly articulate this in a language easily understandable by our members and partners. For these reasons I think the process facilitated by iScale is as important as its end product, the review report. Thank you.

Categories
M&E

Network Health as “Harmonic Vibrancy” : Analyze your network

Most multi-stakeholder networks are taking on pretty daunting challenges:  climate change, corruption, sustainability, health, water, food security…  What is critical to their success?  A new perspective on this, accompanied by a very simply administered test, is offered with the concept of “harmonic vibrancy” (HV).

“…harmony refers to the degree of agreeable feeling or accord in how a group’s agreements fit together,” explains Jim Ritchie-Dunham in the draft of a new book on the topic of HV.  “People often describe the value they experience in the five primary relationships (self, other, group, nature, spirit) with some form of vibrancy, meaning the vitality in flourishing relationships.”

Jim is very familiar with Global Action Networks, having participated in numerous meetings with them and followed my work closely.  In fact, in his book he applies HV to GANs.  I’ve worked with him at the Institute for Strategic Clarity.  He comes from the “decision-making” sciences that ask questions about how choices are made to realize effective action.  His HV analysis is very relevant to networks, since it focuses on “relationships” and “agreements” to understand their “health” with a relatively easy analysis.

“When the vibrancy in the group is low,” he writes, “I just want to be told what to do.  It is clear that none of me is needed, other than what I can do right then.  It is very frustrating.  I am in a box, somebody has a whip, and I am submissive to the task at hand.  When I experience high vibrancy in a group, we are all looking for the unique contributions our creativity can make to the group.  It’s like the harmonic we create when we each sing our piece.  Together we are able to take on anything.”

HV seems particularly useful for networks because it is all about relationships.  It orients networks to questions about how to realize a world of “abundance” for a high vibrancy (very healthy and effective) network.  One innovation of the HV perspective is that it does this by integrating five relationship aspects of “health” that are important for a network’s success: 

  1. The “self” or “me”:  do you feel that you are fully participating and working to your potential?
  2. The “other” or “you”:  are others participating fully and expressing their potential?
  3. The “group” or “us” which can mean the network as a whole:  are people fully experiencing the “we” as an energetic, empowering whole?
  4. Nature and the “environment”:  is there a feeling of “support” and “appreciation” from the greater whole that supports manifesting the potential?
  5. Spirit and creativity:  is there a flowing and development of ideas and innovation that generate a feeling that “anything is possible”?

All these high-brow ideas are intellectually interesting, but what is particularly powerful about Jim’s work is that he has developed a very easy analytical tool – a survey – for people to understand how their network (or other group/organization) is doing from these five perspectives.  He’s applied it with several organizations.  And he’s also developed prescriptions about actions to raise the level of vibrancy.   These revolve around the core concept of “agreements” – informal and formal – that define “relationships”.  Formal ones include written contracts and prices negotiated at a market; informal ones include assumptions about “the way things work”:

“There are also many times when people tend not to be conscious of the agreements.  We drive on a specific side of the road, because it is an agreement to do so.  We also agree that you can be arrested for not complying with this agreement.  This might make it seem like a law: that’s just the way it is.  Except, that the law does not always apply.  In the countryside or on a farm, you might simply see that you are driving on the road.  There is not really a left or right lane.  In other words, it depends.  It depends on the situation whether the “law” applies, thus it is an agreement.”

These agreements both support and limit action, and reconfiguring them can enhance HV.  Jim presents tools that help make agreements visible, so people can discuss and reconfigure them to enhance HV.

Try the first step, by completing a survey to find out more about the HV level of your organization or networks.  Learn more at the Institute’s web-site.

Categories
M&E

Assess Impact in Complex Environments

A big challenge people are always asking me about is how to measure impact when many organizations and people are involved and when success depends on flexibly changing course in response to learning what works.  They are characteristics of complex environments.  Attribution is highly problematic with so many actors and measurement against baselines is often simply not useful when the learning actually leads to redefining goals and true innovation.  The International Development Research Centre in Canada has developed Outcome Mapping;  Michael Quinn Patton frames the appropriate response as “developmental evaluation” (more information).  Danny Burns draws from his considerable experience as an Institute for Development Studies action researcher in such situations in a valuable paper:  Burns- Assessing Impact in Dynamic and Complex Environments Systemic Action Research and Participatory Systemic Inquiry.

He starts by emphasizing a systems approach, and that in complex environments “changing system dynamics is critical to sustainable impact.” He notably writes about “assessment” rather than “measurement”:  the latter concept is almost equated with quantitative measures that can have very differ

14-09-24 Burns 1 - Assessing Impact

ent meaning to different stakeholders and on their own, can be largely meaningless.  “Assessment” is much more about understanding context and learning, which are critical ingredient in complex environments.  

Burns modifies the famous learning cycle and nicely addresses a question I’ve always had about the usefulness of approaching complex issues with theories of change when learning from action can make the theory inappropriate:  he integrates generation of new theories of change into the learning cycle.  Duh.  Seems obvious now!

Another challenge working in complex environments is the non-linear nature of change that makes traditional planning process designed around moving from point A to point B inappropriate.  He looks at four aspects of complexity and systems that have particular impact on assessment:

Change is emergent:  Figure 2 illustrates the original belief that A leads to B, which leads to C to get to goal 1.  When learning early on suggests that goal 2 is a more desirable goal and it’s best approached by going from A to K, traditional planning would break down.  Emergence requires assessing against the decisions during the program, not against the original logic.

Unintended consequences:  Changes in one part of a system often result in unforeseen ways within the system (such as a neighborhood) beyond its boundaries (such as in an abutting neighborhood).  This means assessment has to ask questions about impact on the wider system.

System dynamics:  Change is constrained by system dynamics and sustainable change requires change in system dynamics.  This means assessment must focus on how the system dynamic has changed, not on what changes have taken place within the system dynamic.

Tipping point:  Latent change often leads to tipping points characterized by sudden major transformation.  Therefore assessment is required of the underlying change as well as the surface level of change.  This might involve, for example, noting how discourse has changed.  Dave Snowden’s Sensemaker would be great for this.

Iterative processes are particularly critical to support complex environment assessment with respect to:

  • Assessing impacts in real time;
  • Adjusting action to maximize impacts;
  • Adjusting program goals to maximize impacts; and
  • Setting new baselines and indicators of success against those baselines.

Of course all this emphasizes undertaking interventions as learning processes. Attentiveness to what is changing and new potentials must be supported through discussion, documentation and routines.

Categories
Communications Leadership Learning M&E

Seven Complexity Implications for Multi-Stakeholder Networks

Learning to work with complexity is absolutely essential for people working with multi-stakeholder networks.  It’s key to effective leadership, network development, impact measurement, communications, and change strategies.  A multi-stakeholder change network developing all these for a complicated rather than complex system is bound to fail…or at least fall far short of its objectives.   

When co-leading a workshop with The Change Alliance in Nairobi a few weeks ago, colleague Jim Woodhill brought forward again David Snowden’s cynfin (pronounced kun-ev’in) framework that presents the distinctive essence of complexity in an easily-understood way. 

Snowden starts with two major contextual factors that determine the appropriate strategic framework for an initiative.  The factor of level of abstraction is related to  trust – trust in whether people share the same values and goals, whether they believe they have the competency to do what they say they want to do, and whether they actually have worked together enough to knowing the meaning of each others’ use of words.  Where trust is high, people can handle relatively high levels of abstraction.   

A second factor is culture. Snowden distinguishes between a techno- or physical-oriented situation that can be minutely described and controlled, with a learning context where there are many unknowns and knowledge is emergent. 

This produces four different operating environments that require four different ways of strategizing and four different logics for any initiative like a network.  In the Visible Order domain, bureaucratic and highly structured approaches such as those associated with government application of laws and rules, and traditional business production lines are appropriate.  Strong control rules account for the possibility of low trust and a high number of transactions.

In the Hidden Order domain professional skills become much more important, since many more scientific judgment calls are made against an array of options.  This is associated with complicated situations, where there can be a large number of interacting variables but they can be understood through reviews of experiments, and controlled.  Think of sending a person to the moon.  This is the realm of scientific management.  This requires high abstraction and trust in technical abilities. 

Rather than techno-focused, the complex setting is people-focused. Rather than joining around hard science knowledge and standards, the high abstraction comes with people collaborating voluntarily around shared values and concerns.  However, the collective work is usually very broadly defined in terms of objectives – in fact, the work involves actually clarifying the objectives (eg:  what does sustainability really look like?).  People are continually learning about each other and how they can collaboratively realize aspirations.  Cause and effect cannot be separated, because they are intimately intertwined. 

The fourth action domain is chaotic.  In this situation, there is inability to learn or predict because the situation, people and issues are all changing so rapidly.  Patterns do not exist and do not emerge through interactions.  Crisis management is needed.

Seven Implications

1.     One implication is that multi-stakeholder change networks like Global Action Networks (GANs) by their very nature tend to operate in the complex domain.  They operate with high diversity in participants, in terms of culture, language and objectives.  They are addressing “stuck” topics and B-HAG (big hairy audacious goals).  They cannot look to history as a guide.  Their decision model is probe-sense-respond. 

2.     This emphasizes the importance of GANs developing sophisticated learning processes.  Traditional history-based research takes a second seat of importance.  GANs must support action learning projects to test and “emerge” potential answers to collective challenges.  They need to be good at understanding worst practice and adapting good experience in one part of a network, to work in another…as opposed to linear “scaling up” best practices that require a highly predictable environment to be useful. 

3.     Operating in the domain of complexity also emphasizes the importance of not thinking of “a strategic plan”, but of “strategic planning” as an on-going process. Rather than operating with detailed projects, a GAN should identify some broad goals and continually adjust as new opportunities and new learning arise.  “In a complex domain we manage (in order to) recognize, disrupt, reinforce and seed the emergence of patterns;  we allow the interactions to create coherence and meaning,” says Snowden. 

4.     Effective structures tend to take the form of well-connected networks, without a dominant center.  Empowerment for self-organizing is important. 

5.     And complexity once again emphasizes the value of leadership that is leaderful.  More on these implications next week.

6.     For communications, complexity stresses the importance of creating virtual platforms for discussions where people can probe, and sensing what is emerging to formulate appropriate responses – and this de-emphasizes the role of communications as “telling people”.  Supporting development of informal communities is key.

7.     Probably the most problematic implication for operating in the complexity domain is with impact measurement and evaluation.  Linear tools like log frames are appropriate for the techno worlds, but not for the learning domains.  They suppress the ability to respond to emerging knowledge and opportunities.  Moreover, they are predicated upon identifiable cause-effect relationships.  Although learning-based evaluation systems like outcome mapping are beginning to develop, we lag in tools and in understanding of those who are demanding evaluations. 

Significant development issues are associated with this Cynefin model.  For example:  does a GAN aim to shift the issue it is working on, into the realm of scientific management?  Is that possible?