Posted by Steve Waddell in Communications, M&E, Net Dev on August 3, 2010
If you are starting or working in a network, you should use new mapping technologies to “see the whole”. Knowing who is working in your field and their relationships is key for good strategy. In a previous blog, I briefly introduced several mapping technologies. Now I’ll give more details about one of the easier and quicker ways to map: using web crawls. They give a view of the structure of the “virtual (digital) world”, that is becoming an increasingly good description of “real world” relationships as the internet develops.
“Hyper-links” embedded in organizations’ web-sites that link to another organization’s site can be gathered through web crawls of internet sites. A map such as in the diagrams below can then be generated to describe organizations’ virtual relationships.
I did this with the Global Organizational Learning and Development Network (GOLDEN), using the Issue Crawler developed by Richard Rogers at the University of Amsterdam. The mapping was driven by the GOLDEN goals in terms of key stakeholder groups. It aims to bring together leading academic research centers and businesses to spur attainment of sustainability. The issue arena can be labeled “academic-corporate interactions for corporate sustainable responsibility (CSuR)”. The founders speak in terms of engaging 50 research centers and 250 corporations within a short time. “Community organizing” is not framed as a goal, but it is an implicit activity to realize the goal.
Rule number one in initiating a network is to understand that someone is always already working in the issue arena…and to identify them if possible. As in most cases, some of the leaders in the issue arena are among the founders of the new network—although they’re all academic CSuR leaders. And as is also true in most cases in global networks, they are mainly older white men (like me!). To realize a global network with all the complexions that implies for the issue, mapping can help enormously.
Issue crawls begin by identifying key URLs – referred to as “seed URLs” – relevant to your issue arena. In this case, I identified networks of organizations of two major stakeholder groups that are working CSuR. First to note is that the issue arena is already quite crowded: I identified 9 existing academic-business CSuR networks including ABIS, GRLI and UNPRME. Also I identified 14 business CSuR networks including Business for Social Responsibility, the International Business Leaders Forum and the World Business Council for Sustainable Development.
Using these 23 seeds to conduct crawls produces data about URL connections and maps that display connections visually. Some notes on “reading” the maps:

In Map 1 (click on the map to enlarge) only eight of the seed URLs are among the top 200 nodes. The map suggests two centers (clustering of big nodes): one around intergovernmental organizations like the UN and World Bank, and another around multi-stakeholder networks, in particular the Global Compact and the Global Reporting Initiative (GRI). This leads me to do additional runs that:
Map 2 is a run excluding the IGOs. It shows the business CSuR (green) nodes as central, the academic-business CSUR (red) seeds as fewer and more peripheral (suggesting the importance for them of their linkage to IGOs rather than business CSuR networks), and reinforces the idea that the GANs should be included because of their centrality and size.

Map 3 also includes the GANs as seeds (purple). We can see that there are more academic-business (red) and business CSuR (green) network seeds (10), which also supports the decision to include GANs and exclude IGOs. The seeds for the business CSuR networks and GANs group, which would be expected as they tend to link to each other and the same organizations.
The Map 3 academic-business CSuR networks (red) are comparatively small, non-central and dispersed; three are really part of an educational grouping that suggests their orientation towards educational institutions is significant stronger than towards businesses (if they were balanced, you’d expect to see them with the GANs); the two Asian ones are quite different with Asian associations.

Each of these maps is accompanied by several types of data-base outputs summarized in this excel spreadsheet. For example, Columns B-C list all the nodes in the network (I set the maximum at 600 nodes) by inlinks; another data output even gives lists by web-page, to identify locations/people within large organizations that are relevant.
In a run using snowball analysis (rather than co-link) the crawl retains URLs with at least one link from seeds. Run with the three stakeholder groups, this produced a list of 5317 URLs (Column D). And other maps show these by geography which more helps identify, for example, research centers in China. GOLDEN is particularly interested in particular geographies, like China. More runs can be done for China in particular, and using Chinese-language web-sites.
So here are some ways all this work helps strategically. It gives:
Of all the benefits, however, perhaps the greatest is simply helping people to think more in network terms. Although not as helpful in this regard as something like value network analysis, web crawls are a great step forward. And of course if you’re interested in me helping you apply these types of analyses to your situation, email me!
[...] Note. It draws from a system stakeholder analysis, including issue crawl mapping explained in a recent blog. The strategy focuses on strengthening the critics who are pushing for a transformational vision, [...]