That’s perhaps the shortest summary to a request by Anthony Draffin, in a comment to my previous post ‘Not quite bus-pass day‘:

On a selfish note… It’s apparent that the common thread to dowsing, printing and enterprise architecture is your ability to look at a field holistically and apply logical thought to extract inconsistencies and errors, as well as looking at new ways of doing something more efficiently to meet the original aims. That’s a rare skill. Have you given thought to documenting how you go about doing this? While I imagine it’s the application of a number of taught skills, the way you put these together must be far from ubiquitous. Have you considered teaching this? Personally, as a 27 year old, I want to soak up as much of your approach and thought process as you’re willing to offer.

(Warning, this is going to be another (very) long one, mainly because there’ll be several case-studies.)

Amused that Anthony says he’s 27, because that’s about the age that I really got going on this. (A little earlier, actually: the first dowsing book came out when I was still 24. I used to have to apologise for not being the age people expected me to be, namely at least 75! )

I wouldn’t say that any of what I do is a ‘rare skill’, although it’s true that it’s not often acknowledged or respected – perhaps because, by its nature, it necessarily tends to be disruptive to any comfortable status-quo. I’ve been doing it since a very early age – for as long as I can remember, anyway, certainly way back in primary school – but it’s actually the standard approach used in most forms of design-thinking and the like, as taught in art-college or architecture-school or good engineering courses or even in the US military. It’s also what really happens in scientific research – see, for example, WIB Beveridge’s classic The Art of Scientific Investigation.

My own particular twist on it arose because I’m not much good at doing things, or making things (I tend to describe myself as ‘ambi-sinistral’ – the opposite of ‘ambidextrous’… ). Hence I tend to focus instead on the thinking behind the doing or making or whatever, always searching for the simplest way to do things, the most effective way, and so on. Kind of recursive, if you like, but it works well. Except for that little problem that it tends to be so darn disruptive…

Methods, mechanics, approaches

One place to start would be around skill itself, and the key themes of my Masters thesis, way back in 1976. Back there, I described a skill – any skill – as being made up of three components:

• the methods used in the skill
• the mechanics and other real-world constraints of the ‘objective’ context of the skill – that which is common to everyone
• the approaches, assumptions, mindset, paradigms, physical dexterity and other ‘subjective’ context for the individual (the ‘operator’) – that which is specific to the individual

What I found, very quickly, was that most people seem to focus on the methods used in any skill. But that actually misses the point: the methods used by any skilled operator arise from their own personal resolution of the mechanics and the approaches – the ‘objective’ and ‘subjective’ components of the skill. This is why using someone else’s methods doesn’t always work, and why ‘best practice’ can be dangerously misleading: the mechanics of the issue remain the same, by definition, but the context is different, and hence may well need different methods.

Focussing on method also makes it much more difficult to tease apart the separate threads of mechanics and approaches. It should be obvious that blurring the objective and the subjective is not likely to be a good idea, and yet that’s exactly what happens whenever we focus only on method.

In all skills-work – in fact in just about every human context – we also come face to face with Gooch‘s Paradox: “things have not only to be seen to be believed, but also have to be believed to be seen”. In an all too literal sense, in skills-work, reality is what we say it is: we actually create it, from nothing, or rather from a combination of imagination and hard work. (In this kind of context, it doesn’t really make sense to ask the question “Is it real or imaginary?”, because the only possible answer is ‘Yes’ – both, therefore neither.) To resolve Gooch’s Paradox, we treat the approaches – our assumptions and beliefs – as if they are part of the mechanics of the context. The danger is that we may forget that point about ‘as if’, and – if we think about those assumptions at all – think that they are part of the fundamental mechanics of the context, rather than an arbitrary choice to achieve some particular purpose.

Once assumptions creep in – in other words, whenever the subjective is blurred into the objective without conscious intent to do so – what we have is a context to which arbitrary constraints have been applied. Which places arbitrary limits on possibility. Which is kinda pointless, really. But the only way that we’ll be able to see that the constraints are arbitrary is to step back a bit, and re-separate the subjective from the objective. Hence a kind of recursive methods-to-look-at-methods, analysis-to-unpack-analysis, and so on. Which is what I do.

As I mentioned in my reply-comment, much of the ‘how I do what I do’ is already documented in various ways throughout the books, such as in Everyday Enterprise Architecture (which focusses on method in a business context) and The Disciplines of Dowsing (which looks more at ‘thinking about thinking’). The core of the latter book is the ‘four disciplines’ section (see the summary on the separate two-page reference-sheet) and the ‘seven sins of dubious discipline’ (currently listed only in the book): it wouldn’t take much work to translate those into almost any other context.

What I’ll use here is the Five Element / effectiveness framework that I use in a lot of my client-work these days (though often in somewhat covert form). It’s nothing special, in fact it’s little more than a recursive use of a pair of matched checklists. The first of these, as summarised in the ‘Five Elements’ chapter in SEMPER & SCORE, is a set of perspectives on the overall context:

• Purpose – what are we aiming to do here? and why? (see also the slidedeck ‘Vision, Role, Mission, Goal‘)
• People – who would be needed for this purpose? what skills and relations do they need? what are their mutual responsibilities?
• Preparation – what planning and logistics would be needed for this purpose? what assumptions and mindsets apply here? what are the key events that trigger action?
• Process – what needs to be done to achieve the purpose? when, how and with what would this be done? when is each process complete?
• Performance – what constitutes ‘success’, and for whom? what information and metrics are needed to keep everything on track? what would be needed to support continuous improvement?

The other checklist is a set of keywords on effectiveness, which are sort-of orthogonal yet also sort-of linked to the Five Element set. Listing these in the same order as above:

• Appropriate – is this on track towards the purpose?
• Elegant – does this support the human-factors in the context? (e.g. simplicity, ergonomics etc)
• Efficient – does this make the best (e.g. least-wasteful) use of the available resources?
• Reliable – can this be relied upon to deliver the required results?
• Integrated – does this help to link everything to everything else in a consistent way?

To assess a context, we can start from anywhere at all. The point is that we use these checklists not as linear lists, but as a reminder to keep looking round, bouncing back and forth between each of the interconnected themes in the two lists, looking at the context from every possible angle, and at every level from really-big-picture to finest-detail, building up a kind of hologram of the overall context, using one form of sensemaking to bounce off others, and so on. The book Real Enterprise Architecture provides a complete worked-example of this kind of recursive process as applied to whole-enterprise architectures.

Questioning everything

Looking back at the various areas I’ve worked in or with, there’s a fairly consistent pattern about what I’ve done and the sequence in which I’ve done it.

The first stage is just getting involved at all: taking the ideas and practices at face-value, and putting them into practice as if they are entirely ‘true’. That usually works for a while (not least because that’s what everyone else is doing).

I then allow myself to start to notice the niggles, the things that don’t quite seem to work, where ‘what it says on the tin’ doesn’t actually deliver what it says on the tin. The problem, of course, is that we can’t assess the validity of a logic from within the logic itself. Yet we also can’t actually work on the context without being inside the logic (or some form of the logic). This is where we hit Gooch’s Paradox head-on: we have to see it to believe it, yet also have to believe it to see it. The only way out of that dilemma is to start to use beliefs as tools – which can be kinda challenging…

In my experience, there are two parts to this:

• identify the big-picture theme for the overall context (the ‘vision’ or, as architects would put it, the unifying ‘parti‘)
• apply design-thinking tactics to question everything, switching beliefs in order to experience the context in different ways, and test the apparent results

The tactics to identify the key-theme(s) are usually straightforward. A classic example is the ‘Five Whys’: just keep asking “why?” until eventually we hit a ‘Because.’ – or rather, a real ‘Because.’ that makes some degree of sense, rather than one that’s just used to get people to stop asking awkward questions! These days I tend to look for a brief overview-statement – usually only about three to five words – that has a distinct three-part structure: it identifies the ‘things’ or concerns that matter to everyone in the context, what’s being done with or to those items, and why it’s deemed to be important. This gives us a stable anchor to which we know we can return, and against which we can test anything in the context.

Then, following standard ‘design-thinking’ tactics, we use a suite of ‘disruptive’ questions about the context – for example:

• what’s another version of this?
• what does this look like at a smaller scale, or a larger scale?
• what happens if we substitute something else for this?
• what happens if we invert some or all of the rules?
• is there a ‘term-hijack’ here? – does a small subset purport to be the whole, blocking the view to any other aspect of the context?

This is where things often get to be, uh, fun… – because it’s very common to find aspects of the context that a) don’t and can’t make any sense, b) clearly don’t work ‘as advertised’, in fact usually work against the nominal aims of the overall enterprise, yet c) there are key players with a lot of vested interest in ensuring that the status quo remains unquestioned and unchallenged. Don’t be surprised at this: it happens every time.

This is where a certain amount of dogged determination becomes essential… Also essential is a very clear, insistent emphasis on the big-picture, on holding to the overall vision for the shared-enterprise, because that’s often the only thing that will persuade people that there’s no ‘personal attack’ here, that instead the only purpose of the challenge and the enquiry is to make things work better, for everyone. (We have to be real about that, too: we need belief in ourselves in order to keep going, it’s true, but we need to keep questioning ourselves as well. It’s one reason why serious self-doubt is a chronic yet necessary occupational-hazard here.)

We need to keep hammering at this until we do start to get a clear separation between the mechanics of the context – which usually turn out to be surprisingly simple – and the approaches to the context – which are, by definition, individual and subjective. Then we can start to work towards new methods that work with the context under the current conditions.

The same seems to apply to just about any type of context: an individual’s personal challenges in developing their own skill, a business, a social context, a single conceptual tool, or an entire discipline.

Scattered throughout this weblog and the sister-weblog Sidewise, you’ll find examples of those techniques in use. Sometimes it’s reasonably straightforward, sometimes rather more controversial, but you’ll see in each case that’s it’s essentially the same principles, the same tactics.

I’ll also summarise here those same techniques in use in four different large-scale domains that I’ve been involved with over the decades: dowsing, desktop-publishing, domestic-violence resolution, and enterprise-architecture.

Example: Dowsing (1970s)

Big-picture theme: finding things, particularly where conventional (mechanical/physical) techniques either won’t work or are unavailable.

History: as a discipline, has been around ‘forever’, and often highly controversial – first from priests who regarded it as ‘the work of the devil’ etc, then later from would-be scientists who wanted to ‘explain’ it and couldn’t. When I first got involved, in the late 1960s, the field was pretty much moribund, with a random mixture of wild claims, erratic discipline, no formal methodology or theory-base as such, a long history of inconclusive scientific experiments, and the first flush of hype-laden New Age ‘thinking’ (if that’s the right term…). Most of the people involved were well into their sixties, seventies or more (which I, uh, wasn’t…). The key players consisted of a kind of closed ‘military club’ (water-finding being very important to an army on the move), a few variously-erratic practitioners (often with wild-eyed ideas about health and the like), a swathe of armchair-theorist camp-followers who talked a lot but did nothing, and a few people who really did know what they were doing and wisely kept themselves well away from the mess.

Conceptual mismatch: The most common assertion was that it was a special ‘innate’ skill that only certain ‘special people’ could do. Methods that often clashed or even flatly contradicted each other could lead to the same result; the same method used by different people would lead to wildly different results. Most of the theory in use – such as notions of ‘waves’ or vibrations’ or ‘radiations’ – was either meaningless or just plain wrong in terms of conventional physics. (Much of it did sort-of make sense as metaphor, but there seemed to be little understanding of the difference between active-metaphor and concrete fact.) Muddle-headed ‘New Age’ ideas merely added to the overall mess.

Vested interests: On the one side was the moribund ‘military club’, who liked the idea of being ‘special and different’, and/or the ‘right’ to tell the ‘lower ranks’ what to do, whether it made any sense or not. On the other side were the upcoming ‘New-Agers’, who were not going to let anything block their path to potential fame and fortune. (I’m being cynical, I know, but that’s exactly what happened.)

Assessment and action: Assess the purported theory, and scrap most of it: it’s meaningless. The only parts of the theory that do make sense and do have solid experimental backing revolve around perceptual psychology and physiology – particularly around weighted-sum merging of multiple channels (which is why there’s no single ‘the method’) and around edge-triggered reflex-response (which is why some experienced water-finders can’t find static water even when they’re standing on top of it). If some kind of tool is used, almost all of the tools act as some form of mechanical amplifier – if I move my hand a little, the tool moves a lot. (I’ve only ever found one case where that principle didn’t apply at all.) Materials, structures, theories and so on seemed to matter only because people believed that they did: in most cases, a simpler alternative would work just as well, if not better. Keep stripping it back to the bare essentials.

It is a true skill – but it’s not one that’s restricted to only ‘special people’. Instead, it’s a learnable skill: anyone can do it – though whether they may or will do so are entirely separate questions! (There was quite a lot of pushback from the ‘military club’ against the idea that ‘anyone can dowse’.) It’s also a skill that requires a lot of practice and a lot of discipline to get right. (Unsurprisingly, there was a lot of pushback from the ‘New-Agers’ on that point, and there still is – see the book Disciplines of Dowsing.) It’s also a skill which often requires a wide range of psychological ‘tricks’ to help people slide past Batcheldor’s ‘witness-inhibition’ and ‘ownership-resistance’ – in other words, “this isn’t happening, and if it is, it isn’t me”.

End-result: After a few months’ experimentation and subsequent practice over several years with a wide range of students, I’d stripped it down to the point where I could get most people started on the basics within less than two minutes, using two bits of fencing-wire from the garden as simple instruments. The notion that ‘anyone can dowse’ is now firmly established in the canon, and the teaching-methods that I developed (based on, self-responsibility, self-critique and continual-improvement) are still some of the most common currently in use.

Example: Desktop-publishing (1970s-80s)

Big-picture theme: getting ideas and information out into the public space.

History: I trained as a graphic-designer/typographer, and became professionally involved in typesetting in the late 1970s, with the early developments in smaller phototypesetting machines. (‘Smaller’ being a relative term here: the first system we bought required a room of its own and a separate darkroom, and cost more than my house.) The big bottleneck was keyboard input: the typesetting unit was capable of running much faster than a single operator. Although the internal technology was extremely complex, the input was not: some machines still relied on a very simple 6- or 7-channel punch-tape reader, using control-codes to extend the effective size of the character-set.

At the same time, simple but usable microcomputers were just starting to come onto the market. (My first microcomputer had only an 8-character LED display, hexadecimal keypad and 256 bytes of memory; the more usable Ohio Scientific systems that we first used for real had a proper keyboard but still only 8kbytes of memory, and the only storage was on audio-cassettes.) Almost all of these machines used a 7- or 8-channel character-set (ASCII or extended-ASCII); most also provided some form of direct data input/output for interfacing to other systems.

It seemed to me that there should at least be some way to use a basic micro as a much cheaper input-terminal, using simple code-translation and a standard hardware-interface. It also seemed probable that other people would want to do the same – taking control of their own publishing, driving a typesetter direct, or both. In the longer term, that could well be quite a large market.

Conceptual mismatch: This is best summarised by the phrase (exact quote, in fact) that “there is no interest in typesetting from microcomputers, and there never will be”. There were all manner of arbitrary demarcation-lines across the whole context, both on the pre-press side – such as between authors, publishers, unions and printers – and on the technical side – particularly between typesetter-manufacturers, computer-manufacturers and various hobbyists and hackers – most of which arose more from historical ‘turf-wars’, ‘positioning’, and mutual misunderstanding than from any concrete distinctions. On the union side especially, there were many arbitrary assumptions, based on the belief that technology could not and would not change, or if it did, it could not and would not be allowed to make any difference to existing processes or roles.

Vested interests: The entire context was riddled with vested interests, almost all of which were in conflict. A stream of intermediaries – agent, publisher, pre-press, press, retail – stood between author and audience. Typesetting-systems were expensive pieces of equipment, yet with not all that much to justify their cost: there was lot of money to made there, both from machinery-sales and from fonts and other consumables, and hence a lot of ‘need’ to protect those sources of income. Until IBM eventually stepped in, most of the microcomputer manufacturers were trying to establish themselves as ‘the manufacturer’, resulting in a plethora of mostly-proprietary, mostly-incompatible hardware and software non-’standards’ – at one point we had to buy two machines whose sole function was to read the two hundred or more different disk-formats used on the four distinct disk form-factors then in common use: 8″, 5.25″, 3.5″ and 3″. Weaving a path between all the different vested-interests and proprietary structures was, frankly, a time-wasting nightmare.

Assessment and action: On our first machine, we’d been told emphatically that it was physically impossible to connect a microcomputer; a weekend spent poring over technical specs and waving a soldering-iron around a bit on a prototype-board soon proved that ‘fact’ wrong, whilst the only software we needed at first was a straightforward lookup-table to translate between character-sets. It really was that simple. (We avoided warranty risks by using opto-isolators, so there was no electrical connection between the two machines.) For our later, larger systems – which were capable of typesetting a reasonable-sized book in less than an hour – the hardware-interfaces were already built in. This gave us ‘direct typesetting’ capability, but it still required operators to know – and use – the distinct formatting-codes for each type of machine.

The next step was to hide the complexity, using the format-code in common word-processors such as WordStar to trigger font-changes and the like. (I believe we were the first people to use style-codes, such that a single hideable code – *F1, for example – would change the entire style, including paragraphs, indents, font-family and so on.) At that point, people could use ordinary word-processors to typeset text: the first true precursor to desktop-publishing.

It worked, but there were still limitations. (Our main competitor, meanwhile, was using a mangled form of SGML which still required people to embed hard-codes in the text; in our system, all of the formatting could be invisible.) The main problem was that people couldn’t see beforehand exactly how much space any text would take up – a very important concern to two of our customers, who were producing page-spread books and partworks, Dorling-Kindersley style. Hence some serious code-hacking (all assembly-language, with multiple overlays to squeeze into no more than 40kb of memory) to create a post-processor that would copyfit line-by-line for the correct fonts and sizes, and output a symbolic result to a dot-matrix printer. This was probably the first viable attempt at a true desktop-publishing system – several years before Macintosh and, later, PageMaker.

End-result: I’m good at creating ideas and markets, and all the preliminary work that gets things going, but I’m not good at running businesses – that’s a different mindset entirely. Eventually we sold out to another pre-press company and (in an all too literal sense) I ran away, first to the US, and then onward to Australia. I believe it’s still running, and certainly made millions for the new owners. (I didn’t, of course.)

Example: Domestic-violence resolution (1980s-90s)

Big-picture theme: reducing and repairing the damage from social harm, particularly between individuals.

History: Fights and power-games between individuals in a domestic context have been part of the human story since forever, but had usually been largely covert and ignored as ‘a private matter’ for most of that time. It was brought into public notice in 1970s by women’s activists, most notably Erin Pizzey, founder of Chiswick Women’s Aid. Unlike Pizzey herself (who has always insisted that domestic-violence (DV) is a human problem, not a gendered one), most activists purport that DV is something that happens almost exclusively to women, and caused almost exclusively by men – so much so that some have called for the term ‘domestic-violence’ to be replaced always by the term ‘violence against women’. Most current law (e.g. US ‘Violence Against Women Act’), support-structures (domestic-violence help-lines) and formal theory (e.g. Duluth) reflect this assertion. I became involved in the field during the 1980s as a member of a pro-feminist men’s group who were taking up the feminist challenge that all violence was caused by men alone, and therefore men’s responsibility alone to resolve the (purportedly) ever-rising tide of men’s violence against women. The issues became more personal later when two of my lesbian friends asked me for advice after they had ended their relationship with a knife fight (without injuring each other, fortunately) but had been explicitly shut out from any help because no man could be blamed for the violence.

Conceptual mismatch: The theory was straightforward: men are the problem, women are the solution, and the only useful thing that men can do is blame themselves for everything that goes wrong in the world. Everything in my background supported that assertion, hence it seemed to make sense: self-blame had been a very deeply ingrained habit for me, going right back to earliest childhood. Yet the whole field seemed riddled with gendered special-cases: behaviours that were definitely violence if done by a man were, if done by a woman, either deemed ‘not violence’ or ‘indirectly caused by men, therefore men’s fault’. In the Duluth model, blame itself was classed as a form of violence only if done by a man, and only if the person being blamed was an adult woman: blaming of men (or in essence almost any other form of abuse of men), was explicitly not classed as violence. And the real catch was that, in terms of outcomes, it clearly wasn’t working: no matter how much we blamed ourselves, and blamed other men, the overall level of violence in the culture around us still seemed to continue to rise.

Vested interests: Looking around, it was very clear that there were a large number of players – mostly but not all women – whose identity and self-worth depended on putting men down, regardless of whether or not this actually helped women in general, or anyone in general. There were also very large sums of money, and large numbers of jobs, that depended on maintaining the assertions around women’s purported exclusive victimhood in this context.

Assessment and action: The first warning-signs appeared in one of our standard text-books, Paul Kivel’s Men’s Work: How To Stop The Violence That Tears Our Lives Apart, which is designed around a series of workshops for senior-school students. The book includes many oddly-unrealistic role-play scenarios in which an adolescent boy or young man is suddenly violent or abusive to a woman; yet the only real example of violence described in the whole book is an actual incident in which two girls had a full claws-out fight when one insulted the other in the classroom – and in which no boys were involved at all, other than to separate the warring parties.

After my lesbian friends had their knife-fight, we discovered that no violence-resolution material was available that acknowledged even the possibility that a woman could be a perpetrator of violence. The standard Duluth model defines violence as inherently ‘male’; on the Duluth Wheel, female pronouns are used exclusively throughout to indicate victim, and male pronouns exclusively for perpetrator, and mutuality (where both parties are both ‘perpetrator’ and ‘victim’ of each other and of themselves) – which clearly applied in my friends’ case – is explicitly denied. I decided to try a very simple thought-experiment: swap the gender-pronouns throughout, and see if it still makes sense in terms of real-world evidence and experience. It did: in fact for most of the Duluth categories of abuse it made more sense than the ‘official’ way round. Also – importantly – two key categories of abuse were absent from the original model: sexual abuse, and third-party-abuse. It became immediately clear that the Duluth model itself was structured as third-party abuse, primarily leveraged through other-blame – in other words, far from reducing violence and abuse, it was actually designed to increase it. (Whether that mis-design was intentional, or merely arose from incompetence and excess zeal, is a separate issue that I will not discuss here… – but the fact of its unfitness for purpose cannot be in any doubt.) A simple ‘de-gendered’ redesign resolved almost all of the structural problems, sufficient at least to satisfy my friends’ immediate needs.

That exposure of the extreme inadequacies of the original Duluth model forced our group to reassess all of our previous assumptions about gender and violence, and thence to look again at the research on whose purported facts we’d based those beliefs. I did two analyses of a much-published study on which Australian public policy was based – the first analysis on the public version of the paper and political assertions from it, and the second analysis on the original academic study, which took quite a bit of work to obtain, since it was not publicly available. Another colleague, as his MA thesis, undertook a meta-analysis of domestic-violence studies in Australia. The results were shocking. None of the original studies were based on defensible methodologies – in fact many were so riddled with basic methodological errors such as circular-reasoning that they were essentially meaningless. And in all cases, all of the methodological errors either inflated the female injury-rate or risk, diminished or denied the male injury-rate or risk, or both: there were no exceptions. In short, almost none of what we’d previously taken as ‘fact’ was fact at all. The only genuine facts we could establish was that domestic-violence was a systemic issue with some gendered overtones, and that although it that affected both sexes in different ways, overall it seemed to do so almost equally – though there were strong indications from hospital data and the like that the majority of victims were male, not female.

We then looked at public policy, and the provision of domestic-violence support-services. These too were based on the same fundamentally-flawed assumptions and the same unquestioned circular reasoning: women are the only victims, hence support-services are only available to women; and since only women use these services, this proves that women are the only victims. In some of our interviews we discovered that men who’d been abused – knifed, in one case – were referred to police for charges, simply because the models in use automatically deemed men to be the sole perpetrators, regardless of the actual context or evidence. In short, the entire domestic-violence resolution ‘industry’ it was, and still is, an unworkable and fundamentally dysfunctional mess whose structures and methods are all but guaranteed to cause far more harm than good: an archetypal example of the Shirky Principle that any institution will attempt to preserve the problem to which it purports to be the ‘solution’.

End-result: The domestic-violence ‘industry’ is the outcome of a classic example of a ‘term-hijack‘, in which a small subset of systemic issue is misframed as the whole, and strenuous efforts are made to deny or conceal any other aspect of that issue. In effect, the term-hijack converts a resolvable systemic context into a non-resolvable ‘wicked-problem‘, in which every attempt to resolve a problem is constrained by the structural myopia, inevitably making things worse with each iteration. Unfortunately, there are huge vested-interests in maintaining the term-hijack. Anyone who challenges it – as I and many others have learnt to our cost – is likely to come face to face with extreme violence from women who somehow purport that no woman is ever violent. It seems clear that resolving these structural problems would require a high level of honesty and humility from those players – an honesty that in most cases at present seems conspicuous only by its absence…

Some of the material I wrote is out there and in daily front-line use by others – with real success, according to the occasional emails I still receive on the subject. But to be blunt, after a decade of relentless ongoing abuse from almost all sides, I just gave up and literally threw away most of the work that I’d done… the structural dishonesties in this mess are so entrenched and so ‘political’ that I found it just too painful to be involved at all, and it still seems that resolving the mess would require fundamental shifts in societal attitudes and beliefs that would be unlikely to occur within my own lifetime. Oh well.

The issues are generic, though, and can be resolved at a more generic level. You’ll see how some of these exact same issues are addressed in the business-context in my book Power and Response-ability: the human side of systems and its accompanying ‘manifesto‘.

Example: Enterprise-architecture (2000s-to-present)

Big-picture theme: helping organisations and overall shared-enterprises become more efficient and effective (‘doing the right things right, on purpose’).

History: The main focus of enterprise-architecture is around the relationships between structure, purpose and business-execution.As a discipline, it’s been around for at least a century in various forms, such as Taylorism (‘scientific management’), operations-research and organisational cybernetics. I often describe it as based on a single, very simple idea: that things work better when they work together. Although my work often touched on it over the decades, I first became actively involved perhaps fifteen years ago, when trying to tackle issues around long-term knowledge-management in aircraft research. Over the past decade, most of my work has revolved around various aspects of enterprise-architectures.

Conceptual mismatch: The term ‘enterprise-architecture’ implies a very broad whole-enterprise scope. In recent decades, though, the term ‘enterprise-architecture’ has often been (mis)used to denote a very small subset of the real scope, relating to IT-infrastructure or IT-systems in general. This (mis)usage probably arose from a simple conflation of the term ‘enterprise- or organisation-wide IT-architecture’. The result, however, is a very serious term-hijack: the tiny subset of the overall enterprise represented by IT purports to be the whole, with all other aspects of the enterprise – including people, purpose, physical facilities and non-IT machines of any kind – either concealed or denied. In effect, it becomes all but impossible to discuss any aspect of enterprise-architecture without being forced to describe everything in terms of IT – even in contexts where IT-systems are either not relevant or not available.

Vested interests: There are huge vested interests in maintaining the story that ‘enterprise-architecture’ relates only to IT. Many, many billions of dollars are invested each year on IT-systems that purport to resolve inherently-complex enterprise-scale concerns such as customer-relationships, market-relationships, regulatory-compliance and the like. However, by definition, many if not most of these systems are incapable of resolving all aspects of the respective concerns, in effect converting them into non-resolvable wicked-problems; maintaining the ‘enterprise-architecture’ term-hijack makes it possible to conceal or deny the inherent dysfunctionality of the systems, instead maintaining the faith or fiction that the problems created can only be solved by yet another IT-centric system at yet further cost. There are also large vested-interests in training, certification and the like for IT-centric ‘enterprise’-architectures.

Assessment and action: The starting-point for assessment was a simple review of the term itself, deriving the natural-meaning via term-inversion. The ‘natural-meaning’ of a term is the meaning implied by the individual words of the term. The term-inversion here is ‘the architecture of the enterprise’: hence the natural-meaning is ‘anything to do with the structure and purpose [architecture] that underpin the emotional drivers and actions (the animal spirits of the entrepreneur”) in the shared context [enterprise]‘. The purported exclusive-association of enterprise-architecture with IT does not occur in the natural-meaning: in fact the role of IT in the enterprise-architecture is implied only peripherally, as a minor aspect of support for ‘the animal spirits of the entrepreneur’. In other words, what we’re dealing with here is definitely a term-hijack – and an extremely unhelpful one at that, because the constraint on the scope (i.e. ‘enterprise’-architecture constrained solely to IT aspects of the enterprise) has such a limited connection with the actual scope (which would naturally focus more around people than machines).

Most of my work in the past decade, and particularly the past five years, has been focussed on finding ways to highlight the term-hijack, to resolve the resultant problems and dysfunctionalities, and to create models, methods and frameworks to guide a true enterprise-scope architecture, in some cases all the way out to a global scale. The public outcomes of this work so far include several books, a couple of dozen conference-presentations and other slidedecks, and many, many weblog posts.

End-result: We are getting somewhere with this one. Most ‘enterprise’-architecture conferences these days do explicitly include some discussion of the enterprise-scope beyond IT, usually under a banner of ‘business-architecture’, and there’s much stronger linkage to true business-architecture models and techniques such as Business Model Canvas. The real danger now is there’s a tendency towards ‘business-centrism’ rather than ‘IT-centrism’ – in other words, where the architecture sub-domain of ‘the business of the business’ rather than the sub-domain of ‘the IT-systems’ becomes used as the base for yet another term-hijack. The crucial understanding that we’re still somewhat struggling to get across to most of the players in the field is that in a true enterprise-architecture, everywhere and nowhere is ‘the centre’.

But yes, we are getting somewhere with this one. Slowly…

Summary

So that’s what I do, and how I do it:

• explore a context that is of interest to me
• identify the conceptual mismatches that occur within that context, and that make it difficult to achieve effective results within that context
• identify the vested-interests that drive and maintain the current dysfunctionalities in the context, and, where possible, devise strategies and tactics to disarm and disengage those vested-interests
• assess the details of the dysfunctionalities in the context, and identify or design workarounds for those problems, and methods to work on the context when the dysfunctionalities are disengaged
• document the end-results in various forms, as appropriate

It’s a lot of work, and sometimes very painful work, but someone has do it?

A gentle warning on occupational-hazards

To anyone who might want to do this kind of work, I really ought to add some important caveats.

The work itself is actually not that hard. All it requires is a willingness to let go of assumptions, and tackle each of the issues with a rigorous attention to discipline, following the ever-changing rules of the different disciplines that apply at each moment whilst working in that context. Using beliefs as tools can be kind of challenging at times, but again it’s just another skill, and one that’s not that hard to build up over time.

It’s the social aspects of the work that are hard: sometimes very hard…

For starters, it’s often lonely. Very lonely. Part of that is because there aren’t many people who do this kind of work: at a guess, from what I’ve seen around the net and elsewhere, there may be as few as five or ten thousand people in the entire world who work in this space. Social-media does help to ease the loneliness a bit – the people I work most closely with are scattered literally across the entire globe – but it’s not the same as working in close proximity with close colleagues every working day.

Another part of the loneliness is that the feeling of loneliness – and likewise insistent sense of self-doubt – is actually inherent in the work. It’s almost an indicator of success: as Whitney Johnson put it in her HBR article ‘Disrupt Yourself‘, “If it feels scary and lonely, you’re probably on the right track”. To put it the other way round, the times when we feel most certain are probably the times when we’ve most likely missed the point. It’s hard, and it usually hurts, every single day: so if you can’t cope with a relentless, all-pervading feeling of failure, and yet somehow still create the required results, you really shouldn’t to do this work. There are plenty of other much easier ways to make a living, after all. (This isn’t a macho thing, “I’m tough” and that kind of garbage: in my own case, to be honest, I’m probably not suited to do most other kinds of work anyway. For me, though, there’s a real sense of ‘a calling’, an inner drive to do this work, whether I want to or not: and often that’s the only thing that keeps me going… )

Another crucial point is that whilst there’s a great need for this kind of work, there’s also a huge ‘anti-want’ for it. Every aspect of this work implies some kind of mythquake; and anyone who has a vested interest in the status-quo – which in effect that includes most of our would-be employers, amongst many, many others – will not want that mythquake to occur. It’s disruptive: it is, in a very literal sense, often anarchic. So for much if not most of the time, we’ll need to do the work ‘by stealth’, embedding it in other more conventional analysis-work or the like. Doing it ‘by stealth’ is often the only option if you’re an employee, and even then it can be risky: as one of my ProFuturist colleagues put it, “if you’re employed as a professional futurist, and you’re not being fired at least once every year or so, you’re probably not doing your job properly!”

In my own case, I’ve never been an employee: only ever a self-employed contractor, an independent consultant or running my own business. I’ve survived somehow, though often I don’t know quite know how – it’s certainly not an easy way to run one’s professional-life. But I’m well aware that’s not a viable option for many people, especially those with young families. If you are an employee, and you want or need to do this kind of work, you definitely need a Plan B – and work hard on building and maintaining your professional reputation, such that you can recover from being fired after that ‘one disruption too many’.

Another subtle problem that affects many of us arises from the fact that this work requires us to be very good generalists. The good part of being a generalist is that we’re able to learn fast and be interested in anything, at any level of the enterprise. The disadvantage is that, when people compare us to specialists, we almost always come off second-best – and the fact that we specialise in being generalists doesn’t seem to count, especially where the over-simplistic assessments of recruiters and the like so often come into play. In almost all of my contract- or consultancy-work in the past couple of decades, I’ve ended up doing a different (and much broader-scope) role than the one I was nominally employed for: the problem was that I somehow needed to employed for something in the first place, and that can be a real hurdle. So the catch for us is that we need to be at least as skilled as the typical specialist, whilst also being very skilled as a generalist. It’s not easy, and is one reason why the really good enterprise-architects tend to be older, often into their fifties or more – simply because it takes that long to build up the generalist portfolio and experience whilst embedded in what is (to be honest) often a complete waste of time and effort in a ‘required’ but irrelevant specialist role.

Overall, though, it’s probably the loneliness that hurts the most. But if you can cope with that, and with all of the other challenges of ‘the trade’, then yes, we definitely need you… come and join the club, perhaps?

In principle, power should be straightforward. The physics definition – roughly speaking – is that power is the ability to do work. Wherever there’s work to be done – in whatever form that that ‘work’ might take – there’s a need for the power to do that work. Should be simple enough to identify and model that within an enterprise-architecture, surely?

Unfortunately, no, it’s not that simple – because most social definitions of ‘power’ tend to be closer to ‘the ability to avoid work‘. Therein lie lots of, uh, interesting problems for enterprise-architecture…

Hence power is something that we really do need to address in enterprise-architecture – even in an IT-centric architecture, let alone one which covers a true whole-of-enterprise scope. Read on?

To say that questions of power are hugely political is perhaps the understatement of the century… so I’ll take some care here to use definitions and descriptions that are strictly generic. (So generic, in fact, that you should also be able to use them to assess relationships between machines, between IT-systems, and between ‘systems’ in any abstract or concrete sense. More on that later.)

For the architecture, the core requirement is to distinguish between power-relations that are functional (aligning to the physics definition of ‘ability to do work’) versus those that are usually dysfunctional (aligning to that social definition of ‘ability to avoid work’), and then develop an understanding about what – if anything – to do about the latter.

Before we do that, we need some clarity on what we mean by ‘work’. Again following a physics definition, we could say that work is the rate at which energy is expended – and in the human context, usually energy expended towards some desired end. But again in a human context, there are many forms of energy, and hence many kinds of work. Digging a ditch is work; so is solving a technical problem; so is building a working-relationship with others; and so is reclaiming hope from despair, finding the energy to carry on after apparent failure. Different forms of energy are required to do those different types of work – physical, mental, relational or emotional, aspirational or ‘spiritual’ (the latter being defined as related to “a sense of meaning and purpose, a sense of self and of relationship with that which is greater than self” – which applies in the business-context just as much as in anywhere else). Machines are great at doing physical work; IT-systems can be configured to do some kinds of mental work; but only living-creatures (usually humans, in this case ) seem to have the ability to do the other forms as well.

People are great at transforming between energies – converting excitement and motivation into physical work, for example, or gaining aspirational satisfaction from resolving a technical challenge. But they’re also great at avoiding the work – and that’s where the problem really lies.

The SEMPER metric for ‘ability to do work’ uses a simple five-step scale:

• 5: Wholeness-responsibility: ‘command’ relinquished – individual actively committed to organisation and enterprise
• 4: Adaptation: ‘control’ relinquished – organisation enables and support individual difference
• 3: ‘Best practice’: best that can be achieved with a ‘command and control’ organisational model
• 2: Passive dysfunction: silo mentality – locally efficient but globally ineffective
• 1: Active dysfunction: destructive, ultimately often fatal to the organisation

In general, machines will only be able to reach a level-3 on this scale, whilst some ‘autonomous’ IT-systems will reach to a level-4. Only real-people will be able to achieve a level-5, because it requires personal commitment and personal responsibility; but likewise only real-people will fall to a level-2 or level-1, which are the outcomes of an explicit absence or rejection of personal responsibility…

Level-1, ‘Active Dysfunction’, or ‘power-over‘, can be summarised as any attempt, in any form, to prop Self up by putting Other down. (That’s the ‘win/lose’ version: there’s also a less-common ‘lose/win’ version, ‘propping Other up by putting Self down’, which some people might think of as praiseworthy, but in the long run is equally dysfunctional.) We see this often in businesses that believe that they depend on destructive-competition: the sales-people, for example, soon learn that if bonuses are based on relative performance, they can seemingly ‘make more money’ by sabotaging each others’ work, sending the company into a downward spiral from which it is unlikely to recover. Much the same occurs with machine-systems that are (usually unintentionally) set to ‘compete’ with each other, or business-systems with fixed budgets that force business-units to fight against each other for ‘their’ slice of the budget cake. We also see it in sometimes within computing systems, such as mutual-deadlock or ‘deadly embrace‘. Note too the ‘the Other’ may in some cases actually be the self – for example, putting oneself down in the past to provide the illusion of propping oneself up in the present.

Level-2, ‘Passive Dysfunction’, or ‘power-under‘, can be summarised as any attempt, in any form, to offload responsibility onto an Other without their engagement and consent. (As above, there’s also a ‘lose/win’ version, ‘taking on responsibility from an Other without their engagement and consent’, which again might seem praiseworthy but is again equally dysfunctional.) In business, the archetypal form of this is organisational silos, or the “not my responsibility, mate” buck-passing. It can also be seen in Dilbert-like phrases such as “the only reward for responsibility is more responsibility”, and “no good deed goes unpunished” – the latter being closely related to the dangerous tendency to equate responsibility with blame, acting as a huge disincentive against functional behaviour.

Collectively, power-over and power-under could be categorised as ‘power-against‘, since that’s one of its main characteristics: being against something, being ‘against’ others. It’s likewise typified by competition-against – where the aim is not so much to ‘win’ as to make all others seem to ‘lose’ – and (where collaboration occurs at all) by collaboration-against – collaborating with ‘same-as-Self’ against an often arbitrarily-selected not-Self, or Other. By contrast, the other levels are characterised by ‘power-with‘, where competition, for example, usually takes forms in which competition-with is used to push each of the players to greater achievement, and hence, in a sense, everyone ‘wins’.

The crucial point is that power-against – power-over and/or power-under – is inherently ineffective, especially in the longer term, because much of the available effort is placed into avoiding the work, or into finding means to offload it onto others, rather than in actually getting the work done. It’s also highly addictive, because although it provides a short-term illusion that work has been done, in reality the work still remains to be done – leading to a tendency to spend even greater effort in avoiding the respective work. This is especially true where the work to be done is inherently personal, such as relational or aspirational/’spiritual’ work – which by definition cannot be done by anyone else, no matter how much we might attempt to offload it onto others.

This matters to enterprise-architecture because, at a systems-level, much of the inefficiency or ineffectiveness of a system can be traced to various forms of power-against, whether between real-people, machines or other systems. It also represents an increasing order of organisational risk, since level-2 (e.g. silo-mentality) may easily decay to level-1 (e.g. destructive competition); and most level-1 problems can easily become ‘undiscussables’, with so much (self-)dishonesty tied up in them that they can only be addressed indirectly. (The SEMPER model describes various methods for how to tackle this type of ‘wicked problem‘, by the way.)

The colloquial term for level-1 or power-over is ‘violence’, and for level-2 the usual term is ‘abuse’. We’re perhaps at some risk here of wandering off into very dangerous political-territory, but there’s a lot we can learn from looking at the use of those terms in the more usual social contexts. One such context that I’ve worked in professionally quite a bit over the years is domestic-violence (DV). For safety’s sake, I’ll elide over the political minefield associated with DV, other than note that the usual approaches show quite a strong similarity with IT-centric ‘enterprise’-architecture: a small subset of the actual scope, but purporting to be the whole of the scope, and hence acting as a block to actual progress, much as described for EA in the post about ‘Crossing the chasm‘. As with IT-centric ‘EA’, only a small amount of rework is sufficient to capture the actual underlying generic themes, at least to the level where it’s possible to work past the conceptual block. For example, a rework of the ‘standard’ Duluth model on DV yields a set of categories of ‘power-against’ that are directly usable in any business context, entirely separate from any preconceived notions of gender or the like:

• Coercion and threats
• Intimidation
• Economic abuse (price/value mismatches, etc)
• Emotional abuse
• Using privilege (such as hierarchical ‘authority’)
• Using isolation (dysfunctional misuse of ‘need to know’ etc)
• Using children (e.g. issues around disadvantaging parents etc)
• Using others (third-party abuse)
• Minimising, denying and blaming

(A tenth category, ‘Sexuality’, is less actively present in business contexts, but does occur in issues such as sexual-harassment and ‘glass-ceiling’ / ‘glass-floor’ issues – which, by the way, do affect both sexes, not solely women alone.)

There’s a lot of detail that needs to be worked-through there, as can be seen in this reference-sheet. (That’s the ‘neutral’ version of the revised-model which, with a certain amount of thought, can be adapted to assess inter-system and even inter-machine relationships. If you specifically want to assess interpersonal or intrapersonal issues, take a look at the ‘both-gender’ version, with matched gender-pronouns used throughout: be warned, though, that many people may find it personally challenging, even though it’s the only approach that actually works.)

Putting it into practice

So, how do we put all of this into practice in enterprise-architecture? And why should we bother anyway?

It matters, because the core of EA really comes down to a single phrase: “things work better when they work together”. What we’re after in practice, in any EA design, is overall effectiveness – ‘efficient on purpose’, if you like, though there’s actually a bit more to it than that. And functional power is what will be needed to achieve that aim; whilst dysfunctional ‘power’  - power-against, the believed ‘ability to avoid work’ is what will always act against it.

So, in short, we need to maximise functional power, in any and every form, within our EA analyses and our EA models. Conversely, we need to find ways to minimise dysfunctional ‘power’, in every form in which it occurs – between people, between machines, between processes, services, whatever.

To apply all of this:

Memorise those definitions of functional-power – the ability to do work – and dysfunctional ‘power’ – the purported ‘right’ or whatever to avoid work. Watch for how each of them occur and act out in practice in any EA context.

Memorise those definitions of power-over – ‘any attempt to prop Self up by putting Other down’ – and power-under – ‘any attempt to offload responsibility onto the other without their engagement and consent’. Watch for how each of these occur and act out in practice in any EA context – and remember that any occurrence of either of these will reduce overall effectiveness and increase organisational and/or enterprise risk.

Search always for ways to convert ‘ability to avoid work’ to ‘ability to do work’. (This is closely parallel to Deming’s exhortation to ‘Drive out fear!‘, by the way – and is needed for much the same reasons.) Seek it out in performance-metrics, for example, that currently reward behaviours that are damaging to overall enterprise effectiveness. Seek it out in systems that assert priority over other systems (‘Using privilege’) without any defensible reason for needing to do so. Seek it out in systems that focus only on financial metrics (‘Economic abuse’), without reference to other forms of value that are relevant or important to the overall enterprise. Seek it out, and find ways to restructure the systems towards a functional ‘ability to do work’ relative to the vision and values of the shared extended-enterprise.

Seek always to express our own responsibilities in this: for example, as Nick Gall put it, “Motivating people is more important than modeling them“. And in a practical expression of ‘power-with’, seek always to find ways to collaborate with others – organisational change, knowledge-management, security, safety, quality, strategy and just about everything else – especially where our authority or responsibility overlaps with theirs.

Expect this to be challenging, and highly political, every step of the way.

Have fun?

Update: I forgot to mention that there’s a detailed ‘manifesto‘ on all of this in the business context that I wrote some years back, as part of what became my book Power and Response-ability: the human side of systems. Perhaps take a look? – free download of three-page PDF here, anyway.

But there’s a catch. A big catch. People should not be ‘designed in’ to the architecture.

We can see this well in building-architecture: there, the only case where people have been literally part of the architecture is that of the anchorite in the mediaeval church. An anchorite would choose to be permanently walled into a cell that was part of the fabric of the building, to act as a spiritual anchor for the people’s faith. Somehow I don’t think we’ll be likely to find many people willing to do the same for today’s for-profit enterprises…

Yet in most building-architecture, evidently, people are everywhere. Other than in certain special-cases – the equivalent of an IT-specific ‘enterprise’-architecture, perhaps – the building and its purpose and design will centre around people, about what people do, why they do it, what the building represents to people, and so on. And the equivalent of that ‘people-centrism’ is what we need in a viable enterprise-architecture.

But if we can’t design people into the architecture itself, how do we map people within the architecture? I would suggest four key themes:

• vision and values
• relational-assets
• roles and responsibilities
• capabilities and competencies

Let’s look at each of these in some detail.

Vision and values

Conventional enterprise-architecture is not good at this. TOGAF, for example, uses the term ‘vision’ to mean a particular goal, a description of the desired ‘target state’ of the IT-architecture: it’s one valid meaning of ‘vision’, I suppose, but not much use in relation to people. Business-architecture frameworks are often less helpful: the BRG/OMG Business Motivation Model, for example, describes as a similar ‘future state’ for the organisation, but framed in terms of some kind of ghastly marketing-puff – “to be the best provider of…!” etc – that really is worse than useless in practice.

What’s needed instead is something that provides a permanent stable anchor that is also ‘greater than’ any individual or the organisation as a whole. Daniel Pink refers to this aspect of motivation in his book Drive, as ‘purpose’; this is also the sense of ‘vision’ used in the ISO9000:2000 quality system. In my own EA work – as described in the section ‘Identifying the enterprise’ in the post ‘Context-space mapping with the Enterprise Canvas‘ – I use a distinct three-part structure to define this form of vision:

• a descriptor for the content or focus for this enterprise
• some kind of action on that content or focus
• a qualifier that validates and bridges between content and action

These components may occur in any order, but all of them need to be present. The aim is to identify a short phrase that gives sufficient motivation for anyone involved to ‘want to get out of bed in the morning’ on behalf of the overall enterprise. One particularly clear example is that for the TED conferences, “ideas worth spreading”: ‘ideas’ [content]; ‘worth’ [qualifier]; ’spreading’ [action]. Note that none of these components describe the organisation at such – but do describe the focus, the area of action, and the key value-metrics which define the meaning of ’success’.

Another key point revolves around the crucial distinction between ‘organisation’ and ‘enterprise’ – because although many business-folk view their organisation as ‘the enterprise’, they’re not synonymous. As noted above, vision is also the anchor of the quality-system, and in effect the hierarchical structure of ISO9000 provides us with a useful way to clarify that distinction:

• ISO9000 ‘vision’ :: architecture ‘vision’ – anchor for the overall architecture
• ISO9000 ‘policy’ :: architecture ‘enterprise’ – loose often-dynamic affiliation, bounded by shared values and shared commitment to the vision
• ISO9000 ‘procedure’ :: architecture ‘organisation’ – formal collective entity, bounded by usually-explicit roles and responsibilities
• ISO9000 ‘work-instruction’ :: architecture ‘process’ etc – practical context-specific entity and/or activity

The point here is that the organisation is merely a means to implement the vision, in relation to the broader shared-enterprise within which the organisation exists – its supply-chain, market and broader social context. Other than in terms of its declared or implied responsibilities towards this shared-enterprise, the organisation does not ‘own’ the enterprise at all – another crucial point that is frequently misunderstood in business-architectures and business-models. The organisation does choose the shared-enterprise to which it relates and within which it operates: but having done so, it in effect commits itself to the respective vision and values – and will be assessed by other participants in the shared-enterprise in terms of those values.

Which, in turn, leads us to the centrality of values in enterprise-architectures. The values are what determine ‘success’ within the shared-enterprise, especially in the longer-term. Any for-profit business in a money-based economy needs to ‘make money’, of course, and hence concerns such as monetary profit and ‘shareholder-value’ are of real importance to such a business. But value is rarely measured solely in monetary terms within a shared-enterprise: attempting to measure ‘success’ by monetary profit alone is a sure-fire way to kill the company, especially in the longer-term – hence Michael Porter’s warning that the obsession with shareholder-value is “the Bermuda Triangle of strategy” [PDF], within which companies sink without trace. The point here is that are three distinct sets of values in play in the architecture:

• required values – the values implied by, inherent in and derived from the enterprise-vision, and against which the organisation will be assessed by the broader shared-enterprise
• espoused values – the values that the organisation purports to live by, and against which it and its members supposedly expect to be assessed
• actual values – the values that are actually expressed by the organisation and/or by individuals within the organisation, and against which actions and inactions of individuals are actually assessed, rewarded and/or punished

It is rare for these sets of values to match exactly… in fact there are often glaring disparities between them, often unconscious, yet critical all the same. One of the most difficult and politically-explosive aspects of enterprise-architecture is to map these value-sets, the disparities between them, and the practical implications and risks of those disparities, and identify appropriate means to resolve them. Difficult though it may be, this is probably one of the most essential tasks of a whole-of-enterprise architecture.

Relational-assets

People are essential to an architecture, but should never be included as such within it. To resolve this requirement, we turn to an old computing tactic: we include people not ‘by value’ – as ‘objects’ within the architecture – but ‘by reference’ – via relational assets.

That much-used phrase “our people are our greatest asset!” may be well-meant, but it’s actually an insult of the most extreme form: the only time when people are ‘assets’ is when they are slaves… Instead, the relationship is the asset: the ‘asset’ in context is the relationship that the organisation has with each other player in the shared-enterprise, not those people themselves. (This relationship is what a Customer Relationship Management system and the like should assist in maintaining, yet so very rarely does…)

As described elsewhere, I use a model in which there are four distinct categories of assets that an organisation and enterprise will need to maintain: physical (things), virtual (information), relational (real person-to-person links) and aspirational (person-to-abstract links, such as brands, beliefs, goals etc). Many real-world entities – perhaps most – are composites of these categories: a book or a video-disc, for example, is ‘thing that carries information’, and hence a composite of physical and virtual. These four categories of assets are fundamentally different: an object is alienable (if I give it to you, I no longer have it), whereas a virtual entity is non-alienable (if I give it to you, I still have it). Failure to understand these fundamental differences within an architecture will guarantee failure of that architecture: hence, for example, the inherent invalidity of media-industry business-models which try to ‘control’ virtual-only information-assets (digital-only books, music and video) as if they’re still in their old partly-physical ‘pre-digital’ form.

People are physical, yes: but they are not solely physical ‘things’, ‘units’, ‘consumers’ – even though too many employment-models and business-models seem to treat them that way.

People may be represented by information, yes: but they are not solely information-entities – even though too many CRM- and HRM-systems seem to treat them that way.

People have relationships with other people. These relationships – relational assets - are direct, and personal: they cannot be exchanged directly with anyone else.

People have relationships with emotive ideas, often an idea or image as proxy for person or for a desired memory or future (such as the relationship with someone deceased, or the aroma that brings back memories of another place and time). In a business context, such proxies include logos, brands, and the reputation of the organisation itself. These relationships – aspirational assets - are indirect, but personal: they cannot be exchanged directly with anyone else.

(The latter, by the way, is the reason why the accounting concept of ‘goodwill’, which purports that aspirational-assets can be transferred directly from one ‘owner’ to another, is only marginally better than outright fraud. In essence, the ‘goodwill’ is mostly dependent on ‘relational inertia’, the perceived physical, virtual, relational and aspirational effort required to abandon one aspirational-asset relationship and shift to another. For aspirational-assets, much of that inertia is linked to reputation, but is also dependent on the medium through which the relationships take place: for example, the ‘friction-coefficient’ of a web-relationship is far lower than that in an in-store relationship – hence one of the key drivers for the ‘dot-com bomb’…)

In the architecture, we would map real-people primarily via relational-assets and/or aspirational-assets. (Virtual-assets describe those links, but not are those links: the distinction is important! Likewise some aspects of architecture may need to describe people as if ‘things’ – for example, space-allocation in buildings, or workspace in manual-processes – but should never describe people as things: that distinction is important too…) Within the architecture, an employee ‘is’ a complex composite of many ‘sub-assets’, such as (physical) space-allocation plus (virtual) employee-record plus (relational) personal-relationships with peers and co-workers and managers and suppliers and customers and many others, plus (aspirational) sense of belonging to a team, a business-unit, an organisation, the core enterprise of the organisation, and any number of professional or other affiliations. Of these, only the physical-asset is directly exchangeable: everything else is linked to the person, not solely to the organisation.

Note also that relationships have to be maintained at both ends: the asset exists between the respective entities, not as an attribute of either of them. In effect, it exists as an asset only as long as it is maintained: and all of the usual asset-lifecycle concerns also apply to relational and aspirational assets. A customer may choose to drop the relationship – and there is nothing that the company can do directly to prevent that from happening. If strong person-to-person relationships are maintained, a customer may well follow a (former) employee to a new employer: a classic problem in consultancy organisations, for example. If there is strong aspirational attachment to a brand, there is likely to be a ‘customer revolt’ if the brand is changed or dropped’ – as in the Gap logo debacle.

So in Enterprise Canvas, for example, we would model all of these as assets of the respective primitive and/or composite types, using the single-row extended-Zachman frame:

Each asset also implies a responsibility for the asset-lifecycle of that asset: its creation, usage, maintenance and, where appropriate, destruction. (The latter being a key reason why not to refer to real-people as ‘assets’!) Responsibility, in turn, implies real-people to enact those responsibilities, either directly or by proxy. Real-people to whom the organisation must link via relational- and//or aspirational-assets… This might seem somewhat circular at first, but it does make sense once we think about it for a whole!

Roles and responsibilities

It’s not just assets that require responsibilities: in principle, every entity in the architecture should have its own ‘responsible owner’. And each of those owners, as above, is a real-person – and ultimately can only be a real-person, not an IT-system, a machine or an amorphous ‘the organisation’.

The architecture should link to each respective ‘responsible real-person’ via a relational- and/or aspirational-asset. In principle, each of these relationships should be modelled and mapped within the architecture itself. In practice, it’s huge job, requiring near-real-time links to HR systems and/or CMDB systems – so it’s unlikely anyone would attempt to do it for everything. Instead, we need to use the architecture to identify the criticality of each entity, assigning priorities appropriately. For true business-critical items, we do need to be able to identify the ‘responsible owner’ at all times; for the rest… well, we do what we can, really, guided by the usual trade-offs of diminishing-returns.

Within the organisation – where legal or similar boundaries of responsibility will usually apply – these responsibilities should typically be mapped in terms of a RACI-matrix:

• responsible – the one person formally responsible for the entity (legally a ‘delegate’ of the CEO or equivalent, who in principle has ultimate responsibility for everything in the organisation)
• assists – any number of additional people who do the actual work related to that responsibility
• consulted – other people whose work and/or responsibilities may be directly affected by any changes or status-issues associated with this entity (two-way engagement)
• informed – other people who have a ‘need to know’, such as for indirect impact on their work (one-way broadcast)

We may also need to note other categories of (non-)responsibility such as not-consulted or not-informed – for example, a senior manager who does not need to be distracted by detail-level status-changes.

Responsibilities often overlap – another set of concerns that should be identifiable via the architecture models. Some of the overlaps occur at intersections of different architectural layers or domains – for example, a senior manager having responsibility for a ‘business capability’, with operations-layer managers responsible for the underlying infrastructure, the equipment in use, the data and applications referenced by that equipment, the processes that traverse through that equipment, and then also managers for other ‘cross-cutting concerns’ such as security, safety, quality and cost-assurance. A layered architecture helps to clarify the respective boundaries of authority and responsibility, and the need for negotiation at each of those boundaries and intersections.

Note too that for many entities – especially business-critical ones – the effective ‘ownership’ and RACI-matrix will need to change often, with handovers of responsibility at each change of shift, or whenever the respective people go on vacation, or whatever. Particularly important are changeover-events such as joining or leaving a company: in particular, each person’s individual RACI-matrix must be reassessed and reassigned on exit from the organisation. Also important on exit is to ensure that relational- and aspirational-assets with each departing person are maintained as far as practicable: this forms a key link in the chain of long-term knowledge-management, and also many organisations find that maintaining an ‘alumni’-style relationship with former staff pays real dividends in both directions.

And note also that responsibility cannot simply be assigned to someone: it must be accepted and taken on by that person at a personal level, a personal commitment, otherwise it’s likely to fall back to ‘responsibility’ in name alone, with a concomitant increase in organisational risk. Again, this is where explicit linkage to enterprise vision and values can present real value to the organisation, because it provides an overt anchor for aspirational-assets – a sense of belonging, of being engaged in something ‘greater-than-self’ – that make it that much easier to take on and enact those personal responsibilities.

Finally, a person should be assigned responsibility only if they have the skill and ability – the competence – to take it on. This, however, can lead us into yet another architectural hornet’s-nest: modelling of capabilities and competencies in relation to real-people.

Capabilities and competencies

In principle, capabilities are straightforward: they represent the ability to act on specific types of assets in specified ways. That’s how we model the capabilities of machines and IT-systems, anyway. We can then map these capabilities to what might call a set of skill-levels:

• rule-based – simple real-time action and response, predictable and repeatable
• algorithmic – more complicated contexts affected by multiple dependencies, delays, etc, yet still follow a straightforward true/false logic (‘hard-systems’)
• pattern-based or heuristic – true complex contexts that are highly-sensitive to starting-conditions (wicked-problems etc), in which outputs can become inputs to the next iteration, and which often follow modal-logics (‘soft-systems’)
• principle-based – near-unique, unique or true chaotic contexts in which there is no actual repeatability, hence new ‘solutions’ most be created in each case

A given capability at a given skill-level is described as a competence. Capabilities are linked with functions, and appropriate business-rules (‘decisions‘) and the like, to present services that respond to specific events at the respective locations, as per the single-row extended-Zachman above. For a service-oriented architecture, we then assert that everything is or provides a service, and that any service could in principle be done by any combination of machines, IT-systems and real-people. This makes it possible, for example, to design IT-services such that real-people can take over in the case of IT-system failure or other disaster-recovery, using essentially the same service-interfaces, and then hand back to IT again when recovery is complete. And to a significant extent, I’d argue that this is how we should model our architecture – hence the design and structure and usage of the Enterprise Canvas model-type (two-page reference-sheet here).

Simple enough in principle: perhaps not quite so simple in practice…

To me, there are two key concerns here: the order/unorder dichotomy, and the Taylorist trap.

The order/unorder dichotomy mainly relates to that stack of skill-levels.

The first two skill-levels – rule-based and algorithmic – fall into what we might call the ordered domains – a world in which simple true/false logic is assumed to apply, and in which doing the same thing will always lead to the same result.

The other two skill-levels – pattern-based and principle-based – operate more in the unordered domains, in which modal logics (should, could, may, might, etc) usually apply, where simple true/false logic can be dangerously misleading, and where doing the same thing often leads to different results or doing different things can end up with the same result.

Machines follow physical rules; IT systems usually follow a predefined true/false logic. That makes them very suitable for use in the ordered-domains – but not good in the unordered domains. Therein lies a key-cause of many ‘IT disasters’ such as IT-centric ‘business process-reengineering’: they tried to use IT as the sole means for managing unordered contexts for which, almost by definition, IT was not suited.  The two usual ‘solutions’ were either to try to control everything, planning for every possible exception or eventuality; or simply ignoring anything that didn’t fit with the system’s predefined logic. Both approaches were all but guaranteed to fail, and usually did so, expensively…

On the other side, real-people can be very good at handling the inherent uncertainties of unordered contexts. What they’re often not good at is ‘following the rules’ in a predictable, ordered way. Itr’s possible to force people to work in an ordered manner, but it’s rarely efficient or effective; and – as Daniel Pink and others have shown – if the work involves any cognitive skill, performance drops off rapidly if attempts are made to motivate people in any ‘mechanical’ way.

So although in principle service-handovers and the like should be a straightforward mapping between machine- or IT- capabilities versus human-capabilities, in practice it’s not quite that simple: we need to design-in appropriate balances for ‘unorder’ on the machine-side, and misplaced ‘order’ on the human side. Some nice architectural complexities there…

Then there’s the Taylorist trap – or rather, several of them.

One trap is implied from the order/unorder dichotomy. In classic Taylorism, manual-workers are essentially viewed as components in a machine: there is no concept of skill, only a strict separation between ‘brain’ (management) and ‘brawn’ (workers). All thinking and reevaluation is done by managers or by external ‘experts’ (e.g. the dreaded ‘time and motion man’). Every exception is escalated to the next higher level. Since (as lated pointed out by Deming) the knowledge needed to solve a problem usually occurs at the point o contact with the problem, one of the key results of Taylorism is for the problem to be ‘escalated’ to people who may have greater theoretical knowledge, but less and less practical knowledge, creating an apparently ‘unsolvable’  problem.

Another trap arises from the same source: treating workers as components in a machine. The competencies required by the work itself are listed in a job-description or the like, in much the same way as a functional-specification for a machine. Anyone who has the required competencies is likely to be capable of doing the work. This approach is extremely useful when designing services that may need to be implemented in different ways with different combinations of manual, machine and/or IT-based ‘actors. The catch is that in effect it defines a ‘lowest common denominator’, and takes no account of skill-levels or abilities beyond the specified minimum. The result is that innovations and creativity – such as would definitely be required in a ‘chaotic’-type near-unique context – may be all but designed out of the overall service, reducing the effectiveness that could otherwise be available. Treating workers as mindless ‘robots’ is also dangerously demotivating, frequently with dire results for overall performance and for overall risk. Hence although in the Enterprise Canvas we would typically model the nominal capabilities and competencies required for a given service, we need to take care to ensure that the relational-asset aspects are also full addressed in any enterprise-models that involve real-people.

One further Taylorist trap is rather more subtle and insidious, and relates to the process via which skills are learnt. The classic learning-process involves working the way through the skill-level stack, first tackling repetition of ‘simple’ rule-based tasks and problems, then moving on to ‘complicated’-type tasks, and thence to true complexity. But when all simple tasks and even many complicated tasks are automated, there is no routine method to learn complex-level skills: instead, people are somehow expected to take over when the machines fail – in other words, when ‘order’ moves into ‘unorder’ – but without any means to understand what’s actually going on. This trap is explained in more depth in the post ‘Where have all the good skills gone?‘

In summary, we can and probably should model capabilities and competencies within an enterprise-architecture: however, we do need to beware of the various traps and problems that can arise if we’re careful how we do this, and how we apply it. You Have Been Warned?

The next post will explore another key aspect of any architecture that includes real-people: the problem of power. Watch This Space?

See also:

• A question of who
• Principles, values, ends and means
• People, assets, relationships and responsibility
• The meaning of service
• Context-space mapping with Enterprise Canvas, Part 5: Service content
• Enterprise Canvas, Part 6: Models
• Economics as enterprise-architecture
• New economics models – what impact on enterprise architecture?
• Capabilities, reference-models and job-descriptions
• Zachman and capability
• More on Zachman and capability
• On EA ‘owners’ and primitives
• Rethinking Zachman – the ‘Who’ column

• bergmart: I’m more and more convinced that it should be forbidden to show ArchiMate diagrams to business management #entarch #bizarch
• EAatWork: @bergmart why should it be forbidden to show archimate diagrams to business management?
• blomr: @bergmart @eaatwork Depends on level/type of business management + depends on complexity of the diagram(amount of (different) objects&rel’s)
• ArchiTool: @bergmart I’m intrigued to know why. Are they infrastructure models?
• bergmart: @ArchiTool I’m not talking about infrastructure models but business / information models
• ArchiTool: @bergmart @EAatWork Then maybe break them down into smaller focussed viewpoints?
• bergmart: I think we need to find other ways. Business Model Canvas is a much friendlier way or use the operating model stuff of Ross
• ArchiTool: @bergmart Right. Higher levels. And ArchiMate could be used in addition for more detail if needed? // Hmm….Business Model Canvas in Archi? Development of the Sketch View? #archimate #bmc
• bergmart: @ArchiTool Yes, in the architecture & engineering communities
• ArchiTool: @tetradian Synchronicity is nudging me toward possible Bus Model Canvas & Ent Canvas implementation in Archi. Reading up on it now…
• tetradian: @ArchiTool aiming to have 2 new posts for you soon: models as anchors for discussion/decision-records; roles of tools in toolset-spectrum
• tetradian: @bergmart ‘don’t show Archimate diags to biz mgmt’ – in my experience showing them BPMN diagrams is an even worse idea…
• bergmart: @tetradian Can imagine!
• EAatWork: @bergmart too complex from a modelling language point of view (archimate) or too complex because the diagrams are too complex?
• bergmart: @EAatWork Both
• EAatWork: @bergmart  I think that the concepts are ok but the  different relation types are causing the confusion (look all similar) for bizz mgmt
• tetradian: @bergmart @EAatWork with BPMN diags, prob was need to translate from abstract to concrete – so overlay Archimate rigour w visual images?
• ArchiTool: @bergmart But with Business Model Canvas approach maybe managers are concerned with $$$$$$ rather than architecture and internal workings?
• ceri: @tetradian @bergmart To quote @wilm on Tuesday “never ever show the entire model to ANYONE, only the view that is relevant to them” #jiscfsd
• bergmart: @EAatWork Yes, the arrows are the main problem. The concepts are ok.
• bergmart: @ArchiTool Nice  with Business Model Canvas is the combination of $$$$ with coherency.

As mentioned in that back-and-forth above, I’ve had painful first-hand experience of what happens when we show ‘formal’-type EA diagrams to executives, as described in this quoted from my book Real Enterprise-Architecture:

In our first attempt to show the true value of [a proposed] logistics early-warning system, we made the mistake of showing the process-flows [to the executive] in the form of a Business Process Modelling Notation diagram. BPMN is great for the formal modelling rigour needed by software engineers, but not for a board-level presentation: the blank stares and silence from round the table sent us away in shame.

For the next meeting, we redrew the diagram, with the exact same process-flows, but replacing the bland BPMN boxes with clip-art pictures of trucks, conveyor-belts, fork-lifts, sorting-machines, delivery staff. This time it clearly made sense: we gained our go-ahead.

These senior people weren’t ‘stupid’: when we explained it in their own terms, they understood straight away what we were aiming to do. The problem before was that they didn’t have time to learn an abstract language and translate it back into their concrete world. As architects, we did need that level of abstraction: but it was also our responsibility to each audience to do the translation from the abstract into the everyday.

This is going to be a long one… sorry…

In effect, what we need from our toolsets for enterprise-architecture and the like is strong coverage of several different spectra of requirements:

• from formal rigour to freeform, for capture, for display and (in some cases) for execution
• capture, edit, moderate, maintain, analyse, explore, search, cross-link, describe, display
• architecture- and project-governance, requirements tracking, version-control, architecture-dynamics (change over time)
• from binary-logic (true/false, linked/not-linked) to modal-logics (must, may, can, could, conditional/contextual, ‘known-to-work’ etc)
• broad range of model-types, including industry-standard models and metamodels and user-defined types (metamodel/metametamodel edit etc)
• single-user to meeting to team to group to business-unit to whole-organisation and beyond, with security and access-controls to match
• enterprise-wide repository to desktop to laptop to tablet to handheld, with different user-interfaces and user-experiences for each
• any and every possible stakeholder-group, in every discipline in the enterprise, and at every level from front-line operations to executive and external stakeholders
• support for full range of the order/unorder spectrum, and for the full range of the development lifecycle from ‘messy’ uncertainty to executable models
• full support for lifecycles ranging from milliseconds to decades or centuries

No existing toolset comes anywhere even remotely close to covering that full range of requirements, and I suspect it’s probable that none will ever do so: the scope is just too large to handle in one go – especially as that ‘one toolset’ would need to run within several fundamentally-different user-interface paradigms. Which leads us to another crucial requirement:

• able to share and ’round-trip’ information and models between toolsets

What we actually have at present we could perhaps summarise as follows:

• ‘enterprise’-scale toolsets [examples: IBM/Rational System Architect, Troux Metis, ARIS, MEGA]: typically based on a large repository, preferably though not always with strong version-control; usually based on a software-architecture/software-development metaphor, often capable of presenting ‘executable’ models (database, process etc); usually strong on rigour but with poor or non-existent support for the freeform experimental stages of architecture-development; broad range of model-types; some support for metamodel-edit, though usually proprietary/’own-standard’; limited to no support for governance, project-tracking and/or requirements-tracking; limited to poor support for architecture-dynamics; limited to no support for modal-logics; strong support for one-way publishing but weak to non-existent support for ‘social-media’ style multi-way collaboration
• ‘team’ toolsets [examples: Orbus, Sparx Enterprise Architect, Alfabet, Abacus Avolution, BizzDesign Architect, Essential]: often either a scaled-down ‘enterprise’-style toolset (using a simpler underlying repository) or scaled-up ‘desktop’-style toolset (expansion of repository, or addition of multi-user features to an otherwise single-use system); usually strong on rigour, poor on freeform; more limited range of model-types (sometimes only one); usually some metamodelling, with very strong support in some toolsets (e.g. Avolution, Essential); limited to no support for governance etc, for architecture-dynamics, or for modal-logics; variable publishing capabilities (sometimes only to desktop software), almost always one-way only
• ‘solo’ toolsets [examples: Archi (Archimate), Visual Paradigm, single-user versions of Sparx or BizzDesign]: basic single-user repository; usually only one or two model-types or sets (e.g. UML, Archimate); in some cases little more than repurposed or re-badged software-design tools; often very limited publishing facilities; otherwise similar to ‘team’-type toolsets
• non-repository toolset (office software) [examples: MS Powerpoint, MS Visio, MS Excel, Apple Keynote, Google Apps, Prezi]: still the most commonly-used ‘toolset’ applications for enterprise-architecture, yet in many ways the least appropriate toolset-type; usually single-user, sometimes sharable; no repository, hence no entity-linkage between diagrams (hence high probability of duplication, mis-versioning etc); emphasis is on diagrams rather than semantics; functionality is split between different applications (e.g. diagramming on Visio, governance or cataloguing on Excel); no metamodelling as such; awkward mix of pseudo-rigour (i.e. rigour in appearance only) yet only limited support for freeform (constructed slowly from uncontrolled image-objects)
• manual drawing [examples: whiteboard, flipchart-pad, sketchbook, 'drawing' application on tablet-computer or smartphone]: by far the most commonly used tools, though rarely acknowledged as such; allows true freeform exploration; no repository, no verification of standards-conformance, usually no linkage to any other model, usually no publishing as such, often no permanent record
• media record [examples: video recording, audio recording, photos]: record of live discussion-session, seminar or workshop; often paralleled with yet not actually linked to manual-drawing; captures decision-making process and resultant decisions in raw form (i.e. ‘raw data’ of the architecture); provides key component of ‘models as decision-records‘

Which, collectively, still only covers part of the overall requirements, in a very fragmented, disjointed way. Not good.

And, to anyone who’s done this kind of  work in practice, one of the more obvious yet painful points is that at present very little of this content can be transferred from one toolset to another… Some toolsets will allow import of some subsets of this information, but successful ’round-tripping’ between toolsets is almost unheard-of. Oh well.

Then there’s the different user-interface/user-experience types:

• repository, text-oriented [examples: Essential, DOORS, metamodel-editors within toolsets]: emphasis is on development of metamodels, ontologies and other rigorous metadata-structures, and capture of models, requirements etc via text-based forms
  • advantages: precision, formal rigour, often automatic verification and completeness/correctness-checking
  • disadvantages: often too much emphasis on ‘correctness’ and not enough on real-world modal-logics, also often accurately described as ‘user-hostile’…
• repository, diagram-oriented [examples: most 'enterprise-architecture' toolsets]: emphasis is on development of diagram-based models using entities captured and maintained within the repository; diagrams constructed by linking repository-entities, and relationships between those entities, in the terms specified by some formal notation (Archimate, UML, BPMN, Zachman etc)
  • advantages: as for text-oriented repository, though also often seemingly much easier to use and ‘read’
  • disadvantages: often no support for real-world non-binary logic (i.e. modal logics), leading to illusory ‘certainty’; modelling also often driven by assumptions from model-type (pictograms, visual notation etc) rather than by underlying entity (which in principle should be independent of its representation); still much harder to ‘drive’ than a simple diagramming tool, yet easily mistaken for the latter, leading to an unusable repository riddled with incompletions and thesaurus-errors (duplications, synonyms, heteronyms etc)
• ‘precision’ diagramming/text [examples: MS Visio, MS Powerpoint, Dia, Sketch, MS Excel]: emphasis is on visual and/or textual representation rather than underlying semantics; diagrams constructed from a ‘palette’ of visual objects, usually with no inherent underlying semantics; user-interface is ‘mouse’-type, with emphasis on precision placement; text is structured (e.g. spreadsheet) yet ‘standalone’, without linkage to controlled metadata-driven repository
  • advantages: looks good, relatively easy to ‘drive’, easy to ‘read’, allows for purported modal-logics and for any required exceptions; presentation may be in any required form; often the simplest way (or even the only viable way) to present technical information to a non-technical audience
  • disadvantages: no inherent underlying rigour, frequently delivers spurious sense of certainty; often (and easily) mistaken for proper repository-based diagramming etc
• freeform drawing/text [examples: bitmap and vector-graphics editors, text-editors, whiteboard, pen-and-paper]: emphasis is on speed of capture, without regard to rigour; user-interface is usually ‘touch’ type (stylus or fingers)
  • advantages: real-time or near-real-time; requires little to no prior technical knowledge to operate the tools
  • disadvantages: easy to do at a basic level (with mediocre communication at best), but requires real skill and experience to capture essence and meaning, especially at high speed; no inherent structure – rigour, metamodel etc depend entirely on knowledge and skill of user
• voice/gesture [examples: voice-recorders, voice/text recorders, video, Wii, Kinect]: emphasis is on capture of unstructured and often tacit cues and content in real-time
  • advantages: user-interface ‘disappears’, or is entirely automatic after setup; minimal knowledge required to operate
  • disadvantages: as for ‘freeform drawing/text’; content often requires separate transcription to enable re-use
• consumption-only [examples: web-browser, report-browser, smartphone app, text-to-voice]: emphasis is on search and reporting from existing information; variety of user-interfaces/user-metaphor types, but user-experience is similar in each case; interaction usually limited to query-creation and/or metadata-append (tagging etc)
  • advantages: can be configured to fit into almost any format – screen, e-book, paper, smartphone, media-player
  • disadvantages: no content-capture, and often no means to provide feedback or review

When we put all of this together, what we need is not so much ‘a toolset’ as a toolset ecosystem in which all of the parts and roles and user-experiences can mesh together as a unified whole, that allows and supports both structured and unstructured information, that can span the full truth/value spectrum from true/false logic to if-but-perhaps-maybe, and in which the best use is made of the strengths and limitations of each platform.

Is that such a big ask? I don’t think so: all that’s really standing in the way is a lack of imagination in how to tackle inherent uncertainty and non-binary forms of ‘truth’, and the determination to push through the shambles of proprietary pseudo-’standards’ that stand in the way of proper information-interchange.

For example, consider for enterprise-architecture the following as one option for a bridge that could cut across just about all of the toolset-categories above:

• consistent underlying standards-based metametamodel (equivalent to MOF) or metametametamodel (equivalent to ORM) that supports modal-logics (models, model-types and metamodels built on this are exchangeable with any system that uses the same underlying core)
• lifecycle-managed repository for all entities
• full support for ‘unidentified’ entities that do not belong (or belong only partially) to any metamodel or model-type)
• explicit separation between entity and its representation (also allow any number of representations for any entity in addition to defined defaults)
• full support for Zachman-style concept of ‘primitives’ and ‘composites’ (‘primitive’ as associated with single cell in a given taxonomy-matrix, ‘composite’ may bridge multiple cells and/or aggregate multiple primitives, etc)
• full support for modal-logics (such as the MoSCoW set)  in reference-frameworks etc
• Prezi-style scaling for model-diagram development, to permit literal ‘drill-down’ into models
• capture, embed and linking of unstructured media, especially via tablets, handhelds and similar devices: freeform drawing (e.g. as per Adobe Ideas), video, photo, audio, synchronised audio/notetaking/drawing (e.g. as per AudioNote)
• consistent user-experience across all interface-metaphors (mouse, stylus, touch, gesture)

And to come back to the point where we first started: about not showing Archimate models to executives. What we need instead to show them is something is freeform, concrete, visually descriptive in everyday business terms – but that underneath that surface view actually is an Archimate model (or BPMN model, or UML model, or whatever), with full version-controlled, verified formal rigour. Far too many existing toolsets rigidly link their graphic modelling to the notation of a specific model-type: what we should be doing instead is sticking strictly to the principle of separation of concerns – that the representation of an entity is merely a transitory, optional attribute associated with that entity. And that’s doable if we build the metamodel in such a way as as to embody that separation of concerns – and not entrap each entity into a single predefined form.

All of that list above is doable with what exists right now: all that’s missing is the interchange-standards to make it work, and the willingness of enough people to sit down and do it. (My own coding skills are way out of date, unfortunately, but I’m happy to help in any way I can.)

I’ve written quite a lot on this over the past few years – for example:

• Meanderings on metamodels
• ‘Bindedness’ in metamodels
• More metamodel stuff (pointer to metamodel descriptions on ea.openfutures.org/OsEATools )
• A call for collaboration on enterprise-architecture
• Big EA, Little EA and Personal EA
• Executable enterprise-architecture
• Time for open-source enterprise-architecture?
• On EA tools and paper trials
• Next-generation toolsets for enterprise-architecture?
• Enabling enterprise-architecture conversations

So how about it, folks? Is anyone else actually interested in this?

• the internal enterprise-architect
• the external enterprise-architect

They’re both focused on ‘the architecture of the enterprise’, but it’s important not to mix them up, because they require different temperaments and different approaches to business-relationships.

The internal enterprise-architect is actively responsible for the ‘enterprise DNA’ of a single organisation. They typically either report direct to the CEO (who has the ultimate authority and responsibility for that ‘DNA’, but in practice probably doesn’t have the time to do much about it), or else are attached to the CEO Office or a senior-level strategy-group.

The key point here is that, to use Kevin Smith‘s term, this enterprise-architect role acts as “the glue between strategy and execution” – which means that they need direct person-to-person relations with people at all levels and in all domains of the organisation and enterprise. Developing these relationships takes time, and a lot of time at that – often ten or more years in a typical large organisation. Hence the best people for this kind of role are those who’ve “come up through the ranks” and built a personal network on the way, or – even better – have grown up with the company, “have been in from the start” or suchlike. (The CEO can and should do the enterprise-architect role for a start-up, but once the organisation grows beyond about a dozen people the role will typically need to be part of someone else’s explicit responsibilities, and once we get past the crucial social-network boundary of Dunbar’s number – roughly a dozen-squared – it needs to be a distinct role in its own right.)

This role is about how the idea of enterprise-architecture – that “things work better when they work together, with efficiency, with elegance, on purpose, in practice” – is expressed throughout this organisation, in terms of this organisation’s business-purpose.

The external enterprise-architect (or consultant enterprise-architect) is actively responsible for promoting the ‘idea’ and practice of enterprise-architecture itself. It’s important that they maintain their independence as much as practicable, though for practical reason many will work via the auspices of a consulting-organisation of some kind. Their person-to-person relations are with other architects – again in many different domains and at all levels, but across many different organisations and enterprise-types.

Their primary role is practice-refresh for in-house enterprise-architects: they help to keep the overall architecture up-to-date on methods, practices, and frameworks, and help to lift the architecture-maturity and architecture-capability of the internal team. They also act as external peer-review, to reduce the risk of an insular and often destructive ‘groupthink’ developing within an organisation. This role too requires many years of experience, but this experience will have been gained across multiple disciplines in multiple organisations and, preferably, multiple industries.

The internal enterprise-architect deals with architecture in depth; the external enterprise-architect deals with architecture in breadth. An organisation’s architecture gains most from an appropriate balance of these two roles – not one or the other, but always some of both.

The worst possible combination of these two roles is unfortunately that which many large consultancies try to promote: full long-term control of an organisation’s enterprise-architecture by an external consultancy. An enterprise-architecture works right at the core of an organisation and enterprise: hence assigning responsibility for the organisation’s DNA to an external party is not a good idea… You Have Been Warned!

The comment that started it off was from Ric Phillips. (I’ve trimmed it slightly, but you can see the original here.)

The innovations that led to mini-computers led to the increasing importance of information processing based on the technology’s ability to capture and model transactions (atomistic events). It really did change the nature of work and organisations and made a new kind of information available.

It wasn’t really the advent of PCs that changed things. If the information about the world that could be stored in them and used had not changed radically they would have simply replaced the niche occupied by terminals. But they allowed people to simulate sheets of paper and type writers. And spreadsheets – which were existed prior to software and were done on very large sheets of paper. Later came sound files, photographs, building designs, industrial machinery, complex electronics (like audio mixing decks) and a thousand other things that are now simulated in software.

In this wave computers became personal productivity tools. The changes to how personal productivity expressed itself in our lives when assisted by the new ‘virtual’ things PCs could provide is what changed our jobs, our professions and be extension our lives.

The internet started out as an extension of publication and communications models that already existed. But (in this case much more slowly that in previous transformations) our activity on the internet started to capture large amounts of information that previously wasn’t subject to computation – social information, information about opinions, subjective value, and what we might call (tentatively) knowledge.

There are intersecting trends (consumerisation for example). But mobile computing, ubiquitous data, web 2.0 and so on are all converging to create a new domain of information – information that allows us to model and manipulate in computers new and extremely complex things. Once again this will transform organisations. But this time maybe even whole societies.

I don’t see this as an impending disaster. Our world is changing again. As a strategic profession EAs need to get their heads around this. We are leaving the era of ‘information processing’ and ‘ICT’ and entering the era of social computing and Knowledge Technology.

Reading it again, I now realise that this critique has completely missed the point: all it’s doing is extolling the virtues of each of the transformations in technology, yet seemingly ignoring any possibility that there might also be vices associated with those virtues. Yes, each of those transformations are real and valuable to some context, and that is indeed a key driver for change. Yet the change itself is not the risk, and neither is the technology: it is the dependence on that technology that creates the risk.

So, as I put it in my response, I strongly agree that “mobile computing, ubiquitous data, web 2.0 and so on” are not in themselves an impending disaster. The same applies to their initial impact on organisations and “maybe even whole communities” – in general I see those impacts as desirable, even if certainly not something we can ‘control’.

What does worry me is what happens next. As an EA I’ve spent many months at clients tracking down all those small private-to-a-workgroup spreadsheets and databases and log-files and the like that were a) business-critical and b) unmaintained, undocumented, not backed up, inherently fragile [such as trying to use MS Access as a multi-user database, which it was never designed to do], unregistered, and in many other ways a real business risk. Whenever some key person changed jobs, or a single hard-drive failed, or a sysadmin triggered an automated application-upgrade, or any other of a myriad of seeming-trivial events, that business-unit would literally lose that part of its mind – and an entire business-process, affecting an entire cross-functional workstream, would grind to a halt until someone could work out what had gone missing and how to set up yet another kludged workaround.

When the business-application is non-critical, kludges usually don’t matter: it’s how people learn, it helps get things done, and it’s exactly what ’shadow-IT’ is for. The new mobile technologies and the like are brilliant for this – just as spreadsheets and single-user databases were (and still are). Everything’s fine as long as they’re essentially used in the same way as Lego bricks or a Meccano set or the like – a ’serious toy’ that can be used to knock out a quick prototype to test out an idea, or perhaps even to keep around as a vaguely-useful tool and talking-point. And as long as they’re used for that kind of purpose, it shouldn’t matter much when they do fail – especially if we can use that failure as a way to learn what to do differently next time. In other words, we accept failure as part of the deal – it’s ’safe-fail’.

But don’t try to use a ’serious toy’ for anything that’s business-critical. It’s not inherently wrong, but it’s simply not ‘fit for purpose’: they’re not robust enough, resilient enough, agile enough, secure enough, and so on – which means that as a system we cannot set them up to ’safe-fail’ in such a context. Sure, you could use Lego to build a house (it’s been done), or Meccano to build a bridge (that’s been done, too), but the effectiveness of doing so is questionable at best, especially over the longer term.

It’s the ‘-ilities’ that usually matter most in architecture. The functional requirements for a system are usually much the same at any scope or scale, but the qualitative or so-called ‘non-functional’ requirements are what will usually make or break the system in practice. Building an IT system that can handle half a dozen strictly-sequential requests in half an hour or even half a minute is relatively trivial; building one that can handle thousands or even millions of parallel, interleaving, fragmented, potentially-incomplete requests every second is not trivial at all; and yet the functional requirements are essentially the same. That’s the difference between a ’serious-toy’ prototype, and serious engineering with serious architecture and serious service-management and support behind it.

What we have right now in mobile-computing, ubiquitous-information and cloud is a whole bunch of serious-toys desperately pretending to be more than they are, and – more worryingly – being sold and used as if they’re more than they are. Sure, the function is there – but that’s easy. It always is. Getting them beyond that ’serious toy’ stage is not easy – and because it’s hard work to get there, it hacks into the short-term profits, too, so it’s not exactly popular amongst the money-obsessed.

So we have here all the ingredients for a ‘perfect storm’: more and more of individual people’s lives and livelihoods being placed onto platforms that are inherently unstable and unsustainable, because little or none of the work to make them stable and sustainable is as yet in place or even in progress. If you’re not already seriously worried about what will happen when large chunks of our society literally lose their collective mind and memory through the failures of these kludged-together toys, you’re not thinking hard enough about the architecture of the enterprise… 

The lessons of history are plain to see, and it’s also plain to see that the level of unaddressed risk has been raised each time, with even the earliest-period risks still not fully addressed even now. You Have Been Warned?

It’s a story as old as the IT department: New technology arrives in the market, it makes some type of work easier to accomplish, the business asks for it, and IT reacts and delivers it. Not always however, and usually somewhat slowly. It was this way with PCs, it was this way with the Internet, and now IT is faced with what is turning out to be a veritable perfect storm of technology and social change. …

Today’s highly mobile, social cloud has set everyone’s expectations for how easy, powerful, and simple IT can be. The genie will never be put back into the bottle.

For once I’m going to stand firmly on the side of the IT-folks on this one – because no matter how wonderful this looks right now, this is not good news at all. Looking at this with a futurist’s eye, I’m wondering how long it will take before we wish we could put the genie back into the bottle… because what I’m seeing here is a full-on disaster-in-the-making. Or rather, a double disaster-in-the-making, given how much this will interact with the ongoing disaster that is ‘cloud-computing’…

One of the first lessons any futurist learns is to look back at history, to seek out any equivalent occurrences in the past. And the blunt fact is that we’ve been here before… not just once, but several times already. Each time that we came back to the same place – if perhaps from a slightly different direction – it’s clear that the fundamental lessons were not learned, in fact were wilfully ignored; and each time it took a lot of effort, a lot of skill, and a lot discipline, to tidy up the mess – just in time for the next batch of overly-excited idiots to trash the place all over again. This is the dirty end of Gartner’s ‘hype-cycle’: someone has to tidy up the mess. And yes, “it’s a story as old as the IT department”, because in every case so far, that ‘someone’ has been the much-derided IT department – and also enterprise-architecture, in its broader sense, beyond IT alone.

Go back sixty years or so, to the first beginnings of mainframes and ‘big computing’. Watch the hype-cycle at work: slow adoption, then a huge take-off in ‘data-processing’ (we didn’t get round to calling it IT until quite a bit later). It will solve every business problem! Control the world! Unlimited information on tap, right here, right now! Except it wasn’t quite as simple as that… turns out it was a lot of work to get standards happening (COBOL, the IBM-360 architecture, and so on), and then all the boring stuff about requirements, governance, maintenance, data-cleansing, service-management…

Twenty years later, it’s the mini-computer boom. It will solve every business problem! Now even medium-sized businesses can control the world! Unlimited information on tap, right here, right now! Except that it wasn’t quite as simple as that… turns out it was a lot of work to get standards happening (the C language, the Digital PDP-series architecture, and so on), and then all the boring stuff about requirements, governance, maintenance, data-cleansing, service-management…

Ten years later, we get the microcomputer revolution. It will solve every business problem! Now you too can control the world, right here on your desktop! Unlimited information on tap, right here, right now! Except it wasn’t quite as simple as that… turns out it was a lot of work to get standards happening (disk-formats, file-formats, data-architectures, the IBM-PC architecture, and so on), and then all the boring stuff about requirements, maintenance, data-cleansing, service-management…

Yup, you’ll be seeing the pattern here. The exact same sequence applied to the rise of the internet ten years later, the web five years after that (with a merry little hiatus called the Dot.Com.Bomb), the rise of cloud over the past few years, and now the rise of Hinchcliffe’s mobile IT or ‘CoIT’. In every case, there’s the same wild hype, the initial push from outside the IT-department (as ‘shadow IT’) which gets the basic idea going to point where it’s usable.

(And to be fair, if that push hadn’t happened, those new developments would probably never have been usable: as Hinchcliffe implies, it’s actually quite rare that innovations arises from within the IT department itself. Because that isn’t it’s job: IT’s real job, unfortunately, is to tidy up the mess that will inevitably follow…)

In every case we see the same exuberance… then the slowly-dawning awareness that it isn’t quite as simple as that. It turns out that there’s a lot of work that’s needed in order to get standards happening – otherwise the new ‘revolution’ turns out to be something that can’t be shared, which means that the whole thing fizzles out quite quickly because we need that sharing to happen. We need clear standards for hardware, software, data-architectures, information-architectures, interchange protocols and much more besides. We need distinct disciplines around requirements, governance, maintenance, data-cleansing, quality-management, service-management and a whole swathe of other areas. And all of those, it’s now clear, need to allow for customisation, agility, security, versatility, adaptability, resilience and the like – none of which are easy to balance with conventional ‘control’-style disciplines.

So here I am, looking at the rise of Hinchcliffe’s ‘CoIT’ – particularly cloud-computing and mobile-apps. And what I’m seeing is an architectural disaster waiting to happen, if not unfolding right before our eyes:

• security – where is it? does it exist at all? (I’ve seen lots of hype and promises, but not much reality as yet)
• file-formats – half the iPad apps I’ve seen seem to embed their data actually within the app itself – they don’t even have a file-format other than perhaps plain-text or unstructured PDF
• interchange-formats -if they have a file-format at all,  most of the apps seem to rely on unpublished proprietary file-structures with no means to enable exchange between different apps, whilst cloud-providers will often deliberately make it difficult to exchange, so as to enforce ‘lock-in’
• escrow – information-lifetimes range between seconds and decades – yet no-one seems to be thinking beyond a year or more at most, and no-one at all seems to be planning for what happens when a cloud-provider or app-provider goes bust – which they will, often (over the long-term at least), and often very expensively
• system-standards – where are they? do they exist at all? – we seem to back in the worst days of early microcomputing, where just about every man-and-his-dog-in-a-garage could and did create an entirely different architecture for everything, often intentionally incompatible with everything else

I could go on… and on… and on… there’s no shortage of other nightmare-level architectural risk-factors that aren’t being addressed at all. Other than by the much-maligned IT-department, that is (who unfortunately tend to be able to see only the IT-related risks, which represent only a relatively small proportion of the whole); or by the few enterprise-architects who actually do think about whole-of-enterprise scope (and who are mostly derided, by the hype-merchants and their ilk, as doomsayers who’ve lost the plot). Not funny… Oh well…

Yes, it’s true that the excitement (or the oft-forlorn hope that it will finally be better this time?) is what gets people going to create new ideas; so yes, the exuberance does matter. Hence, in turn, I suppose, the hype does matter too. And safe-fail experiments are also always a good idea, because they show us where things will break but without causing much damage in the process. ‘Safe-fail’ can get quite extreme, too: for example, think of the buildings in a fireworks-factory, with very solid walls, very lightweight roofs – because when you know there’s a high risk that things can go badly wrong, you can indeed design for that fact. Yet there are also many types of structures that we can’t allow to fail: anyone who’s lived through a major earthquake or major storm-event will know that fact firsthand… Architecturally we need to be able to tell the difference between those two extremes, and design accordingly.

Yet that’s exactly what’s not happening here with cloud or CoIT: architecture of any valid kind, it seems, has all but been abandoned in the usual wild rush towards The Next Best Thing… So might it not be wise to take a brief pause for thought at this point, before we rush headlong into yet another insanely-expensive IT-disaster? Or is that too much to ask of anyone whilst the hype is in full flow?

• miket0181: @tetradian IBM’s CBM isn’t new. I think it’s at least 5 years old…
• operninha: @tetradian aqui na empresa contratamos a IBM e eles usaram o CBM (“here the company hired IBM and they used the CBM”)
• seabird20: @tetradian I have seen CBM in RFPs for at least 5 years. Original work 10+ yrs. ago. Takes a while to get to rank and file though
• richardveryard: @miket0181 @tetradian … IBM’s CBM came sometime after my 2001 book on Component-Based Business http://tinyurl.com/23gelj7

So yes, I was wrong on that: badly wrong. A critique about outdatedness that would have made sense if the model had been ‘new’ just looks peevish and petty when it isn’t…

Even the critique about the structure of the model is barely fair. Sure, Stafford Beer’s Viable System Model has been around since at least the mid-1970s, but the number of people who knew about it and were applying it in practice (and I actually could include myself amongst them by the mid-1990s) was and still is very small. It’s better-known these days, and its value and importance is much better-understood; but at the time the Component Business Model was devised, I would doubt that anyone in that IBM team would have even heard of it, let alone known why those kind of whole-of-system cross-checks are so crucial. The critique would definitely be valid for an equivalent model developed now, but most of the current knowledge on whole-of-enterprise impacts simply wasn’t available a decade ago. And whilst critiquing a (relatively) old model on the basis of current knowledge is valid enough, it’s only fair to do so when it’s clear when that fact of history is acknowledged – which I didn’t, because I didn’t know it was that old.

I know why I screwed up: after five years of constant struggle against IT-centrism in ‘enterprise’-architecture, I’m now seeing management-centrism promoted as an ‘improvement’, and it’s frustrating as heck… The fundamental point in all enterprise-architecture is that there is no centre – everywhere and nowhere is ‘the centre’, all at the same time. In true whole-of-enterprise architecture, making anything ‘the centre’ – IT, finance, management, processes, security, whatever, even the business-organisation itself – will guarantee failure of the architecture over the longer term. When I saw the Tweet that triggered this, I thought it was yet another example of this lethal mistake. So I over-reacted.

In my defence, I did check the IBM site with some care. But the annoying point there is that there are no dates on that part of the site – nothing to give any clue as to when the material was posted, or its probable vintage. (Compare that to, say Apple or Microsoft, where just about everything has a ‘Last Updated’ timestamp.) I looked quite hard for anything there that would give me any clue as to the date. What I didn’t do, though, was search elsewhere – and yes, that was a mistake too. So I misread the implication of the Tweet, and mistakenly assumed the model was new – after all, it still says so, several years later…

Moral(s) of the story:

• Fact-check everything, via multiple sources – not just the ‘official’ site for the respective information
• If key metadata-items such as dates are missing, fact-check elsewhere again, and treat any implied derivatives (e.g. ‘new’) as suspect
• Frustration is fine, and often all too understandable, but don’t let it rule the roost – engage doubt before pressing the ‘Send’ key…

Overall, though, the blunt fact remains that yes, I did indeed screw up there. Mea culpa…

[Update: this post was a reaction to a tweet I received yesterday, but Mike T. (@miket0181) tells me that the IBM CBM described here isn't new, in fact is apparently some years old, so my complaints on that regard are unfair. (Doesn't help that IBM don't put up any dates on their website-posts.) On that part, yes, I ought to apologise, and do - see 'How to screw up in one easy lesson...'. Yet the core critique still stands: it's not a complete model, and potentially is dangerously misleading if used as the basis for a business-architecture. That's my view for now an' I'm stickin' to it, anyways. ]

A couple of weeks back, as part of the ‘Patterns‘ section in the Enterprise Canvas series, I put up a an example of a variant of the Canvas which I said was definitely dysfunctional, all but guaranteed to be ineffective, and definitely not recommended – a kind of Taylorist-style model of the organisation and its (non-)relationship with its business-ecosystem:

I said explicitly that it was a stereotype, almost a parody – a guide as to how not to view an organisation, with quality-management and coordination subsumed into ‘management’, and rigid separation between the organisation and its broader shared-enterprise.

I was quite certain that no-one would be daft enough to try to model any real organisation in that way.

I was wrong.

Welcome to IBM’s new Component Business Model, where the organisation’s business-world is partitioned into just three layers: Direct, Control, Execute:

I’m going to have to be rude here, and describe this as a kind of ‘bastard child’ of Taylorism and TOGAF, combining many of the worst features of both and not many of their best. The one good item, and a definite improvement on TOGAF, is that the model does explicitly include People as well as Process and Technology:

Using the Component Business Model methodology, our consultants identify the basic building blocks of your business. Each building block includes the people, processes and technology needed by this component to act as a standalone entity and deliver value to your organisation.

But beyond that? – well, let’s compare it to Stafford Beer’s Viable System Model, which I regard as the minimum requirement for whole-of-business modelling:

• system-5, ‘policy / purpose‘ – uh… might be tucked away somewhere in IBM’s ‘Direct’?
• system-4, ‘outside / future‘ – sort-of in IBM’s ‘Direct’, but no reference to ‘outside’?
• system-3, ‘inside / now‘ – yup, right there in IBM’s ‘Control’ – lots of it
• system-3*, ‘monitor / audit‘ (including overall quality-management) – nope, not a sign of it – presumably squeezed into IBM’s ‘Control’?
• system-2, ‘coordination‘ – nope – no sign of it anywhere
• system-1, ‘operations‘ – yup, that’s IBM’s ‘Execute’ – probably…

In terms of the Enterprise Canvas model above, all it has is Staff-Management (what should be the guidance-services, but all scrunched up together in a nearly-unusable way), Line-Management (the Value-Management cell, blown up out of all proportion to its actual relevance) and, uh, Everything-Else…

In other words, there’s probably less than half of what’s needed to make sense of the organisation – but presented as if it’s the whole of it, much like TOGAF’s hopelessly-IT-centric model purports to be ‘enterprise’-architecture.

The four other worked examples are slightly better, but still dangerously incomplete: a Taylorist manager’s-eye view of the business-world, without any clue as to any of the glue-functions that hold it all together. You’ll also note that each one of those examples has a very different structure in its ‘horizontal’ axis – but no indication at all as to how it’s derived. Presumably only IBM’s own consultants could be considered competent to understand the ‘magic sauce’ needed to do this, and the rest of us mere mortals may do nothing else but bow down in awe?

What’s also irksome is that IBM have the temerity to present this as something ‘new’:

IBM’s Component Business Model is a new way of looking at your business. It represents the entire business in a simple framework that fits on a single page. It is an evolution of traditional views of a business, such as business unit, function, geographic or process.

Fact is that this is nothing ‘new’ at all: okay, it might seem new to IBM, but not to just about anyone else in ‘the trade’. We were doing it more than half a decade ago in Australia Post – certainly 2004, and probably earlier. It was only a Visio hack, but in business terms it proved straight away to be one of the most valuable artifacts from our Business Transformation team: just about every single manager in the whole organisation grabbed hold of their own copy and placed it, much annotated, above their desk. Since then I’ve done one or more of these models for just about every one of my enterprise-architecture clients: you’ll find a couple of (de-identified) examples in that VSM slidedeck referenced above, and in probably half of my other TOGAF-conference presentations over the past few years. I even published the instructions on how to build an ‘organisation-on-a-page’ map, complete with Visio templates, on my Tetradian Books website some two or three years ago. Aleks Buterman and his colleagues have had their own generic version – which they call an Enterprise Architecture Capability Map – up on their website for almost a year now. And it’s even built into some of the EA toolsets, such as Troux Metis, and, I believe, IBM’s own System Architect, and again has been so for years. So what’s ‘new’ about it? Nothing, frankly – other than the fact that IBM have finally cottoned-on to what the rest of us already knew anyway.

And I hate to think how much they charge for this ‘new’ approach… a lot, no doubt…

Sorry, folks, but I’d have to say I’m underwhelmed at all of this. Seriously underwhelmed. Oh well.

Bah.

The classic Taylorist solution has been to emphasise extreme sameness: to force everything – and everyone – to be the same, because it keeps things simple, controllable and easily replicable. But we now know that it’s too simple to work well with the real complexities of the real world. In fact it only ‘works’ as long as we can maintain the delusion that it does work: and whenever it fails – which eventually it always does – there’s a tendency to collapse into complete chaos. Which is not much of a ‘solution’ at all.

Yet to focus only on uniqueness all but takes us back to a pre-industrial age where everything is custom-made, everything is different, nothing is actually designed to work with anything else, there are no possible economies of scale, and no certain means to communicate with each other – all of which would seem to be the antithesis of architecture. Which is likewise not a good idea.

Clearly there is an architectural tradeoff there: and hence we need something – some conceptual framework – to help us tackle it.

For at least the past half-decade, and probably longer – see, for example, Andrew Johnston’s 2005 article ‘Architects: Masters of Order and Unorder?‘ – enterprise-architects have turned to Kurtz & Snowden’s Cynefin framework for help on this. For many of us, the Cynefin categorisation of Simple [aka 'Known'], Complicated [aka 'Knowable'], Complex and Chaotic has proven extremely valuable, such as for identifying structural themes and potential problems in conceptual misalignment. One example of the latter, as Dave Snowden has also often pointed out, is the misuse of Simple-domain techniques such as Six Sigma: by definition, these depend on very high degrees of repeatability – literally millions of identical events, in Six Sigma’s case – yet there are frequent attempts to apply them in contexts that have little or no repeatability (‘Complex’ or ‘Chaotic’ respectively, in Cynefin terms), which makes no sense at all.

Beyond that basic categorisation, though, attempting to use Cynefin in enterprise-architecture can itself be problematic, particularly where we need to tackle inherent uniqueness. The explicit focus of Cynefin, and Snowden’s Cognitive Edge consultancy, is the application of techniques derived from complexity-science to inherently-complex areas such as policy. (Which, from a cross-map of Cynefin to the ISO9000 quality-standard ‘stack’, is exactly what we should expect: ‘Complex’ maps to the ISO9000 ‘Policy’ layer, in the same way that ‘Simple’ maps to ISO9000′s ‘Work Instruction’.) Yet whilst Cynefin uses the sciences to tackle complexity, what we also need in enterprise-architecture is some means to use complexity to tackle ‘chaotic’ uniqueness – which is not the same at all. Therein lie some serious problems – and some potentially-serious mistakes – if we try to re-use Cynefin concepts in contexts for which it was not designed.

I’ll admit that I’ve probably made some of those mistakes myself. Over the past couple of years I’ve written a number of articles on Cynefin on enterprise-architectures, which made a lot of practical sense to many people, but unfortunately also led to some extremely unpleasant arguments that I have no wish to revisit. What’s become clear to me over the past few months is that the beguiling simplicity of the Cynefin categorisation can blind us to the fact that although its descriptions of the Complex and Complicated domains are essentially the same as we would use for context-space mapping in enterprise-architectures, its definitions and usage of the Chaotic and Simple domains are fundamentally different to those that are needed to tackle uniqueness and sameness in architecture. It’s like comparing a cross-head screwdriver with a flat-head one: at a cursory glance they may seem to be the same, and it’s clear that they are related in the sense that they have similar functions – but in practice they’re not interchangeable, and trying to use them as such will cause a lot of frustration and possibly a lot of damage too. Not a good idea.

So I’d like here to explore what aspects of Cynefin can be used in enterprise-architectures, how and why and where it should not be used, and what we could use as an alternative in those contexts. [I perhaps need to emphasise here that this is not a critique about Cynefin itself, but solely about certain (mis-)uses of Cynefin in enterprise-architecture.]

This again will need to be quite long – apologies – but at least this time there’ll be a fair number of diagrams to break the verbiage into more manageable chunks.

What is the Cynefin framework?

The basic ‘Cynefin framework’ is a way to categorise and interpret different types of cause-effect relationships in a given context. Starting from an initial position where we cannot identify what type of cause-effect relationships exist (the domain of ‘disorder’, shown as the unlabelled grey region in the centre of the diagram below), we categorise the relationships as follows:

The right side is about order, certainty and repeatability; the left-side is about ‘unorder’, where things do not repeat in an ordered way. (Note that this early diagram from Johnston’s ‘Masters of Order and Unorder’ article uses the original labels of ‘Known’ and ‘Knowable’ for the two ‘order’ domains; for several practical reasons, explained elsewhere by Dave Snowden, these domains are nowadays labelled ‘Simple’ and ‘Complicated’ respectively.)

One important point is that Cynefin is not a simple two-axis matrix – hence the curved lines separating the domains. But it’s also important to note that the purpose of Cynefin is to describe and act on themes that apply primarily in the Complex domain – contexts in which “cause and effect are only coherent in retrospect and do not repeat”. The ways in which we would act on those themes are familiar to architects: patterns, perspectives and the use of exploratory experiments to probe for appropriate options. This contrasts with the ‘Knowable’ or Complicated domain, in which it’s assumed that if we can identify all the factors in play – the analytical or so-called ‘hard-systems thinking’ approach – it will lead us automatically to the single repeatable ‘right answer’ to any problem.

That specific contrast is extremely important in enterprise-architectures, because there’s a very common misperception – especially amongst IT-oriented folks – that ‘complexity’ is an exact synonym for ‘very complicated’. All we have to do to remove complexity, it’s said, is to break it down into smaller, simpler parts, and the complexity will go away of its own accord. The Cynefin framework makes it clear that this is a fallacy – particularly when the system includes real people in any real, non-machine-like way, but also in many, many types of ‘network-effects’ and the like. The blunt fact is that systems that are designed solely on ‘ordered’ principles – Simple and/or Complicated – will inevitably fail in any real-world context: hence the importance of something like Cynefin.

What’s different about architecture?

As I understand it, the primary purpose and use of Cynefin is in development of policy, and related programmes of action around large-scale themes involving complex issues that engage or need inputs from a large number of people. It’s been used a lot recently in the defence and anti-terrorism contexts, for example, including new ways to work with narrative-enquiry and the like.

Enterprise-architectures do indeed work on some themes that are similar to those: yet the real point is that we’re also strongly focussed on individual responses to specific, unique requirements – which is not the same as Cynefin’s focus on policy, which is more about the collective than the individual. In architectures we also make very strong use of uniqueness in the ideation process; and for IT-architectures especially, we have a strong need for ‘solutions’ that are simple enough for IT-systems to operate in real-time.

And real-time itself is another strong focus: for much of what we do, we don’t have time for analysis or experiment – and yet whatever solutions we build have to be effective, to work well in accordance with the real needs of the context. That too is different from the needs of policy: recent work by Cognitive Edge has brought complexity-based tactics much closer to real-time, but it still is not actually operating in real-time – time is important there, but it’s not the real focus.

So there are two key distinctions here:

• real-time is a key concern in enterprise-architecture, yet much less so in Cynefin
• the Chaotic and Simple domains (real-time unorder and order) seem all but undesirable in Cynefin, yet are central to enterprise-architecture

Cynefin provides a (very) good match for enterprise-architecture’s needs in the Complex and Complicated [Knowable] domains; but it does not fit well in the Chaotic and Simple [Known] domains. Using Cynefin for enterprise-architecture in the latter types of contexts would not only yield misleading results, but would also be a misuse of Cynefin itself. Not recommended.

So there’s a strong suggestion here that we do need something else that looks much the same as Cynefin in the non-time-critical contexts, and is compatible with it in those domains – yet takes a different approach as we move closer to real-time.

Dynamics in context-space

The same themes come up when we consider Cynefin’s dynamics – the pathways via which we move between conceptual-domains. To illustrate this, I’ll again use a diagram from Johnston’s ‘Masters of Order and Unorder’ article:

As shown by the orange pathways, most new ideas arise in the Chaotic domain. The dead-ends of some of these lines in the Complex domain and the continuation of others (via blue lines) into the Knowable/Complicated domain aligns well with the classic sequence of scientific development: Idea (Chaotic) to Hypothesis (Complex) to Theory (Complicated) to Law (Simple). The back-and-forth between Chaotic and Complex (path 7), Complex and Complicated (paths 4 and 5), and Complicated and Simple (path 3), are also typical – and necessary – in scientific development. Yet the only links shown between Chaotic and Simple – the key areas of concern for real-time enterprise-architectures – are failure-modes: the collapse of the over-simplistic (path 1) or the collapse of access to new ideas (path 2).

This too is exactly what we would expect in social contexts where a probably-spurious sense of certainty (Simple order) is privileged over everything else: the cycle of scientific-development dead-ends at ‘law’, preventing any further progress or adaptation until there’s some form of catastrophic collapse (a ‘paradigm shift’, as in Kuhn’s ‘scientific revolutions’).

Snowden & Kurtz describe the opposite collapse (path 2 in the ‘dynamics’ diagram above) as ‘imposition of order’ – such as the point at which a dictator ‘takes control’ to enforce their own brand of order upon apparent social chaos. In more mundane form, we see this pattern very often in enterprise-architectures and project-management, such as in the ‘cart before house’ attempts to implement a quick pre-packaged ‘solution’ without giving sufficient space to identify the actual requirements. This particularly occurs when perceived time-constraints remove the apparent option for the more conventional requirements-to-implementation loop via pilot-projects (Complex) and analysis (Complicated).

So the standard Cynefin model above shows us only the dysfunctional pathways between Chaotic and Simple, without any functional paths as such. It also tends to describe both of these domains as context-spaces to avoid: the Chaotic because it is too unstable for any science-based cause-effect approach to sensemaking, and the Simple because it’s so often too simple – and hence dangerously simplistic. This limitation of standard Cynefin is perhaps not surprising, because the main focus of the model is on relationships and differences between Complicated and Complex (such as ‘hard-systems’ versus ‘soft-systems’), where time is important but not a key factor. Yet it does also mean that we’ll need an alternate yet related approach for architectures at near-real-time scales, that does describe both dysfunctional and functional paths between them, and that gives us a richer and perhaps more balanced view of the roles of each of the domains.

An alternate frame

One option for an alternate approach comes from classic Jungian psychology. At first glance it’s just a simple two-axis matrix: ‘truth’ (order) versus ‘value’ (unorder – or related to Cynefin’s ‘unorder’, anyway), and outer-world/’objective’ versus inner-world/’subjective’:

Yet we should also apply that key Cynefin insight that each of these domains is simply a means to ‘make sense’ of the totality of what’s going on and what we’re experiencing, and hence that there’s also a ‘none-of-the-above’ domain that precedes any form of sensemaking, which we might label ‘reality’ or ‘disorder’. And we can also derive other labels from elsewhere in the Jungian-related traditions, of which probably the most useful set are the Artist (idea/Chaotic), Technologist (hypothesis/Complex), Scientist (theory/Complicated) and Priest (law/Simple):

One of the most important points that this adds to the standard Cynefin set (Chaotic, Complex, Complicated, Simple) is that the Priest domain is not solely about the real-time simplicity of law – it’s also about certainty. Each of the domains has its own form of ‘certainty’, and its own form of passion too; but in the Simple domain, the Priest is also a passion that certain things are true – and that everything else is not ‘true’. And it’s that passion that seeks to prevent change – even when change is needed. It needs things to be simple, to be certain, to be clear, to be defined forever in some kind of ‘divine Order’ – and it guards that order and certainty with a deep, unyielding, passion that has what we could accurately describe as a religious intensity.

This is a side-effect of the push to simplify, and is also a key driver in the process by which, as Dave Snowden has again commented, valid Simple-domain techniques such as Six Sigma or BPR (Business Process Reengineering) are always at risk of becoming ‘cult-like’. Techniques become cult-like when their proponents forget that the real world cannot all be reduced to Simple rules: it will always also include components that are Complicated, Complex and/or Chaotic. It’s not just that “order is real”, as Dave Snowden asserted in one of our unfortunate arguments, because it is equally true that “unorder is real”: both are true, for a given sense of ‘true’ – which in a sense also means that there are circumstances where each of them is ‘not-true’. What often matters most, perhaps, is which of them is useful in any given context; which has more value.

And so on, and so on – which can quickly recede into a mind-bending kind of recursion. One of the real dangers that I’ve seen in techniques that place their main roots in science – such as Six Sigma or BPR, or Cynefin itself, for that matter – is that the push for Simple laws can sometimes cause a withdrawal into ’scientism’, a bizarre ‘religion of science’ that is actually closer to an antithesis of science itself. (The antics of many so-called Skeptics represent some of the more infamous examples; likewise people who consider that they’ve experienced a ‘conversion’ to Dawkins-style atheism.) Often key cues arise from language used: phrases such as “must be” or “it’s obvious” are common warnings of a collapse into the over-Simple. The recursion here means that we need to use the frame to review the frame itself: so Cynefin, for example, is oriented towards the Complex, but its roots are in complexity-science (a Complicated-domain modality), and its dependence on scientific ‘law’ (Simple) means that it must always guard itself against becoming overly-simplistic – and also against the resultant ‘natural’ tendency to regard any new ideas and interpretations (the Chaotic domain) as ‘heresy’, a ‘threat’.

Yet the Simple-domain (the Priest) is also one end – the ‘ordered’ end – of a spectrum that implements action in real-time, where there is no time available for thought and reflection. No matter how Complex or Complicated the behaviour may appear be, most real-world processes are anchored in Simple rules: that’s one of the key foundations of all the sciences – including chaos-science – and we ignore that fact at our peril. It applies just as much in business too: hence the everyday foundation of every quality-system is the plain, ordinary, everyday work-instruction, describing the routine choices in routine action. Most things are Simple; most things, most of the time, do indeed need to happen in much the same way – and we ignore that fact at our peril, too.

The Simple goes wrong – often badly wrong – when it makes the mistake of assuming that ‘most’ is the same as ‘all’, or that ‘most of the of time’ is the same as ‘always’. The point here is that there are times when we need to move to the other end – the ‘value/unorder’ end – of that real-time spectrum: and we need to move there as an intentional choice, rather than as the enforced result of an unintended collapse.

For example, every time we start a new project or a new architecture-cycle, we need to intentionally block out everything we believe to be ‘true’, so as to be able to identify the real needs and real requirements in the context. When we first start on a new design, we must deliberately face ‘the unknown’ – otherwise the best we’ll get is another iteration of the known. One excellent description of this process comes from the Indian film director Shekhar Kapur, in a presentation at TED India in late 2009:

When I go out to direct a film, every day we prepare too much, we think too much. Knowledge becomes a weight upon wisdom. You know, simple words lost in the quicksand of experience. So I come up, and I say, “What am I going to do today?” I’m not going to do what I planned to do, and I put myself into absolute panic. It’s my one way of getting rid of my mind, getting rid of this mind that says, “Hey, you know what you’re doing. You know exactly what you’re doing. You’re a director, you’ve done it for years.” So I’ve got to get there and be in complete panic. So it’s a symbolic gesture. I tear up the script. I go on to the set. I panic myself. I get scared. I’m doing it right now. You can watch me. I’m getting nervous. I don’t know what to say. I don’t know what I’m doing. I don’t want to go there.

And as I go there, of course, my AD says, “You know what you’re going to do, sir.” I say, “Of course, I do.”

And … inside … I’m allowing myself to go into chaos because out of chaos, I’m hoping some moments of truth will come. All preparation is preparation. I don’t even know if it’s honest. I don’t even know if it’s truthful. The truth of it all comes on the moment, organically, and if you get five great moments of great, organic stuff in your storytelling, in your film, your film audiences will get it. So I’m looking for those moments, and I’m standing there and saying, “I don’t know what to do”.

Here we deliberately go into the Chaotic – a structured serendipity where we use principles rather than pre-packaged ‘truths’ as our guide. The principles act as a sort-of constraint, but more as a guiding-star to a direction, rather than a boundary or ‘attractor’ as it is in the Complex domain. It creates a space in which the new, the unexpected-yet-appropriate, can happen, can create itself out of nowhere, using us as the means by which it does so – or that’s how it will typically feel, anyway. And the ‘enemy’ here – the ‘giggle-wrecker’ that destroys that bizarrely timeless state of ‘flow’ – is ‘control’, the state of possibly-spurious ‘certainty’ that’s so desirable (and, in many ways, so necessary) in the Simple.

This is very different from Cynefin, in which the Chaotic is essentially somewhere that we might perhaps visit for a brief moment, but otherwise aim to get away from as quickly as possible. Instead, for this purpose, the Chaotic is often somewhere we need to stay in for as long as possible, though it often takes a great deal of courage to rest with the ‘panic’ that this will naturally entail.

More on recursion

Recursion (and likewise its inverse, reflexion – ‘the whole seen within any part’) is a key theme in enterprise-architectures, not least because it can aid in creating simplicity without falling into the over-Simple. One of the problems we do see with careless use of Cynefin in enterprise-architecture is a failure to make appropriate use of the recursion/reflexion pair, in part because whilst recursion is a key pattern that’s used in Cynefin – especially in the Complex domain – it’s not inherent in the framework itself. In effect, the basic Cynefin diagram is often used solely as a Simple frame, as a straightforward single-layer checklist and not much more.

Yet once we introduce recursion as an inherent attribute of everything within the frame, some interesting options open up. [I perhaps need to reiterate that this is not 'standard Cynefin' here, though it is one way in which the Cynefin-style frame can be (re)used.] One example above was the recursion in which Cynefin’s focus on the Complex is based on the Complicated ‘truth’-frames of complexity-science, which in turn are grounded in scientific concepts of Simple laws, which in part loop back to the Chaotic in that there are elements of complexity-science that are inherently uncertain.

Complexity-science and its scientific siblings such as chaos-mathematics provide us with powerful tools to make sense of the uncertain at a larger scale – such as in Cognitive Edge’s work on narrative interpretation and abductive reasoning. But the closer we move towards uniqueness, the individual event, the complexity-science breaks down – or more accurately, moves outside of the scope to which it can appropriately apply. When dealing with inherently unpredictable events such as weather-systems or radioactive decay, chaos-mathematics can quantify the unpredictability, to make the degree of unpredictability more predictable, yet it still does not make any individual event any less unpredictable: we can often measure a fissile half-life, or a mean-time-between-failure, very accurately indeed, yet still have no means whatsoever to predict when an individual atom will split, or when an individual unit will fail. And it’s often those individual events that matter most in the architecture – and certainly so in the ideation or sensemaking processes at the start of a new project.

We can see much the same recursion if we use a Cynefin-like frame on methods and methodologies. For example, we could use it to categorise methods and the extent to which we could modify those methods:

• Simple (‘Priest’): the method is fixed, predetermined, invariant – ‘true for all cases’
• Complicated (‘Scientist’): the method can be configured, in accordance with predefined (Simple) parameters or (Complicated) algorithms or (Complex) usage patterns, etc
• Complex (‘Technologist’): the method must be adapted to the context, and ideally should be self-adapting (i.e. Complex Complex) to each context
• Chaotic (‘Artist’): the method cannot be predicted in advance, and may indicate itself without apparent warning, though pre-seeding with (Complex) patterns to provide direction will usually help

Also notable in many Chaotic-domain methods is the exact inverse of the warning that applies so strongly elsewhere. In the Complicated-domain, it’s (rightly) considered crazy to repeat the same action and expect different results; yet in the Complex-domain, doing the same thing will usually lead to different results, whilst in a true Chaotic-domain doing the same thing will always lead to different results. Hence in the Chaotic-domain, constant repetition of Simple actions – as typified by meditation and the like, or the musician’s endless practice of scales and sequences – will often provide ‘just-enough’ predictability to make the unpredictability usable in practice.

It’s also essential to note that whilst Complex-domain patterns are useful in all skills, ultimately every true skill is personal – in other words, must always in part be Chaotic. We cannot learn on others’ behalf; there is no substitute for personal experience. Which also leads to another key point in every architecture: it’s always in part about values – sometimes collective but also always personal – which are, by definition, ‘subjective’: so we can’t and shouldn’t expect that all of it will necessarily ‘make sense’ to anyone else. That’s one aspect of our work that will always be ‘unscientific’ – yet also a strange blend of scientific law and the science and art of inherent uncertainty.

In the actionable world, order is real, yet so is unorder: a real-world, real-time spectrum that wanders between predictability and unpredictability, engineering and imagination, sameness and difference. We can quantify uncertainty in the enterprise, but we can never make it certain. Getting that balance right across all of those multitudinous interdependencies and layers and complexities is, to me, the real essence of enterprise-architecture.