Tom Graves / Tetradian

The real art of enterprise-architecture, and perhaps its hardest challenge, is in presenting the right level of detail. Not too little, not too much, but just enough.

Just Enough Detail.

To which people will, of course, immediately ask, “Okay, but how much detail is ‘Just Enough Detail’?”. And I’ll have to admit that there isn’t a simple. certain, predefined answer. You just have to kinda know when enough is enough, you know? – which is why it’s more art than science, I guess. And why experience – usually gained by not getting it right… – is so important here.

One thing I do know is that one of the most-quoted answers is usually just plain wrong for this. John Zachman has always said that we need to document everything in ‘excruciating detail’. In a sense, yes, he’s sort-of right, especially if you hold to his metaphor that enterprise-architecture is essentially the same as engineering an aircraft. (I happen to believe that that’s a seriously-misleading metaphor, but that’s another story.) Yet in the real world – even in aircraft-engineering, as I know from much first-hand experience – much of the detail won’t stay the same for long enough to make that ‘excruciating detail’ requirement achievable in practice. Tricky…

Reality is that everything changes, everything moves. And the more they change, the more the demand for ever-more-detail becomes a trap. And when the pace of change itself is accelerating fast – as is definitely the case in most enterprise-architecture contexts right now – the more dangerous that ‘too-much-detail’ trap becomes, and the more we risk falling into it.

Yet on the other side, not enough detail means we won’t have enough of an anchor for meaningful sensemaking or decision-making – so we risk making bad decisions on the basis of too many arbitrary assumptions. That’s not a good idea either.

Hence Just Enough Detail.

The point is that that ‘just enough’ of Just Enough Detail varies all the time, from context to context, depending on who we’re with, what we’re doing, what we’re aiming to do, the type and rate of change, and all manner of other factors. Take this example from one of my favourite ‘show this to clients’ books, Matthew Frederick’s 101 Things I Learned In Architecture School:

There’s actually not much detail in that image. There’s no detail at all of the wall – and yet that’s still enough detail to make out that it is a wall (and probably a white-plaster wall at that). Other than the outline, there’s almost no detail of the woman, or her clothing – and yet it’s enough to get a good sense of who she is, what she looks like. There’s a bit more detail of the church and its dome – enough to tell that it is Brunelleschi‘s masterpiece in Florence – and of the townscape around it. Not much detail, then – and yet that’s all the detail it needs to tell the story. Not too much; not too little; Just Enough Detail.

So, over to you: how much or how little is Just Enough Detail in each part of your enterprise-architecture? How do you show that Just Enough Detail to whoever needs to see the story?

How much does Just Enough Detail change between different layers of abstraction, between different audiences, between backbone versus edge?

How do you know when it’s too much detail, or too little? How do you know when it’s just right? – when it’s Just Enough Detail?

How do you learn this delicate, ever-changing balance of ‘just enough’? From where and in what ways do you learn that balance – without causing too much damage whilst learning it?

Just Enough Detail, always. An interesting challenge, yes?

At least he was open about it, I guess. “Tell you what I’ll do”, he says to my colleague here in Guatemala, “I’ll find you a client, then I’ll sit in, learn everything you do, and then I’ll apply it in my own business. How does that sound to you?”

Uh, no. Not a good idea. Not just because it’s a really bad deal from our perspective, but much more that Reality Department really doesn’t work that way: there’s no short-cut to experience.

Yes, it all looks simple enough – in fact that’s the whole point. A lot of simple visual summaries, and surprisingly simple-seeming methods, too. Yet it only looks simple because we’ve been through a heck of lot of hard work to make it that way. Hard-won experience, won the hard way through years and years of practice in many, many different contexts, getting it ‘wrong’ time and time again, in many, many different ways in order to get it right.

The real trap is that these simple-seeming ideas and methods aren’t simple rules, prepackaged sense-making and decision-making that will always work in every context. These are simple principles, simple guidelines, the kind of easy-to-memorise information that helps decision-making in real-time, in circumstances that are subtly different every time. (See my SCAN posts for more on these distinctions.) If you try to use them as ‘rules’ for inherently-uncertain contexts, without understanding why those principles apply, or how they need to be tweaked every time to match each different context, you’re going to be in real trouble – along with everyone else around you. Not a good idea…

The same often applies even to things that really are ’rules’. Take that example of perhaps the greatest simplification ever made: e=mc². All the core information you need to build a nuclear power-station is right there in that equation: but there’s a heck of a long way – a heck of a lot of engineering-experience – to go from that one equation to building a nuclear-power station that actually works.

Same with everything else, really: simplification is essential, but can also be deceptive – especially when people mistake ‘simple’ for ‘easy’…

Which is also why I get a bit hot-under-the-collar about the current proliferation of ‘certification-schemes’ in enterprise-architecture and elsewhere. Some of them are genuinely valuable, but others – to be blunt – are little better than money-spinning scams, in terms of the actual value that they (don’t) deliver. And the crucial distinction revolves around the role and recognition of experience.

For example, the TOGAF Foundation and Archimate Foundation certifications have real value. They verify that the respective person has a credible command of the terminology and language – a requirement that matters a lot for communication across a dispersed and disparate team.

Likewise the ATAC certifications should have real value, because each explicitly tests practical experience in the respective area.

But unless they’ve changed it in the past year or so, the full TOGAF certification is delivered through the absurdly-inappropriate mechanism of a multiple-choice test. And to my mind, that’s not merely useless, it’s actually worse than useless, because it’s exactly how not to test for the kind of experience that that type of competence requires. (When I did the TOGAF 8 exam some years back, I almost failed because I answered several key questions correctly in terms of real-world experience, rather than the theory-based wrong-assumptions that the test thought were ‘right’.) The result of that kind of pseudo-test is a bevy of people who can wave a certificate around, but have no idea how to do the work in any real-world context.

A good training-course can make all the difference, and the better training-providers do take up some of the slack here. (I’ll wave a flag at this point for John Polgreen at Architecting The Enterprise, who’s been doing sterling work for years on adapting TOGAF for the US-government context.) Yet none of that competence carries through anywhere into the actual test: so unless we know each of the training-providers, we have no way to tell whether a candidate does actually know what they’re doing, or merely that they have a piece of paper to prove that they know just enough to get into real trouble, but not enough to get out of it again.

In effect, right now, the full TOGAF certification is of less real-world value than the Foundation certification – which is both bizarre and sadly pointless. And I’ll hasten to add that I’m using TOGAF here merely as one example amongst many: it may well be that most of the so-called ‘certifications’ in this field are even more meaningless than that. And the results can be seen everywhere in the trade…

In short, it’s a mess.

What we need to be testing for is genuine understanding of a context, and the ability to adapt for uniqueness. And that calls for much, much more than can be covered in a crude multiple-choice test delivered through a mindless machine. Sure, that kind of test is cheap, and relatively easy to administer: but it’s also all but meaningless for anything than foundation-level rote-knowledge. It really does take years of painful practice to develop the experience needed to do this work well: and if this trade is to gain the credibility that it needs, we need to stop pretending that we don’t need to test for that experience.

Time to re-think how we do this, and how we respect this, too: there’s no short-cut to experience.

If it’s not a cycle, don’t call it a cycle.

In the past few days I’ve had a fair bit of struggle to get clients to understand the difference between a linear-sequence with a beginning, a middle and an end, versus a true cycle where the end of one sequence links to or becomes the start of the next.

Cycles are literally cyclic: they’re not just linear sequences, they repeat, often in self-similar ways that are rarely ever quite the same. And the problem is that there are a lot of so-called ‘cycles’ that aren’t cycles at all. Some examples:

• Tuckman’s ‘Forming, Storming…’ lifecycle
• Adizes’ organisational lifecycle
• Gartner’s hype-cycle

At root, these are just linear sequences. For example, Tuckman’s ‘Forming’ stage (purpose) leads to ‘Storming’ (the all-too-necessary-yet-often-avoided people-stuff), thence to ‘Norming’ (planning and preparation) and ‘Performing’ (the actual process of delivering the project). And there it stops: if we’re wise, there’ll also be a final ‘Mourning’ or ‘Adjourning’ phase (closure, completions, lessons-learned), but as far as the individual project is concerned, that’s it. The End – the end-point of a linear sequence.

It’s not a cycle.

To make it a cycle, we need to be able to link the end of one sequence to the start of another: ‘Adjourning’ feeds into and informs the ‘Forming’ of the next project.

Once we have a true cycle, iteration-effects such as complexity and emergence start to appear; continuous-improvement becomes possible; agile self-adapting strategy in a fast-changing environment starts to make sense.

Yet those benefits only become available or visible where there’s a true cycle – not merely a one-shot linear-sequence that happens to call itself a cycle, but isn’t.

Cycles enable visibility of iteration-effects; one-shot linear-sequences don’t. And it confuses the heck out of people that we can have those two very different types of structures arbitrarily assigned the same name.

So if it’s only a linear-sequence, call it a sequence. If it’s a true iterative cycle, call it a cycle. If, like Tuckman’s project-lifetime model, it’s a sequence that can also be linked back to itself to create a true cycle, call it a sequence when it’s a sequence, and a cycle when it’s a cycle. Don’t mix them up!

In short, if it’s not a cycle, don’t call it a cycle. Please?

Also launched at the Integrated EA 2012 conference was my new book ‘The enterprise as story‘:

Full title: The Enterprise As Story: the role of narrative in enterprise-architecture

ISBN: 978-1-906681-34-0

Description:

Most current approaches to enterprise-architecture describe everything in terms of structure. Yet people work better with story than with structure – and people are the enterprise. As we expand the architecture towards a true whole-of-enterprise scope, we need to describe the enterprise as story. Story is everywhere in the architecture – even the enterprise itself is a story.

This ground-breaking book places story at centre-stage for the architecture, itself using a narrative structure to explore the role of narrative in enterprise-architecture. Via business story-structures such as the Market-Cycle, and genres such as We Sell Certainty, it shows how stories underpin every aspect of the enterprise – and how we can use story within the architecture to enhance overall enterprise effectiveness.

Topics covered include:

• how to use story and narrative to assist in sensemaking for architecture
• how to create engagement in the architecture through story
• how to balance structure and story for better business results
• how to identify and use business-story genres to guide overall architecture
• how to change the organisation’s relationships with its ‘anti-clients’ from business-risk to business-opportunity
• how to use story-patterns to identify and resolve strategic business-issues
• how to leverage your own experience to create stronger architecture stories

If you want to create real engagement in the architecture and the enterprise, this is one book you’ll definitely need.

—

You can already order the printed book from Amazon.co.uk or Amazon.com, and presumably most other book-retailers as well.

(Ignore the comment on Amazon about ‘Temporarily out of stock’: Amazon say that for any print-on-demand book that they themselves don’t produce… It’s at most a couple extra days’ delivery-time, that’s all.)

I’ll also be adding it to the book-set deals on Kevin Smith’s Pragmatic EA bookshop: should be set up within the next few days, anyway.

And new - you can now buy the e-book from Leanpub, as a complete set of PDF (portrait-format), EPUB (for iPad, Sony-Reader etc) and MOBI (for Kindle).

I’ll be doing a lot more publishing via Leanpub from now on: not just e-books of the existing books, but also smaller more focussed e-books on topics such as SCAN sensemaking and modelling with Enterprise Canvas. More details on that in an upcoming post, anyway.

‘The enterprise is the story‘ – my presentation from the recent Integrated-EA enterprise-architecture conference in London – is now online on Slideshare:

The slidedeck is just under 80 slides, split into five sequences:

• “What’s the story?” – introducing the idea of story as a way of working within enterprise-architectures, using the example of Carnaval, in Rio de Janeiro
• “A cast of thousands!” - describing the ‘sharedness’ of enterprises and the enterprise-story, again using Carnaval as its example
• “The plot thickens…” - linking story to process and the practical details of the enterprise
• “To be continued…” - exploring the structure of story, and strategic-structures that cause failure of the organisation’s story
• “Every picture tells a story” - a plea for stronger support of story in our enterprise-architecture toolsets

For once, I did a slidedeck that’s more about visuals than words – and it certainly seemed to go down well with the audience, which is always good fun.

The conference is, for me, one of the highlights of the year, because they cover architectures with such an enormously varied scope: most of the attendees are from defence / security contexts or high-reliability areas such as rail-transport or air-traffic control. I put in a a few sort-of visual jokes that I put in specifically for them – which seemed to go down well, too.

I also did a audio-recording, but it’s a bit crackly. I’ll try to clean it up and, if so, attach it to the slidedeck to make a bit more of a standalone presentation.

Share and enjoy, anyway?

The echoes of the ‘naming-problem‘ around business-architecture and the like continue to rumble on, this time via another happy Twitter-exchange with Ron Tolido:

• rtolido: @tetradian just show me the non-IT people that invented #entarch and / or #bizarch
• tetradian: @rtolido we’re in a circular-definition here: what you call #entarch or #bizarch is whatever was ‘invented’ by IT-people… //  crucial problem is that IT-people labelled as ‘enterprise architecture’ to something that wasn’t ‘architecture of the enterprise’ // likewise with IT version of ‘business-architecture’, which _isn’t_ ‘architecture of the business’ // once we sort out that mess, it becomes obvious IT-people did not invent it – but until then, we have those circular-definitions… // non-IT-people: Deming, Boyd, Beer, Alexander, even Taylor, for heavens’ sake…
• rtolido: @tetradian All true! Just pointing to the actual roots of both #entarch and #bizarch . Not saying it’s a good thing per se.
• tetradian: what you’re doing at present is propping up _fundamental_ mistake: mislabelling of ‘IT-view of business’ as ‘business-architecture’ // ‘Open Group Cert in IT-view of Business’ is fine – just don’t call it ‘business-architecture’, because it isn’t! // Data Architecture 101: don’t assign names to things that don’t mean the same as what those things actually are!

And that last point is actually a good idea: apply a bit of bog-standard data-architecture practice to this problem. Let’s look at this whole mess from the perspective of Data Architecture 101:

– Step 1: A core principle: all entities shall be assigned meaningful names or terms – i.e. that the assigned name shall correspond to the ‘natural meaning’ of the entity.

– Step 2a: If a term that is currently assigned to an entity does not match the ‘natural meaning’ of the entity but is not in common use, the updated name for the term shall be promulgated, and action taken to dissuade use of the former misleading-term.

– Step 2b: If a term in common use is currently assigned to an entity but does not match the ‘natural meaning’ of that entity, an architectural ‘waiver‘ or ‘dispensation‘ shall be issued, acknowledging the current usage of that term.

– Step 3: If a waiver is issued, the waiver does not mean that the misleading usage is acceptable, but rather that although the fait-accompli is accepted in the present, all efforts must be made to prevent the misleading-term from becoming further entrenched, and every opportunity taken to promulgate a replacement ‘natural-meaning’ term.

This is basic stuff, the kind of routine data-architecture work I was doing twenty years ago and more. Software-coders do it every day; web-designers do it; database-administrators do it. But not, apparently, the people who purport to maintain the formal standards for this kind of work. To use a certain famous phrase, “this does not compute…”

Let’s look at this in a bit more detail.

First, that principle from Step 1, the notion of a ‘natural meaning’. A term or entity can of course be assigned any name at all: sometimes projects and the like are intentionally assigned misleading names, for security purposes or because they’re being used only as ‘working title’ or suchlike. Sometimes such names do stick, misleading or not: ‘tank’ is a classic example. But in general – especially in a data-architecture or in any part of an enterprise-architecture – an entity should be assigned a name that aligns with its use and function: for architectural purposes it doesn’t help anyone if we decide to use the label ‘Glue Pot’ for a delivery-truck, for example, or ‘Salmonella Breeding Station’ as the label for the cafeteria business-unit. In general, it’s a lot more helpful if we call a spade a spade, and so on.

[We can take this a bit further, perhaps - hence the old adage that "An Englishman calls a spade a spade, but an Australian calls it a bl**dy shovel"... ]

Hence the notion of ‘natural meaning’, that in order to minimise the potential for confusion, things should be named according to what they are or what they do.

And a simple test for ‘natural meaning’ is inversion of the term: if the inversion gives the same meaning as the assigned term, it’s more probable that, overall, the term won’t cause confusion. (There’s a proper grammatical-term for this inversion, but I’ve forgotten it: someone remind me, perhaps?) Take ‘data architecture’, for example: the inversion is ‘the architecture of data’, which in both cases is what actually happens in the practice of data-architecture – so we can be reasonably comfortable that we have something close to ‘natural-meaning’ there. In practice, ‘data-architecture’ is a term we can trust to make sense.

Likewise ‘security-architecture’, as the architecture of security; or ‘brand-architecture’, as the architecture of the relationships and the like between business brands;  or ‘IT-infrastructure architecture’, as the architecture of the infrastructures of IT-systems. These all make clear sense, whichever way round we put it; and keep the same meaning, whichever round we put it.

But when we try this inversion with the supposedly-’official’ usages of ‘enterprise-architecture’ or ‘business-architecture’, it just doesn’t work:

– enterprise-architecture:

• natural-meaning (from inversion): the architecture of the enterprise (i.e. organisation as a whole, plus extensions into the value-network and overall ecosystem within which it operates)
• common-usage in TOGAF, FEAF etc: the architecture of the IT-systems in use within the organisation, with some reference to the usage of those systems within the organisation’s business

– business-architecture:

• natural-meaning (from inversion): the architecture of the business (or, more specifically, ‘the business of the business’)
• common-usage in TOGAF, FEAF etc: anything not-IT that might impact upon IT, organised and described solely in terms of its actual or potential impact on IT

In both cases the IT-oriented common-usage is a very long way from the natural-meaning of the term – which guarantees confusion as soon as we move outside of the narrow confines of an IT-oriented view. And in both cases that common-usage meaning describes only a very small subset of the scope of the natural-meaning – in other words, wherever the IT-oriented common-usage is dominant, it represents a serious term-hijack that blocks visibility of the remainder of the natural-meaning scope.

Which, in any competent data-architecture, would clearly indicate that have a couple of seriously-invalid term-usages here. Which means we need to do something about it. Which brings us to Step 2.

It’s clear that Step 2a can’t apply here, because both of these invalid terms are very much ‘out there’.

Which means that we move to Step 2b: we issue a waiver.

But we do not forget what a waiver actually means here, and what responsibilities it places on all of us, collectively, in terms of action we must take in future to correct the architectural risk. In particular, it does not mean that we simply throw up our hands in the air and say “oh well, it doesn’t matter” – because clearly it does matter, because equally-clearly that usage of the term will not make sense to anyone outside of the ‘in-group’ cabal. Instead, the waiver says that we must take action to correct the fault – exactly as with any other type of architectural fault.

Which brings us to Step 3. What we actually need in this case is the metaphoric equivalent of a full ‘Cease & Desist’ order, to demand that people not only stop all use of this misleading usage of the terms, but take action to correct all materials in which either of those two misleading usages occur. For example, TOGAF would need to be rewritten from scratch: given the natural-meaning, it wouldn’t be allowed to use the term ‘enterprise-architecture’ just about anywhere in the whole document, and ‘Phase B: Business Architecture’ would either cease to exist, or be reconstructed as a proper multi-domain structure, within which ‘the architecture of the business of the business’ is merely one amongst many other domains that can impact upon IT.

Let’s not beat about the bush here: that is what needs to happen. Anything less represents not only only an architectural risk on a major scale, but an ongoing risk whose impacts increase exponentially with every passing day.

Sadly, Reality Department indicates we’re very unlikely to get this – not least because it would require Open Group, CapGemini, Federal Enterprise Architecture, Gartner, Zachman and all the rest to recall every scrap of their past publications on ‘enterprise’-architecture, and rewrite the whole darn lot. But we need to say, and continue to insist, that this is what needs to happen in future. We do not simply allow them to continue promulgating these (and many other) fundamentally-wrong term-usages in the enterprise-architecture space.

That’s Data Architecture 101, as applied in a perfectly straightforward way to current ‘enterprise’-architecture – what the Americans call ‘eating our own dogfood’.

And if we aren’t willing to do it to our own work, why on earth should anyone else trust us to do it to theirs?

Pretty simple, really.

So, whatcha gonna do about it, folks?

—

One separate but related and very important addendum: I am not knocking TOGAF in itself here, nor anything or anyone else in the IT-architecture space. IT-architecture is extremely important, and Open Group and others have been doing sterling work in that space for many years. To my mind, no-one should doubt this, and I’m very happy to sing their praises on that part of the work, and invite and encourage others to do likewise.

All that I’m saying is that what TOGAF etc call ‘enterprise-architecture’ should not be called ‘enterprise-architecture’, for the simple reason that it is not ‘the architecture of the enterprise’.

Likewise the somewhat jumbled collection of bits-and-pieces that TOGAF and its ilk call ‘business-architecture’ should not be called ‘business-architecture’, for the simple reason that it is not ‘the architecture of the business’.

[The latter point should be obvious when we consider that TOGAF's 'business-architecture' assumes that the entirety of the non- IT executive - in other words, the CEO, CFO, COO, CMO, CHO and any non-IT CTO, and all of their respective domains - can all meaningfully be lumped together as 'the business', with only IT needing aany differentiation from the rest. Anyone who's had any dealings at executive-level will know that it's, uh, not quite as simple as that? :wry-grin: ]

Best leave it there for now, I guess. Over to you?

One of the key reasons why I’m so vehemently against any-centrism and suchlike revolves around the question of competence – or, more usually, the lack of it.

Competence is where someone knows what they’re doing, and does it. And, oddly, often don’t bother to say that they’re competent – perhaps because they don’t need to say it, their actions say it well enough instead. The outcome of competence is fairly certain, even in contexts of high uncertainty.

Non-competence is where someone doesn’t know what they’re doing, and will either not do it, or will do the best they can, yet with the explicit intent to use it as a learning to improve their competence. Importantly, they will usually say that they don’t know what they’re doing. The outcome of non-competence is uncertain, even in nominally-certain contexts, but at least we are aware of the risks.

Incompetence is where someone doesn’t know what they’re doing- i.e. is non-competent to do the task – but either purports and/or believes themselves to be competent. They will usually say that they are competent, even though demonstrably they are not; they claim to be responsible, yet have limited ‘response-ability’. The outcome of incompetence is fairly certain, and frequently dire, yet lack of awareness of the risks is often rampant, or in some cases the risks actively concealed.

Someone who is non-competent can become competent by learning the respective skills, or be competent by proxy, via finding someone else who is competent at doing the respective type of task. I treasure my non-competence, because it means there’s always more for me to learn. And as an enterprise-architect, I am, almost by definition, non-competent in much if not most of the detail-aspects of areas that I need to cover: hence one of my key competencies is the ability to learn enough of a new area fast enough to be able to guide meaningful exchanges between people who are fully competent in some detail-area but are not competent in others with which they need to connect.

Yet one of the key criteria for non-competence, and to separate it from incompetence, is a willingness to accept that we are non-competent, and say so. If we’re not aware that we’re non-competent, we automatically increase the risk of being incompetent. And if we know that we’re not competent, yet somehow ‘need’ to claim that we are competent, we would, again, automatically be incompetent – with a very high risk of inappropriate or ineffective outcomes of the work.

In part it’s a cultural problem: the risk of incompetence increases wherever a culture exhibits any of these characteristics:

• prioritises content over context, ‘truth’ over context-dependent usefulness
• has an insistent ideological base (leading to the same as above)
• is typified by rampant egotism, self-advertising and self-centrism
• is frequently swayed by tides of hype and ‘following after the latest fad’
• displays an almost desperate need to be ‘right’

Unfortunately, all of these attributes are extremely common in business, and in many cases are actively prized… By definition, they’re also more likely to be common in any ‘truth’-oriented domain, one which operates primarily on ‘true/false’ decision-making – hence, in practice, the tendencies towards IT-centrism and finance-oriented business-centrism, both of which rely on simple true/false logic for most of their operational decisions.

In SCAN terms, all of these are where the Simple certainties of Belief – either as ideology and/or as self-belief – are inappropriately applied to the far side of the Inverse-Einstein Test, where the uncertainties of the Ambiguous and the Not-Known cannot be avoided.

This gives us a dysfunctional ‘diagonal’ decision-path, where Assertion is imposed on the Not-known, or Ambiguity ‘solved’ by arbitrary Belief:

Yet the real problem here is somewhat more subtle:

• someone who is competent will typically not bother to say so, but will just get on with the work instead
• someone who is non-competent will typically say that are not competent, but will often actually be adequately-competent, or at least willing to learn to become so
• someone who is incompetent will typically claim that they are competent, and will usually not be willing to learn how to become so, because to do so would betray to themselves and others the fact that they are actually not competent

Which, in practice, leaves us with a huge dilemma:

• those who do not claim to be competent usually are competent
• those who do claim to be competent frequently are not competent

Hence, again, the kind of mess that we see so often in enterprise-architectures, wherever IT-centrism, business-centrism and the like predominate… Oh well.

Comments, anyone?

How is it that what we actually do in the heat of the action can differ so much from the intentions and decisions we set beforehand? How can we bring them into better alignment, so that we do ‘keep to the plan’, at the individual level, and across the enterprise? And once again, what implications does this have for our enterprise-architectures?

This extends the previous posts on SCAN sensemaking and real-time decision-making, ‘Belief and faith at the point of action‘ and ‘Decision-making – belief, fact, theory and practice‘, this time to explore the linkage – or lack of it – between ‘considered’ decision-making and real-time decision-making.

[As before, most of this is 'work-in-progress', so be gentle with it, okay? It should be usable as-is, but do expect odd gaps, rough-edges and wobbly-bits in various places, and please give constructive feedback where you can. Thanks!]

We started from the SCAN sensemaking-frame:

And reviewed it from a perspective of decision-making rather than sensemaking:

It’s actually the same frame, so the two axes are the same in both views:

• a ‘horizontal’ axis of modality of sensemaking and decision-making, from simple true/false on the left, to infinite-possibility on the right
• a ‘vertical’ axis of time-to-decision or time-to-action, stretching from a real-time ‘now!‘ to a potentially-infinite future (and some symmetry with time-from-decision etc, into the past)

The vertical-axis is essentially continuous, but the horizontal-axis has a distinct phase-shift where the modality of decision changes from a simple-true/false [0..1] to an open n-ary [0..n] choice. To the ‘left’ of this point, the apocryphal Einstein dictum applies: doing the same thing should lead to – or is believed to lead to – the same results; whereas to the ‘right’ of that point, doing the same thing may lead to different results, or doing different things may lead to the same results.

On the left-side, there is what purports to be ‘objective certainty’; on the right-side, there is, by definition, some degree of inherent-uncertainty, always somewhat context-specific, and often somewhat personal and subjective. A conventional ‘control’-based concept of the world assumes that everything can somehow be forced onto the left-side of the frame; Reality Department and real-world practice indicates that such concepts of ‘control’ are still wishful-thinking at best, and that alternate decision-strategies must be available, dependent on context.

Hence one of the key tasks of an enterprise-architecture is to ensure that all required decision-methods are supported, and also ensure that appropriate methods are applied to each context.

The previous post, ‘Decision-making – fact, belief, theory and practice’, mainly looked the ‘horizontal’ dimension of this frame; here we’ll explore the impacts of the ‘vertical’ dimension – specifically, the separation between intent and action.

Read more…

A kind of announcement, really: a knowledge-base wiki for whole-enterprise architecture is now available and ready for content and use.

I’ve given it a temporary home on my Sidewise server:

• http://ea.sidewise.biz

No doubt it should have a proper domain of its own, but that’ll do for now to get us started.

[By the way, this is another follow-up to my post 'Helping others make sense of my work' - the need for a wiki was a suggestion that came up several times in the comments there.]

It’s a fairly straightforward wiki, based on the WikkaWiki framework – probably the cleanest and simplest wiki-framework I’ve come across. (I’ve struggled with many such frameworks over the years, of which Wikipedia is almost the worst…) Like all wikis, though, it does have its own quirks, hence some quick comments:

– Anyone can read, write or comment. (That’s the default: there’s actually a full access-control system for read, write and comment, all the way down to individual page-level, but that’d take too long to explain here.)

– However, to write, comment or edit, you’ll need to register a user-account. (There’s no charge for this, of course, and should be no privacy-implications: it’s just to stop spam-bots using the site.) There’s a quick summary on how to do this on the wiki home-page.

– One minor ‘gotcha’ is that user-names need to be in wiki-format – what’s known as ‘CamelCase’, beginning with a capital-letter and with at least one additional capital-letter after the start. For example, my user-name is ‘TomG’; you might make yours ‘FredBloggs’ or VikusVdM’.

– Editing is straightforward: click the ‘Edit’ link on the left side of the page-footer, or double-click on the page itself. The ‘Store’ (save) and ‘Preview’ buttons are at the lower-left when you’re editing.

– Formatting is a lot simpler than most wikis: in many cases it’s two repeated-characters. See the ‘Wiki formatting guide’ that’s linked from the home-page. Links are straightforward: ‘[[', then the page wikiname (internal link) or URL (external link), then a space as separator, the link-text, and ']]’.

– Usefully, a page can include a FreeMind-format mindmap: paste the FreeMind XML into the edit-space as the page-content. Read-only, unfortunately, but it’s an easy way to share mindmaps.

– Upload of images and other files is a bit more difficult, and at present only administrators can do it. I’ll hack the code as soon as I can, to allow a broader range of users to upload, but in the meantime, if you want to upload a file, send it to me and I’ll upload it for you.

I’ve put up some initial content to get started – a few dozen definitions, a couple of articles, and a whole load of links to other posts elsewhere – and I’ll continue putting more material up there over the next few days and weeks. But the rest is up to you, really: it’s everyone’s site, not just mine.

Anyway, it’s there, and usable: over to you?

In what ways do ideology and experience inform decision-making in real-time practice? How do we bridge between the intentions we make before and after action, with the decisions we make at the point of action itself? And what implications does this have for our enterprise-architectures?

This extends the previous post on real-time decision-making, ‘Belief and faith at the point of action‘, to crosslink with the earlier ideas on SCAN and sensemaking, and especially about where there is more time available to review and reflect on action.

[A gentle warning and polite request: much of this is still 'work in progress', so do beware the rough edges and knobbly bits, and use it with some caution; and whilst I do need critique on this, please don't be too quick to kick down the scaffolding that's holding it all together. Fair enough?]

The previous post was about how options for sensemaking become more constrained as we approach real-time. Right at the point of action, the options reduce to either a Simple interpretation in terms of of true/false categories, versus a Not-simple interpretation based on a modal-logic of possibility and necessity, which is much harder to explain or even to describe to anyone else. In SCAN we’d depict that compression as follows:

In much the same way, decision-making becomes compressed down to Simple belief versus Not-simple faith – neither of which are actually explainable, and both of which, at the root, are primarily emotional rather than ‘rational’:

In both sensemaking and decision-making, the crucial distinction – indicated in SCAN by where the red-line time-axis crosses the green-line axis of decision-modality – is what I’ve termed the ‘Inverse Einstein test’. Einstein is said to have asserted that “insanity is doing the same thing and expecting different results”: but whilst that’s true in a simple rule-based world, it’s not true – or not necessarily true, anyway – in a more complex world where many things are context-specific or even inherently unique.

So our ‘horizontal’ test is this: if doing the same thing leads to the same results – or is believed to lead to the same results – then it’s a Simple decision; if doing the same thing leads to different results, or if we need to do different things to get the same results, it’s Not-simple.

[Yes, I do know that that's a Simple true/false distinction across a spectrum that in reality is fully modal. If you want to apply the appropriate recursion here, please feel free to do so: I thought it wisest here to keep it as simple as possible, because this can get complicated real fast, and unless we're careful to keep the complexities at bay we could end up with a right old chaos of confusion. Which is, yes, yet another recursion... Hence best to keep it simple for now, as best we can, acknowledge that much of it isn't Simple, and allow the recursions to come back in later when there's a bit more space to work with it.]

The crucial point about real-time is that there’s no time available for a distinct sensemaking-stage: decision links directly to action, and vice-versa. (That’s why it’s called ‘decision’: the same linguistic roots as ‘incision’, it’s literally ‘cutting away’, ‘cutting apart’, the cutting-edge for action in the ‘now’.)

For sensemaking to take place, there must be a gap in time between one decision to the next. The key to John Boyd’s ‘Observe, Orient, Decide, Act’ (OODA) loop – which, importantly, is also not a loop as such – is that it still allows distinct sensemaking (‘Orientation’) to take place, but keeps it as close to real-time as possible: that’s what’s meant by ‘getting inside the opponent’s OODA loop’.

As time-available – the red-line ‘vertical’-axis in SCAN – extends outward either side of real-time, the OODA-’loop’ can become recursive, and thence, given enough time, simplified-out to a Deming-style ‘Plan, Do, Check, Act’ (PDCA) continuous-review cycle, such as is also implied in the US Army’s ‘After Action Review‘:

• “What was supposed to happen?” – what was our Plan?
• “What actually happened?” – what did we Do?
• “What was the source of the difference?” – what do we need to Check?
• “What do we need to do different next time?” – about what do we need to Act?

As I’ve described in other posts, sensemaking-choices tend to split as described in SCAN: there’s a ‘bump’ on the path, indicated by the jump between simple true/false logic versus fully-modal logics of ‘possibility and necessity’ on the ‘horizontal’ axis, contrasted with a much smoother spectrum of choices as available-time extends in the ‘vertical’-axis. Although the ‘vertical’ boundaries are less clear-cut than the ‘horizontal’ ones, this gives us the four SCAN quadrants – Simple, Complicated, Ambiguous, Not-Known:

Those distinctions determine the appropriate tactics for sensemaking, as described in those earlier posts.

Decision-making seems to follow a similar, closely-related pattern – though that’s the part I’m having trouble pinning down right now.

[Boyd's OODA is in part another attempt to pin down the same relationships; likewise Snowden's Cynefin, if rather less so. Jung's frame of 'psychological types' is probably a closer fit than Cynefin for this: I've used a generic decision-types adaptation of it for some decades now, though it's still not quite right. Hence this exploration here.]

So again, it’s ‘work-in-progress’, but this is where I’ve come to at present:

It’s a decision-making frame based on the same horizontal (decision-modality) and vertical (time-available) axes as in SCAN, and hence the same sort-of-quadrants but with a decision-oriented re-labelling: Belief (Simple), Assertion (Complicated), Use (Ambiguous) and Faith (Not-known).

On the left-side of the Inverse-Einstein test, the mechanism that links Assertion and Belief is a drive for certainty, for ‘control’. On the right-side, linking Use or ‘usefulness’ with the real-time openness of Faith, is more a focus on experience, underpinned by a deeper kind of trust – a trust which is often conspicuously absent in any concept of ‘control’.

[For this post I'll focus more on what happens across the horizontal-axis, the relationships between theory and practice, or 'truth' versus 'usefulness'. I'll explore more closely the interactions along the vertical-axis - between what we plan to do versus what we actually do - in a following post.]

In terms of decision-making tactics:

• on the left-side, theory takes precedence over practice – or, in some contexts, ideology rules, which is much the same
• on the right-side, practice takes precedence over theory

In essence, this is CP Snow’s classic ‘The Two Cultures‘, the sciences (left-side) and the arts (right-side). Notice, though, that technology sits on the right, not the left: it uses theory, but that isn’t its actual base – hence the very real dangers in the often-misleading term ‘applied science’.

Bridging the gap, from left to right, is praxis,”the process by which a theory, lesson, or skill is enacted, practised, embodied, or realized”; and from right to left, is pragmatics, “a process where theory is extracted from practice”. As enterprise-architects would be all too aware, the latter always starts from pragma, from “what is expedient rather than technically ideal”: and it usually includes the joys of ‘realpolitik’, of carefully filtering reality to fit in with other people’s prepackaged assumptions…

That boundary denoted by the Inverse Einstein Test is all too real: whether the beliefs in question are ‘scientific’, religious, political or whatever, the ‘need’ for certainty will often trigger huge resistance against anything that doesn’t fit its assumptions. For example, there’s a very close mapping between this frame and the classic scientific-discovery sequence of idea > hypothesis > theory > law, which align with Faith, Use, Assertion and Belief respectively.

In real scientific practice, it’s not a linear sequence, there’s a lot of back-and-forth between each of the steps. And in principle, it should be a continuous-improvement cycle, a broader-scope form of PDCA. But as Thomas Kuhn and many others have documented, that same ‘need’ for certainty often places a near-absolute barrier between supposed ‘scientific law’ and any new ideas – in other words, between Belief and Faith – that brings that cycle to a sudden halt, sometimes for years, decades or even centuries. All too often, in practice, if we take the real-time ‘short-cut’ from Belief to Faith, we will be forcibly forbidden to return along the same path: instead, we’re forced to go ‘the long way round’, via Use and Assertion (hypothesis and theory) – which we may not have time to do. Which is a very real problem. And one that applies as much in enterprise-architecture as in any other field – as we’ve seen with the inane IT-centrism that has dominated the discipline for far too long.

It gets complicated…

What I’ve been seeing, as I’ve explored this frame, is a whole stream of often-subtle misunderstandings and ‘gotchas’ that I’ve noticed time and again in practice in enterprise-architecture and elsewhere. These seem to be where many unnecessary complications and confusions arise – so it’s worth noting them here.

For example, fact arises from experience: its basis is on the right-side of this frame – not the left. What’s on the left-side often purports to be fact: yet it’s not fact as such, but interpretation of fact – a very important difference. The left-side operates on information, an interpretation of raw-data – but it often has no means to identify the source or validity of that information, or its method of interpreting it. (This is the same inherent problem whereby a logic is incapable of assessing the validity of its own assumptions: by definition, it must call on something outside of itself to test those premises.) So on the left-side, there’s actually no difference between ‘real’ and ‘imaginary’ – which can lead to all manner of unpleasant problems if the left-side is allowed to over-dominate in any real-world context…

Importantly, there’s no real difference here between ‘objective’ versus ‘subjective’: that distinction is actually another dimension that’s somewhat orthogonal to this plane. What I feel, or sense, is subjective, but it’s still a fact; whereas how I interpret that feeling or sensation is not a fact – it’s an interpretation. Telling someone that they should or shouldn’t feel something is just plain daft: the feeling itself is a fact – something about which we don’t actually have any choice – whereas the ‘should’ is an interpretation arbitrarily imposed by someone else.

[What we do in response to a feeling is a choice - literally, a 'response-ability' - and is something that can be guided by 'shoulds' and the like: but not the feelings themselves. That's a very important distinction which, sadly, surprisingly few people seem to understand...]

There is a specific sense in which subjective versus objective aligns somewhat with the ‘less-time’ versus ‘more-time’ on the SCAN vertical-axis. More-time means more time available for experimentation and analysis – and that can allow us to identify what’s shared (‘objective fact’) across many people’s experience, versus experiences that are more specific and personal (‘subjective fact’).

But there seems instead to be a tendency to conflate the objective/subjective distinction with the SCAN horizontal-axis – objective-fact as ‘truth’ on the left-side, subjective-fact as ‘not-truth’ on the right-side. There are ways in which that conflation can work – it’s at the core of the Jungian frame, for example – but we need to be careful about it. Using that conflation to dismiss all subjective-fact as ‘irrelevant’ – as the classic ‘command and control’ models would do – not only makes no sense at all, but is extremely unwise in real-world practice…

There also several other key distinctions across either side of the Inverse-Einstein test:

– ‘science’ versus technology, which also parallels ideology versus practice: on the left-side, there’s an assertion that something is ‘true’, whereas on the right-side we proceed as-if it’s true – which is not the same at all.

– organisation versus enterprise: the nature of an organisation is that it’s about left-side themes such as control, beliefs, repeatability and certainty; the nature of an enterprise is that it’s not certain, “a risky venture” and suchlike – with all that that implies.

– structure versus story: most structures within current enterprise architectures will, again, have a left-side focus on providing repeatability and certainty; story and other forms of narrative-knowledge provide an alternate kind of ‘structure’ that holds many of the right-side themes together

– sameness versus uniqueness: another key enterprise-architecture theme, sameness and repeatability is very much a left-side theme, whereas uniqueness is just as much a right-side theme

– ‘best-practice’ versus ‘worst-practice’: the notion of ‘best-practice’ assumes that practice that worked well in one context will be directly applicable to another, the same success repeatable in another; by contrast, maintenance engineers and others who work extensively with unique or near-unique contexts share their learning more through ‘worst-practice’, stories of what didn’t work in a given context. (I think I first heard that one from Dave Snowden? – credit where credit’s due, anyway.)

The trade-offs across each of these dichotomies all have direct implications for the design and structure of any enterprise-architecture.

Implications for enterprise-architecture

Take a look at those dichotomies again: which side do you think is emphasised by current enterprise-architectures?

The obvious answer is that, almost invariably, the left-side is given priority over the right.

However, this has huge consequences for the effectiveness of the overall enterprise, and for the enterprise-architecture that describes it:

• interpretation takes priority over fact: never a good idea…
• theory and ideology takes priority over practice and experience: that’s almost a definition of (misused) Taylorism…
• the need for (spurious) ‘certainty’ and ‘control’ takes priority over trust of anything or anyone: ditto on Taylorism…
• the reliance on true/false decision-methods can render the organisation unable to cope with any form of uniqueness
• the need to force-fit everything into sameness of content – ‘best practice’, IT-centric BPR and the like – fails to grasp the differences of context
• the over-focus on organisation – ‘the letter of the law’ – literally kills off the spirit of enterprise…

Look at most of our existing EA toolsets, too: can you find any toolset that’s actively designed around anything other than true/false logic? Other than in rare model-types such as ORM (Object-Role Modelling), there’s no means to describe modality in relationships – hence, for example, no directly-supported way to describe a usable reference-model that allows for real-world ifs, buts and perhapses.

And whilst every toolset focusses on structure – and most do that very well, too – how many of those toolsets also help us to focus on the counterpart of story? They might support few use-cases, perhaps, but that’s about it: there’s a huge gap in capability there…

What we need, urgently, is a better balance between structure and story, between theory and practice, between organisation and enterprise. And without adequate support in the toolsets, that means that we have to create that balance ourselves.

The crucial point is that this balance is not an ‘either/or’, but a much more modal ‘both/and’:

• theory and experience
• ‘objective’ and ‘subjective’
• ‘science’ and technology
• certainty and trust
• true/false and fully-modal
• organisation and enterprise
• structure and story
• sameness and difference
• ‘sense’ and ‘nonsense‘
• certainty and uncertainty

We will only achieve a real effectiveness in the architecture via a fully-nuanced ‘both/and’ balance across all of these dimensions, and more.

So take a careful look at your own organisation, your own enterprise-architectures and the like: where is it out of balance, in this sense? In SCAN terms, how much does it over-emphasise the left-side at the expense of the right-side? And what can (and must) you do to bring it back into a better balance overall?

Comments/suggestions/experiences on this, anyone?