Classically, this follows the ‘diffusion of innovations‘ pattern described by Everett Rogers and others:

• Innovators – venturesome, educated, multiple info sources;
• Early adopters – social leaders, popular, educated;
• Early majority – deliberate, many informal social contacts;
• Late majority – skeptical, traditional, lower socio-economic status;
• Laggards – neighbours and friends are main info sources, fear of debt.

But the catch, as documented by Geoffrey Moore in his 1991 book Crossing the Chasm, is that there’s that chasm of (lack of) understanding that sits somewhere in the ‘early adopters’ range, blocking the path between innovation and the mainstream – the gap between the people who’ll try anything new, and the people who won’t. Something we need to explore…

For incremental innovation – essentially within the same paradigm – the gap is usually quite easy to bridge. If you can prove some kind of ROI – return on investment, in any appropriate sense of return on the respective investment of effort or whatever – that’ll usually be enough to make the bridge. Often the only real problem is the hype-cycle of over-inflated expectations and disillusionment before sense finally settles in – kind of like hoping that a lightweight rope-bridge is enough to carry heavy traffic, without dealing with any of the heavy engineering needed to make it actually work…

For disruptive new technologies, that gap can easily turn out to be a real chasm that’s hard to bridge. Many worthwhile new technologies have fallen into that chasm, or – like Betamax versus VHS video – succumbed to the dread syndrome of “a triumph of marketing over technical expertise”.

But when the innovation is essentially just an idea – a new way of doing things, a new paradigm in itself – that chasm can be huge: and, for a long time, often apparently unbridgeable. And when there’s a strong wannabe blocking the way, stranded halfway over the bridge but clearly going no further, we first have to clear the wreckage out of the way before we can cross. That’s pretty much the situation we have right now with enterprise-architecture…

The ‘strong wannabe’ here is so-called ‘traditional’ EA – in other words, enterprise-wide IT-architecture with some dangerous delusions as to its business role and value. What it’s designed for, is quite good at, and with some care can actually deliver on, is IT-infrastructure architecture. Anything else IT-related, such as data-architecture, applications-architecture and IT-security architecture, is somewhat of a bonus to that core promise, but with no guarantees as to whether it’ll actually work; whilst anything less IT-related, such as information-architecture or business-architecture, it tends to work either less well again, or not well at all. Unfortunately it’s often busily pretending that it’s ‘the architecture of the enterprise’ – which isn’t. In fact the fundamental design of typical ‘traditional’ EA, as Jeff Scott explains in his Forrester article ‘Is The Current EA Paradigm Right For Business Architecture?‘, is simply not suited to a world beyond IT, and perhaps now not even for that:

Here are the six elements that I see very consistently in the thought patterns of EAs:

Governance – Mechanisms to approve EA designs and enforce adherence to the reference architecture at the project level.

Principles – Decision filters that both EA development and application decisions flow through.

Current state – A snapshot of current issues and technology baseline (often in significant detail).

Reference architecture – The body of work describing EA’s intent, organized in a framework, expressed in strategy, standards, patterns, guidelines, etc.

Target state – An idealized future state viewpoint describing how the organization desires to change the current state based on the current understanding of technology and architecture.

Solutions architecture – Application of principle compliant reference architecture to current problems in order to move the EA closer to the target state viewpoint.

So what’s the problem? We seem to adhere to this model even though it is at best only moderately successful. For example, most EAs have established a governance process, but very few describe it as being impactful. Almost all EA teams have a set of principles, but almost none actually live by them – they are more like a set of good intentions. And who has actually attained a reasonable facsimile of their target state?

In short, there are three myths here that are blocking the bridge over the chasm:

• the myth of causality and control - seen above as ‘Governance’, ‘Reference architecture’ and the causal-style usage of ‘Principles’
• the myth of stasis – seen above as ‘Current state’ and ‘Target state’
• the myth of detail, that detail alone is enough – seen above as ‘Principles’ and ‘Solutions architecture’

And, of course, the myth of IT-centrism – the belief that IT itself is the true core of any enterprise. And elsewhere in business-architecture, there’s another core myth that’s blocking the road: the myth of money as the measure of value – or even as a meaningful measure of value at all. Until we get all of these myths out of the way, enterprise-architecture is going nowhere.

To counter all of those myths and clear the bridge-block, we need the following:

• a better understanding of the balance between causality and complexity (such as in systems-theory)
• a better understanding of the balance between static and dynamic (again, as in systems-theory)
• a better understanding of the balance between detail and ‘big-picture’
• a better understanding of role of IT (and other domains) within the overall enterprise
• a better understanding of value, with money as merely one proxy-form for value

We also need a better understanding of the role of people within an enterprise-architecture – because they hardly appear at all in most models at present…

There are plenty of independent innovators in enterprise-architecture, but almost by definition most of us are somewhat out on the fringes – the Outsiders. There’ve been a few good signs over the past year or so – particularly the increasing focus on business-architecture in Open Group (TOGAF) conferences and elsewhere. Yet to me the crucial signal that real change is about to occur is when the innovators in the big consultancies are finally let loose. Big-consultancies are the archetypal ‘early adopters’: they need to purport to be ahead of the market (as indeed they are, relative to the ‘early majority’ and the rest), but they also rarely make a move until all the hard work has already been done by the ‘innovators’ (because otherwise the idea is too ‘new’ to be saleable to their ‘early-majority’ constituency). That’s why I’m careful to keep track of ‘insider’ innovators that I trust – people like Ron Tolido at CapGemini, Nick Gall at Gartner, Jeff Scott at Forrester, and Michael Porter at Harvard. So when all four of those folks put out important new work in the same week, something’s definitely on the move – clearing the blockage from both sides of the bridge.

Jeff Scott‘s critique-piece ‘Is The Current EA Paradigm Right For Business Architecture?‘ was mentioned earlier above. In essence he’s tackling the problems that ‘traditional’ IT-centrism has caused for enterprise-architecture and its expansion outward to cover a true whole-of-enterprise scope. The article raises some important questions, as do the comments that follow it. Strongly recommended, anyway.

Nick Gall was one of the team who last year introduced the concept of ‘hybrid thinking‘ – Gartner’s variant on ‘design-thinking‘. This week he published a summary-pointer to his somewhat earlier article ‘From Hierarchy to Panarchy: Hybrid Thinking’s Resilient Network of Renewal‘ (previously for Gartner clients only), a long exploratory piece on what he called ‘panarchitecture’, linking the ecology concept of ‘panarchy‘ to enterprise-architecture and Gartner’s notion of ‘hybrid-thinking’. Gall’s version of panarchy takes the standard ‘S-curve’ for implementation, exploitation and conservation (continuing use) of an idea or a resource, and links it to Holling’s ‘back-loop’ of release and reorganization, to make a complete cycle, and then cycles within cycles. This addresses much of the missing material on systems-theory and complexity in EA, and also affirms the centrality of real people in the architecture. Again, strongly recommended for enterprise-architects, though to be honest I suspect Gall’s term ‘panarchitecture’ is even less likely to fly with business-folk than the existing term ‘enterprise-architecture’…

Ron Tolido – he who brought us the notion of ‘Slow IT‘ and the demolition strategy – has come up trumps again with a really nice metaphor for application lifecycles, in his article ‘The Inception of TRAIN and SCOOTER Apps‘:

Actually, we distinguish different ‘Application Lifecycles’, each with their own dynamics around requirements, quality, time-to-market, agility, tools being used and place within the organization. We thought a transport metaphor would be quite effective, so prepare yourselves to be familiarized in the near future with 5 different types of applications, ranging from TRAIN apps (solid, stable, standard, predictable) all the way to SCOOTER apps (highly individual and personalized, ultra-agile, ad-hoc). Much more soon, so stay tuned to this blog. For now, one little visualization [a still from the film 'Inception'] of what happens when you try to make a TRAIN application behave like a SCOOTER. I believe our ‘application city’ streets are full of these trains gone rogue. Let’s hope they are just a bad dream soon.

In other words, a metaphor that addresses that much-needed balance between static and dynamic, detail versus big-picture, big-system ‘control’ and predictability versus agility for a complex, unpredictable world. The article is only a taster for a more detailed exploration due to follow in the next few weeks: watch out for it when it comes, because it’ll be valuable indeed – especially for the IT-oriented aspects of enterprise-architecture.

Finally, Michael Porter, who’s previously brought us Five Forces, Value Chains and the warning that the obsession with shareholder-value is the ‘Bermuda Triangle of Strategy‘ [PDF]. This time, with Mark Kramer, he wrote in HBR on the notion of ‘shared value’ – ‘The Big Idea: Creating Shared Value‘ (see also here on his presentation at Davos)  providing business-architecture with another means to break away from the deadweight stranglehold of money-only business drivers. (A few predictable commenters, of course, doing the usual Friedman-style shrieks about “the only purpose of business is to make a profit” – and, unlike Porter, failing to understand where that profit actually comes from, or any of the complex relationships between currency, value and trust.) To my mind Porter doesn’t quite go far enough as yet towards making the connection with ISO9000-style vision and the shared-enterprise or ‘extended-enterprise‘ – but at least it’s something we can leverage as we build towards a business-architecture that actually is about the business of the business.

And that’s the point here: these people are helping to clear the crossing over the chasm from the other side of the bridge – the ‘early-adopter’ side. Which means that whole-enterprise architecture is just starting to go mainstream. Which, for me and so many of my other enterprise-architecture colleagues, is very good news indeed…

There’s one part of the chasm that we still need to bridge, though: the engagement of people within the architecture. So in the next few days I’ll write at least a couple more posts: one on where the people-themes fit within architecture-models; the other on the problem of power, and how to address it within an architecture.

But in the meantime, I guess it’s Watch This Space – because it seems clear that things are on the move at last!

Sometimes a word will pop up out of nowhere, like the mushrooms did yesterday on the grass verge just down the road on this small suburban block.

But ‘ideafarming’ is a good way to describe the work that I do:  like a old-style farmer, planting seeds for new ideas, tending them, nurturing them, watching them grow.

Perhaps not as exciting as fishing for facts; perhaps not as challenging as herding cats; yet in its own way definitely as much hard work as either of those, and it has its own quiet pleasures too.

Different styles of ideafarming, of course. Some go for a machine-like monoculture, repeating the same ideas over and over again to reap the maximum benefit before the ground itself is exhausted – at which point an overly-artificial hydroponics-style approach may be the only option left. Others are aggressive – almost obsessive, even – in their war against the weeds: “Any idea needs to be challenged, vigorously and early … to make it more resilient”, thundered one erstwhile colleague – a tactic which seems more like ripping every tender young shoot out of the ground to check if it’s growing. My own ideafarming is probably more organic in style: watching, waiting, letting things be, letting things grow together in unexpected ways, companion-planting between disparate ideas and the like.

Some ideas ripen quickly, to give a quick harvest – but those ideas tend to be the most perishable of all, and getting them to market in time can be a chancy business. Other ideas are more predictable, perhaps with a yearly harvest – but that can mean long gaps where the work is as hard as ever but still no return in sight. Others again may take years, decades, even centuries, before they start to yield their crop – the metaphoric grapevines, hazels, chestnuts, walnuts of the ideafarmer’s harvest. They all look much the same when their first shoots first push their way out of the ground – and yet each needs nurturing in their own distinctive ways. Often the nurturing consists of deliberately ‘doing no-thing’ – which is not the same as ‘doing nothing’; and sometimes what we most need to do may make no sense at all to ‘outsiders’ – such as the paradoxical advice that “in order to remember something you never knew, first set out to forget it”.

And we’re always at the mercy of the elements, too. City-folks may see the machinery that we ideafarmers use – the mobile-phone, the library, the computer as metaphoric combine-harvester – and think that that’s what does all the work; but the reality is that those machines do nothing on their own, they help us in our work, but they don’t make the ideas grow at all. If we’re fortunate, and skilled, and careful, we may indeed at times have a bumper harvest, a glut of new ideas; but sometimes – and sometimes even for years – nothing will grow. Stuck. No matter how much we might like it to be otherwise, it’s not something we can control.

So we ideafarmers tend to be of a taciturn temperament: quiet, reflective, often rather solitary, a bit scruffy, perhaps, even a bit eccentric in our ways at times. Observant, yes – because we have to be; careful; innovative, always trying something new, yet always aware of how things work out over the longer term, looking to the future by being carefully aware of the present and the past. Passionate about what we do – as anyone can see at any conference – yet often irritable with those who get overly excited about everything: after all, there’s not much room for excitement in a working life that for the most part consists of watching, very carefully, at the way the grass grows.

And it’s a working life that never stops: get up in the morning, walk the fields, tend the fences, watch for pests and predators, for termites and ‘term-hijacks‘, for wild ideas and other weeds that will run rampant if we don’t watch out for what’s happening to our would-be harvest; a moment’s rest on the porch at sunset, perhaps, but then it’s time to settle down to get ready for yet another day. We don’t have much time for the bustle of the market: the work is calling – never stops calling – for our attention, we know we have to get back to the farm. And ideafarmers don’t take vacations as such: the ideas continue to grow whether we’re there or not, so we’re always working even when we’re not at work. We don’t have much choice about that: ideafarming isn’t a job, it’s more a way of life, a way of being. In reality, it’s not just something that we ‘do’: we’re ideafarmers because that’s who we are.

Ideafarming. A strange job, but someone’s gotta do it, I guess?

Unlike some of his colleagues such as David Tebbutt, I never knew Guy personally, so there’s really nothing I can add in that sense. Yet reading through the comments in David’s wonderful ‘Some words for Guy’ weblog, I came across a pointer to Guy’s very first column for the British computer-magazine Personal Computer World, way back in 1978. (Warning: the whole file is over 16Mb: GoogleDocs version is here.) The article is on p.14, and the last part of it seems amazingly prescient:

“Yet that potential [of the computer] is nearly ended. It was extended by the ignorance of semiconductors designers who decided to imitate computer memory with transistors, rather than looking at the whole field of computer architecture and saying ‘what have we here?’.

“What we have here is a computer which , when asked for the one faulty item in 10,000, has to check all 10,000 to see which it is. How much better to have a computer which merely says to its component parts, ‘Any faulty items report here at once.’

“The ability to give memory that processing power has existed for something like five years already. Associative processing has been largely ignored because the markets for supplying cheaper forms of Von Nuemann components were easier to satisfy – but the limitations of a sequential processor have nearly been reached.

“But not by us, the private users of computers. We are only on the edge of a revolution which will make the printing press, the telephone and the motor car look like minor items on a shopping list, as the population gets ‘on line’. And from here on, the history of computing will be the history of society, not just of calculators.”

His comment about ‘the limitations of a sequential processor have nearly been reached’ might at first seem a bit off, but I think he’s right: all that’s actually been done over the last thirty years has been to try to bypass those limitations by cramming more and more and faster and faster items with the same limits into the one unit of space and effective-time – the limitations haven’t been addressed at all. If so, then we’ve barely even started yet: the past thirty years have been a side-excursion down a wrong-turning, as usual for the wrong reasons (i.e. that it was the cheap-and-quick-and-easy option rather than the most fruitful one), and it may take us some time to get back to the main track again. Very interesting indeed…

Which brings me to another point: about how to do more than merely celebrate a great thinker’s life? Personal reflection is one side of this, yet professional reflection is something different again. The usual suggestion, especially in this context, would be to re-publish selected items from that person’s work: I know some people are already planning to do this with Guy’s writings. Yet what interests me even more is about what we do with a great person’s legacy – how would we continue that legend?

Guy used LiveJournal to document the later progress of his life; it seems to me that there would be real value in an equivalent to record new ideas that are linked to a specific person after their passing. Kind of like academic attribution, or with a blog, the same way that we would use web-links to reference an current article or tweet. Citations not for a completed idea by that person, but to acknowledge further ideas that are in part derived from or influenced by or extend the original work.

So is there some equivalent of LiveJournal that we could create for this, around which references to a specific person could coalesce? The technical side is relatively trivial – some practical problems around security, and ensuring that the work is treated with respect, but that’s about all. Conceptually it sits somewhere between LiveJournal and Wikipedia – much like Wikipedia, in fact, though with different guiding principles.

Seems an idea worth exploring, anyway –  comments or suggestions, anyone?

[And, of course, credit to Guy Kewney for triggering the idea.]

So here was a question from Bala Somasundaram at Honeywell Technology Solutions in India:

How Enterprise Architecture can enable Business Innovation?
I understand Enterprise Architecture is best positioned to be an innovation enabler leveraging its Business-IT alignment, Technology competency and Understanding of the Business Model. Would be interested to hear from fellow members on their viewpoints/experiences.

In the hope that it’ll be useful to others here, this was my response:

The first requirement – and usually the hardest for anyone working in IT in general – is to accept that EA is only peripherally about IT, and that an IT-centric approach to EA and innovation is as likely to destroy the business as it is to enable it.

EA is, first and foremost, the architecture of the enterprise as a whole – not solely its IT. That was the painful lesson of so many failed attempts at BPR (business process reengineering) – you must start from the values, vision, functions, processes and services of the enterprise, not from IT-based ‘solutions’. If you put the ‘solutions’ first – as so many BPR proponents did, and as we’re now seeing with web-services and cloud-computing – you end up forcing the business to fit the ‘solution’: the enterprise loses its way, forgets what it knows or even what it’s in business for, and the whole thing falters and may collapse completely. That’s definitely the wrong kind of ‘innovation’… efficient, perhaps, but not very effective…

One reason for starting EA in IT is that – particularly after repeated mergers and acquisitions – the IT landscape of an enterprise is often a complete shambles, with many incompatible legacy systems and a proliferation of small point-applications to solve local problems. In effect, there are four distinct IT-EA maturity-levels:

Level 1 (e.g. TOGAF 7): ‘horizontal’ optimisation of the IT base, cleaning up the legacy mess and moving towards ‘single source of truth’ etc – any further IT-innovation will create increasingly intractable problems until this is done.

Level 2 (e.g. FEAF, TOGAF 8.1): integration with IT-related aspects of business strategy – IT-innovations are useful, but must be linked to and in response to top-down business strategy, not bottom-up ‘technology for technology’s sake’.

Level 3 (e.g. some aspects of TOGAF 9): bottom-up integration with CMDB etc for disaster-recovery planning and real-time impact-analysis, and cross-integration with service-management, quality-management etc as key sources of innovation.

Level 4 (e.g. Agile business-oriented variants of TOGAF 8.1 / 9): ‘spiral-out’ analysis to resolve business ‘pain-points’ and whole-of-enterprise opportunities for innovation; because we need to address true complexity and emergence here, many solutions at this level may involve little or no IT as such, or be more strongly cultural than technical (e.g. Enterprise 2.0)

EA is the custodian of a body of knowledge about enterprise structure and purpose. It has a key role as a bridge between strategy and implementation, via a Programme Management Office or equivalent. It does need to be proactive about innovation – but that innovation may come from anywhere, not just IT, and must always be linked back to enterprise purpose.

Innovation alone is not a substitute for enterprise purpose. If there isn’t clarity about purpose, and its implications and impacts across the enterprise – one of the key roles for EA – then no amount of innovation is going to help. Use EA to help sort out that concern before pushing for any further business innovation.

After yet another farrago for my would-be publishing, where the planned launch for the new book Disciplines of Dowsing at the British Society of Dowsers congress didn’t actually happen – everyone involved just kind of forgot, I guess – I’ve been taking stock of the actual results of my Tetradian Books venture, and together with that, the whole of the last couple of years’ work and being. Not exactly inspiring, really.

Some examples:

• Total sales at Megalithomania, back in May, where we supposedly launched the new edition of Needles of Stone: eight books (of which only four were the new book); total income, perhaps £70 at best; total cost to go there, a bit more than £300; overall loss, around £250 or so.

• Total sales at BSD congress, this weekend: eleven books retail, plus perhaps 20 wholesale; total income, perhaps £250 at best; total cost to go there, about £350; overall loss, around £100 or so.

• Total online sales of all six titles since May: 67 books; total income, somewhat under £500; total setup cost, somewhat over £1100; overall loss to date, around £650 or so.

So the effective financial result of six months’ flat-out full-time work since March, when the first title went off to press, is that I’ve wasted yet another thousand quid or thereabouts. That’s not including any of the frightening costs of living in this obscenely expensive country, either. Not exactly pension-fund material, shall we say.

In terms of impact, towards creating constructive change anywhere, all my efforts have fared just about as well as my finances. Precisely one (count ‘em – one) person in this benighted country has come close to a real understanding what I’m trying to do in enterprise architecture. If I’m lucky, the best I get from most people in ‘the trade’ is stares of blank incomprehension; if I’m less lucky – which has happened quite often here in Britain – I get a full-in-the face denigration not just of my work but myself as a person, for the unacceptable sin of ‘thinking different’. Not far off the same with most of the dowsers, and the rest of the pointless, pathetically self-obsessed ‘alternative’ scene: it’s painfully clear that most want to cling onto their delusory newage just as long as they can, and have no wish or intention to face any form of reality. Which, in turn, is equally true of the IT industry – utterly lost in their own self-important delusions, wasting everyone’s time, money and everything else, selling dreams that they know damn well they can’t deliver. Same is true, in fact, of pretty much everything else I’ve looked at professionally over the past decade – just don’t get me started on the failings and outright fraud of the domestic-violence ‘industry’, for example…

And I must admit I’m utterly sick of it all. I’ve been struggling too long, too hard, in too many areas and contexts, trying to get anyone to think, to see how ludicrously stupid so many – almost all? – of the usual approaches and models and frameworks really are, and that we really must do better, really, really urgently…

But I have to accept it ain’t going nowhere. Pretty much no-one is interested in what I do or what I say; and certainly no-one here is willing to pay for it. So in practice I’m back at being the Outsider again: as far as this society and culture and milieu is concerned, it seems, I have no societal function, no purpose, no role to play, and no place anywhere within it, in pretty much any sense of the word. Hence, unsurprisingly, no support either. And the endless loneliness out here on the Outsider edge hurts like hell: it always has, always does.

Quite where that leaves me, right now, I don’t know. Somewhere not exactly pleasant, that’s clear. Some painful choices up ahead, that’s also clear.

So yeah, a bit dispirited at present.

Oh well.