Product design--lessons from the reuse field

What software reuse can teach KM about collaborative innovation

KM is often presented as a way for a company to get more consistently good at what it is already good at, through the effective capture, codification and dissemination of existing knowledge. Less often discussed as a value proposition is the role of KM in innovation. Many KM advocates proceed based on an unspoken faith that getting better control of a company’s knowledge assets will enhance innovation. But there is little hard evidence to back up those intuitions. And although innovation is often cited as a broad motivator for KM efforts, it is not supported explicitly in most KM process models and tools.

Some techniques directly address the innovation aspects needed within KM. Those techniques were originally developed within the field of software reuse, which has been wrestling with knowledge management problems (under a different name) for decades. Starting with the 1968 NATO software conference, researchers and practitioners have confronted the complex technical and organizational challenges of getting software organizations to more systematically reuse software across multiple systems. Some of those challenges are particular to the software industry or to the nature of software as an intellectual asset; others are general KM problems now being faced by many organizations in the knowledge-based economy.

Due to the nature of software and the complexity of reusing software as compared to other forms of codified knowledge such as documents, the reuse field has looked in considerable depth at the processes required to re-engineer artifacts into true reusable assets. That has resulted in an extensive body of formal methods, process models, guidebooks, example projects, metrics and repositories of reusable code. Much of that work is relatively unknown in the KM community.

To address the need to systematically redesign software for reuse, the field developed a body of methods generally known as "domain analysis" or domain engineering. In the reuse context, the term "domain" denotes a well scoped area of functionality within a set or class of software systems. The need for formal domain engineering arose within software organizations developing large-scale applications in vertical markets--systems still built by and large as "one-off" custom developments.

Re-engineering artifacts into assets

Creating software assets that can be reused in multiple applications is a more complex problem than that usually addressed in KM, as it involves re-engineering or repurposing software (or other knowledge-intensive artifacts). Simply put, an artifact created for use in one project context is placed into a central repository where it is made available to others.

Consider the process of including a customer proposal in a document repository. In reuse terms, it is an artifact created for a specific contract or setting. As an artifact in that sense, the proposal can serve multiple uses. It provides information about the specific project where it originated; thus, it belongs in a project repository and eventually in a historical archive. In addition, it can provide an example of a proposal for other proposal writers; thus, in typical document management, the proposal might be placed in a corporate archive to be accessed by people working with other customers.

There is an important limitation in the proposal’s use in that latter sense. It cannot be directly reused as a document for a new customer; some parts might be directly usable (company boilerplate), some relevant (a contracting strategy) and some specific to the original customer or project. To be repurposed in that way, the structure and content of the original proposal would need to be analyzed at a relatively fine-grained level; in effect, it must be converted to some sort of template or customizable document. That also requires analysis of the original context of the application: which features are sensitive to that context and which aspects of that context will hold true for the range of application opportunities anticipated for the document. All of that is implied in converting the proposal from "artifact" to "asset."

In a document like the example proposal, though, some of the original context might be lost. It is relatively easy to inspect the final form and discern where contextual dependencies have crept in. One can look at a proposal, see questions reflecting a certain type of customer and realize that the proposal was intended for only that class of customer. While that results in an artifact that gets tweaked and modified with each occasion of reuse, it is at least manageable.

With software, we have no such luxury. The experience of software engineers has been that this kind of ad hoc or opportunistic reuse is not cost-effective in many instances. Contextual assumptions are buried invisibly in the structure of the software itself. Documentation is rarely in step with the final form of the software, and in any case is generally inadequate because the original author had only tacit understanding of the specific context in which he or she was performing. Furthermore, one contextual assumption can ramify throughout the code in subtle ways. Because of hidden context, reused code is all too often doing almost, but not quite, everything you need done, plus a little bit more you don’t need. Over time, software engineers’ experience is that this form of reuse more often than not was more expensive than rewriting the code from scratch. In some dramatic cases, the consequences are not merely lost time and effort but serious mission-critical failures.

Domain engineering

Ad hoc approaches to reuse often reflect a belief that there is a "right" abstraction for a given component that optimizes for reuse in some absolute or general sense. (That is analogous to one-size-fits-all best practices documents and standards in the KM field.) ODM encourages the insights that spring from studying artifacts drawn from different contexts. By studying a carefully chosen set of example work products developed for different, specific application contexts (exemplars), ODM facilitates reflection and discovery that reveal previously hidden contextual assumptions. Describing both commonalties and variability across that exemplar set, discrepancies and deltas raise implicit assumptions to awareness. By searching for relevant contextual aspects of the different system contexts to explain the observed range of variation, the method allows recovery of implicit context from artifacts studied as part of the domain, with clear criteria for how much context is needed to be captured.

That aspect of the process involves creating a common conceptual framework for looking at a set of examples exhibiting some range of variability. It is equally important to the process to support alternative ways of modeling the same material. Thus, domain engineering is inherently collaborative in a deep sense, facilitating creation of a shared conceptual language for exploring a given design or market space.

Although the desired end product of a domain engineering process is a set of re-engineered assets, experience has shown that the knowledge modeling and synthesis phase of the process produces the most significant and immediate value. While the artifacts examined gain in value due to recovered context, and reusable components created will be more predictably and robustly reusable, the domain models themselves become the most valuable knowledge asset resulting from the process.

Equally important, if less tangible, is the conceptual transformation resulting from the process: a "cognitive shift" that enables software engineers to view the systems they create and their relationship to the surrounding organizational context in new ways. That creates enhanced capacity to envision new system capabilities, both in terms of innovative features and feature combinations, and in terms of new markets for domain functionality. Exploring why certain feature combinations were never provided often elicits subtle contextual information. Domain models reveal the potential for incremental and fine-grained innovations, e.g., novel feature combinations based on precedented features. Innovative ideas for novel designs or applications for domain functionality arise as modelers discover previously hidden contextual assumptions in exemplars. Thus, innovation seems to fall out of the ODM process.

Martin Griss, HP’s "reuse rabbi," is a spokesperson for the benefits of systematic reuse. The landmark book making the case for reuse, "Software Reuse: Architecture, Process and Organization for Business Reuse" (Addison-Wesley-Longman, 1997), cites impressive industry statistics: product times to market cut in half, dramatic cost savings, with associated improvements in software quality and maintainability. Yet Griss’ current perspective underlines the less easily quantified benefits of innovation: "While the time to market and cost savings benefits of reuse are considerable, even more significant may be increased agility in rapidly introducing innovative products for emerging markets. Product line architectures, designed to be robust against change, combined with pluggable and flexibly configurable components, can allow new combinations of product features within a specified domain. This gives reuse strategic, not just tactical, value."

Domain engineering techniques drawn from work in the reuse field can directly support innovation. While market-changing discoveries and leaps of insight will always depend on people’s knowledge, collaborative dynamics and inspiration, there are processes that can directly create conditions to maximize those opportunities. We may not always have time to take the knowledge artifacts studied, categorize, restructure and repurpose them and get them into a well-managed archive. But innovations and insights derived from comparative analysis of those artifacts, in a domain-oriented context, can lead directly to new product, service and market concepts.

In an example of software reuse’s role in product design, the Domain Specific Software Architecture (DSSA) program (www.owego.com/dssa) applied advanced expert system decision support to code generation. The DSSA program was evaluated as having improved productivity by a factor of 14, with an eightfold reduction in errors.

Said Lockheed-Martin’s (www.lockheedmartin.com) Will Tracz, former DSSA program manager, "In my opinion, looking back on the project, where we got the most bang for the buck in reuse was not by reusing components alone, but by reusing the process--the mental steps and decisions involved in developing these systems--supporting these with an automated, codified to-do checklist."

Strong evidence of the software reuse field’s close links with knowledge management: learning the essential importance of codifying knowledge about the software process to achieve desired levels of productivity improvement and accelerated time to deployment.

Author’s comment:

Domain engineering processes offer significant opportunities to support innovation as an integral aspect of knowledge capture, codification and formalization, and can be effectively transferred to broader knowledge management activities. Curiously, though, you will not find these techniques described in terms of innovation within most reuse literature. The early emphasis of the reuse field was on productivity improvement as a primary motivator.

In my opinion, that emphasis on productivity over innovation was a primary mistake of the reuse field--one which led to years of difficulty in getting reuse initiatives adopted. Systematic reuse requires a heavy investment in supporting infrastructure, new kinds of individual roles and skills, team interactions, and organizational structure and strategy. Frankly, for most organizations, it’s just too much trouble to go through to solve a productivity problem. Companies that do institute reuse for that reason are probably so slow moving in the competitive marketplace that they will have strategic problems far outweighing the productivity issues addressed by a reuse effort.

I see some of those mistakes being relived now, as if for the first time, by the KM field. (But the reuse field itself has been notorious for not learning from its own mistakes and constantly reinventing its terminology--ironic commentary on a community whose main topic is the art of not reinventing the wheel!)