Innovation theory for the 1990s – revisited

It makes a long blog post, but this article seems worth putting up again after it came to mind at a meeting in London yesterday (21 Oct) to consider innovation policy in the 21st century, and the agenda of Sussex University’s Science Policy Research Unit under its new director Johan Schot. SPRU has a 50th anniversary in 2016, which reminds me that I went to their 25th anniversary conference way back in 1991. The result was this feature in a well-known weekly science magazine – much longer than any treatment of the subject they’d carry now, I think.

What struck me about the shorter discussion just now in London was how much the language and thinking of global constraints and sustainability has become woven into debates about innovation policy – at least until the contribution from Labour party science person Liam Byrne. He sounded more like a voice from the 1980s than anyone else, which was disappointing: decarbonising the economy did get a mention in his remarks but didn’t sound as if it was a matter of any particular urgency.

In fact his agenda for the UK, which was well thought out in its own terms, would have sounded pretty familiar, I think, to anyone who was part of the conversation in the late 1980s, when I worked (briefly) for the old Advisory Board for the Research Councils – an episode, incidentally, which probably accounts for this piece’s enthusiasm for the possibilities of bibliometrics. Anyhow, it reinforced the impulse to excavate this essay, in case others are interested in what may have changed, what stayed the same…

What drives the engines of innovation?

All scientists think they know how to spend the taxpayer’s money. But researchers in science policy are discovering that the choices are far from simple

When two dogged researchers in IBM’s Swiss research lab demonstrated high temperature superconductivity in 1986, they triggered an amazing burst of scientific activity, government investment and press speculation. Stories abounded of super-efficient power transmission, levitating trains and other wonders to come. The announcement was the starting gun for a race to exploit the discovery, with Japan, the US and Europe all vying for the lead. Academic and industrial laboratories eagerly joined in. In Britain, universities competed to host the first of the Science and Engineering Research Council’s interdisciplinary research centres, dedicated to work on high temperature superconductors. The following year’s Nobel prize award to the IBM researchers, Georg Bednorz and Alex Muller, further heightened the enthusiasm for applying their breakthrough.

What were all those who followed in the wake of Bednorz and Muller doing? Some answers are obvious: they were seeking a share in the glory; trying to understand the physics behind the phenomenon; trying to make money; trying to boost their national economies. But there was also something else. They were all proceeding from a set of shared assumptions about a linear model of innovation.

Of all the ways of thinking about invention and innovation, the linear model is the simplest. Most crudely, it assumes that scientists make unexpected discoveries, technologists take them up and apply them, then engineers and designers turn them into new products or processes. Thus, nuclear physics spawned the atom bomb, solid-state physics led to the transistor and then the microchip, and molecular biology will give us an AIDS vaccine.

Unfortunately, like most simple models, this one starts to come apart as soon as you look at a few real cases, and most people who study how science and technology interact now think that the linear model is wrong. But its grip on policy making is still strong. It is an easy model to think with and to incorporate in arguments to support your particular project. And there is no replacement as simple or appealing.

All this complicates the relations between science policy researchers on the one hand, and decision makers and pressure groups on the other. This was only too clear at a conference held in July to celebrate the silver jubilee of the Science Policy Research Unit (SPRU) at the University of Sussex. Almost the opening words of the conference came from Nathan Rosenberg, professor of economics at Stanford University in California and a leading economist of technical change. ‘Everyone knows that the linear model of innovation is dead,’ he said. There is no academic dissent from this proposition. But one head of a research council was quick to assure me after the session that the linear model is certainly still alive in the minds of members of the Advisory Board for the Research Councils (ABRC), and many civil servants and government ministers. The model may be dead, but it won’t lie down.

Listening to the rest of Rosenberg’s paper, it was easy to see why. For while the linear model is now called into question, few other generalisations emerge so strongly. In Rosenberg’s metaphor, what is needed now is a road map to identify the most influential traffic flows between science and technology. But Rosenberg immediately went on to say that no one can draw such a map yet. Small wonder that the linear logic, often also dubbed a ‘science push’ model of innovation, still creeps back into so many discussions of science policy.

We can best understand how science policy has arrived at this apparent impasse by looking back at its history. The foundation of SPRU did not quite mark the birth of science policy. Its first director, Chris Freeman, rightly harks back to J. D. Bernal’s The Social Function of Science as the fountainhead. But although the great Marxist crystallographer’s book was published on the eve of the Second World War, many of the research questions it raised were not taken up seriously until the early 1960s. So SPRU’s jubilee was a good opportunity to take stock of the growth of academic science policy research since then. The key question here is: what impact has this research had on those wrestling with the twin problems at the heart of science policy – how much to spend on scientific research, and what to spend it on?

The experience of the Second World War, and especially the development of radar and the atomic bomb, shaped in two ways what Freeman now sees as the initial phase of postwar science policy. First, it underlined the enormous importance of science and technology, one of Bernal’s main arguments. But, secondly, whereas Bernal argued that planning should be extended to the heart of scientific research, the physicists who dominated science advice to governments in the postwar years seemed to support an opposite conclusion. They were firm believers in ‘science push’, which they had seen in action in the way that the Manhattan Project to build the atom bomb followed on from the basic research of physicists in the 1920s and 1930s. They argued that you could plan large-scale technological developments, but science could be left to take care of itself.

In fact, the guiding spirit was much more like that of Bernal’s arch critic, the chemist and philosopher Michael Polanyi, who argued fiercely that the ‘pursuit of science can be organised . . . in no manner (other) than by granting complete independence to all mature scientists . . . The function of public authorities is not to plan research, but only to provide opportunities for its pursuit’. No need, then, for much else in the way of a science policy. Certainly no need for science policy research.

The demise of this ‘hands-off’ view had many causes. When economies were growing very rapidly it was easy to give scientists more cash, and easy to believe that research fuelled economic growth. When SPRU started, the research councils’ science budget in Britain was growing by more than 10 per cent a year, and the same was true in most other industrialised countries. Yet it became apparent relatively soon that it was very hard to demonstrate any clear correlation between research spending and the state of a country’s economy. Even if you stick with the linear model, the links between an individual discovery and a particular innovation can be hard to trace. As Rosenberg pointed out at Sussex, the timescales involved can be very long. There is no reason why the knowledge required to make a particular piece of technology work need be new knowledge.

How far back should you look for the science underpinning the laser in a CD player, for example? Further back than the early 1960s, when the first lasers were developed, certainly. Further, even, than the Second World War, although much of the technical know-how appeared then. In fact, you should probably begin in 1916, when Einstein first wrote about stimulated emission of radiation.

True, science today often gives the appearance of being closer to application than in Einstein’s day, as in the current feverish excitement to develop areas like biotechnology. Yet even this depends on the elucidation of the structure of DNA nearly 40 years ago. You can always argue about where to draw the baseline. And in general, as Rosenberg said, many who argue for the economic importance of science overemphasise the significance of the contemporary frontiers of science because they want larger research budgets. They neglect old science because it has already been paid for.

All this implies an unpalatable conclusion for defenders of national basic research spending. As Rosenberg said: ‘The commercial benefits of basic research need not be captured by firms in the country where the basic research was performed.’

At first, this realisation did not diminish enthusiasm for supporting basic research. But policy makers’ concerns did start shifting to what came after basic research. They began to think, for example, that the final product had to be envisaged when judging the value of science. This process is dubbed ‘technology pull’, or nowadays ‘market pull’, in contrast to ‘science push’. But how should these projects be chosen? The question became more urgent when governments felt new pressures in the late 1960s and early 1970s to mitigate the undesirable consequences of earlier economic growth, and to face the reality of a decline in growth.

By the mid-1970s, it had become all too obvious that research could not go on expanding exponentially. There seemed no slowdown in the growth of scientific opportunity, but increases in money to match were out of reach. Researchers could (and still do) write proposals for more and more expensive projects, but the idea that they would continue to spend an expanding share of the national wealth was not practical politics. Polanyi’s view that scientists should be left ‘to distribute themselves over the whole field of possible discoveries’ was simply no longer plausible. Recognising that the science could no longer be left to look after itself, science policy advisory committees were forced to started choosing between disciplines.

And they had no obvious grounds for their choice, except to say that some fields looked more likely to yield benefits than others. As John Kendrew’s crucial report on high energy physics to the Advisory Board for the Research Councils put it in 1985, ‘probabilistic assessments can be made . . . it is not meaningless to assert that research on black holes is less likely to provide utility by the turn of the century than research on self-organising systems.’

Such arguments carried less and less conviction. On the one hand, anyone could play this game, and have opinions about which science to support. On the other hand, the attempt to emphasise particular fields as worthy of support focused critical attention on the process by which the supposed benefits of those fields arrived – and they generally became harder to identify on closer examination.

At its sternest, such an examination can make a policy on basic research seem almost incidental. Rosenberg closed his paper at the SPRU conference by emphasising the influence of general economic conditions on research spending. This is more or less an inversion of the proposition that science spending leads to strong economies. Maybe it is strong economies which lead to science spending? Considered as a series of investments, Rosenberg argued, science and technology policy-making is simply an aspect of economic policy-making. ‘The wrong set of economic policies can guarantee the failure of any set of policies directed towards the realms of science and technology, no matter how ingeniously conceived,’ he said.

The work of SPRU and others has now come to emphasise that basic research is only one of the things a country needs in order to be successful in commercialising new technology. In Technology and the Future of Europe, a collection of essays published to mark the Sussex unit’s jubilee, Pari Patel and Keith Pavitt discuss Europe’s technological performance as the collective product of national systems of innovation which vary widely. Their archetypes are ‘myopic’ systems, exemplified by Britain, and ‘dynamic’ ones, exemplified by Germany. The terms sum up wide differences: in financial systems, where the Germans give greater weight to long-term performance and are much more clued up about the value of companies’ intangible assets, like know-how; in methods of management, where, in Britain, financial competence dominates technical competence; and in education and training, where the German emphasis on vocational training serves its engineering industries especially well, although British strengths in higher education are good for sectors like chemicals and pharmaceuticals. The overall implication is that, if one goal of science policy is to increase national prosperity, there is little point in spending heavily on basic research without also trying to reform some complex social institutions which take a long time to change.

So does this mean the science policy researchers’ conclusions are largely negative for basic research, or at best give it only a small place in a policy framework which should include everything from industrial subsidies to environmental regulation? Not quite. While news of the demise of the linear model of innovation still comes fresh to some, in its wake there is a growing body of work which enriches understanding of how sciences and technologies interact. Although Nathan Rosenberg’s road map is not yet drawn, some of the main routes are visible.

There are perhaps three strands in this kind of research which contribute to an effort to replace the linear model with something that is more open to the complexities of the real world, yet is still tidy enough to be of some use as a model – which after all means there must be a certain amount of simplification.

First, there are the economists, compiling and inspecting their statistics. These cover national R&D spending (civil and military, public and private), as well as patents, balances of trade, and performance of particular industries or companies. Following such figures over time, and comparing countries or industrial sectors, can tell you a number of things. Take, for example, recent work by Pavitt of SPRU, who has long used US patents as an index of technological innovation. With Patel, he is now quarrying a database which profiles the technological strategies of almost 700 of the world’s largest companies over the last 20 years. Between them, these concerns produce about half the world’s technology, judging by the volume of patents filed.

Among other things, the two researchers find that, although there is much talk of the ‘globalisation’ of production, around 90 per cent of the innovative work of these companies takes place in their home countries. It appears to be easier to manage the development and first introduction of new technology in the country where the main centres of planning and decision lie. Very large firms, in fact, tend to be strongly shaped by the histories of their countries of origin, and by their patterns of natural resources, consumer preferences and traditions of skilled work. Companies accumulate a particular set of competences that enable them to innovate in some sectors, and not others.

This rootedness of firms in a national environment is one element in the case for maintaining a national scientific base – not for specific benefits, which remain uncertain, but as a general capability. The results of research may be an international resource, but the people trained in university and polytechnic laboratories are largely a national resource.

A second strand in the study of science policy involves detailed examination of individual innovations by economists and historians of technology. You can call this storytelling, or be more academic and speak of case studies, whose value lies in their illustration of the complexities of the whole business.

In the hands of a scholar like Rosenberg, for example, such studies can be made to reveal some of the many ways technology influences science, rather than the other way round – technology pull in a different sense. At the turn of the last decade, Rosenberg’s celebrated book Inside the Black Box helped to persuade many people that technological advances often shape the scientific agenda. At the Sussex conference, he returned to this theme with a new emphasis, as part of his sketch of the ‘road map’ science policy researchers now need to work towards.

Today, he suggests, one definition of a high-tech industry is that ‘it is one in which problems that arise at the technological frontier exercise a major role in shaping the research agenda of science’. An example of this, he believes, is that industrial problems throw up observations or stumbling blocks which are unlikely to crop up in a university laboratory – an underwater cable which corrodes unexpectedly fast, perhaps, or the surface heating of an aircraft wing as it breaks the sound barrier. ‘The fact is that industrial activity, especially but not only in high-tech sectors, provides unique observational platforms from which to observe unusual classes of natural phenomena.’

On the other side of the interaction, Rosenberg points out that the focus on scientific knowledge has led to a neglect of the importance of scientific instruments. In a number of key industries, instruments which were first developed as tools for laboratory investigation – as a by-product, if you like, of the search for new knowledge – have become essential capital goods, at the heart of production.

‘Much, if not most, of the equipment that one sees today in an up-to-date electronics manufacturing plant had its origin in the university research laboratory,’ he observes. For example, ion implantation, now used for ‘doping’ microchips, stems from expertise in manipulating particle beams first developed in high energy physics machines. Synchrotron radiation, first encountered as an irksome by-product of accelerator rings, became a research tool in condensed matter physics and biology and is now poised to assume a new role in submicron lithography for chips. And microelectronics products have long been examined for quality under the scanning electron microscope, which was originally a laboratory aid.

Similar tales could be told about industrial process control, robotic sensing, cryogenic techniques for handling rocket fuel, or superconducting magnets now used in medical scanners. Taken together, ‘they constitute part of the benefits of basic research that are separate and distinct from the benefits flowing from pure scientific knowledge and the eventual applications of that knowledge’. In this respect, it may be that the linear model, in tandem with traditional academic snobberies about the value of ‘pure’ knowledge as opposed to hardware, actually undervalues science’s contribution to industry.

As these examples indicate, work like this can outline some landmarks on the map, but it tends towards the anecdotal. We don’t know how such features fit into a complete landscape. So is there any prospect of making the mapping Rosenberg seeks a bit more, shall we say, systematic?

Perhaps there is, in a third strand of science and technology policy research which is now beginning to bear fruit: the analysis of scientific publications and citations, normally termed bibliometrics. The introduction of computerised databases of papers in scientific journals, and the citations they list to the rest of the literature – principally by Eugene Garfield’s Institute for Scientific Information – was initially a boon for scientists searching the archives to find out what was going on. But this kind of data has increasingly been used by policy researchers to investigate the output of laboratories that use up taxpayers’ money.

Bibliometrics has found two main uses: as an index of the productivity and impact of countries, institutions or even research groups; and as a probe of interactions between groups of research workers, and of the state of different fields of research. By looking at who is citing whom, for example, and how fast it is happening, you can get some notion of the way the patterns of knowledge growth are changing over time. Dutch researchers have christened this ‘science dynamics’.

These techniques are useful for making relatively crude productivity assessments, and SPRU led the way in using bibliometrics to compare large scientific facilities. The cash squeeze, combined with the general search by the government in the 1980s for ‘performance indicators’ for publicly funded work, meant this approach was adopted, with modifications, by both the ABRC and the University Grants Committee (now the Universities Funding Council). This kind of assessment, though imperfect in many ways, is now entrenched in the bureaucracy of the UFC and the research councils.

However, while these productivity assessments naturally preoccupy many academics, the other uses of bibliometrics may turn out to be more interesting. Part of their appeal to policy makers at first was as a surrogate for knowledge about areas of science which would lead to valuable technologies. Bibliometric analysis can cut short the wait for the moment when benefits arrive according to the linear model, and at least confirm quite quickly that something interesting is happening – a useful aid when you have to decide allocation of funds between fields. That this was a strong reason for looking seriously at such techniques was plain in a paper I wrote in 1988 with the ABRC chairman, David Phillips, summarising the advisory board’s experience until then. As we said: ‘While ABRC has a strong interest in the economic and social benefits of research . . . these commonly take many years to appear and are hard to measure. The appeal of bibliometric techniques, despite their restriction to scientific publication, is their relative immediacy and wide applicability to the study of scientific outputs, whether in basic or strategic research.’

An added appeal is that they enable science policy researchers to test some of the assumptions which too easily creep into debates on science policy. The best example so far is the demonstration by Diana Hicks and Jim Skea of SPRU that there is no link between the number of papers individual researchers publish and the size of the departments where they work. This conclusion, which holds good for chemistry, physics and the earth sciences in Britain, runs directly counter to current funding which favours departments larger than some supposed ‘critical mass’ for effective research.

Beyond this, work which brings in citations as well as publications offers intriguing possibilities for illuminating the details of connections in science – between fields, between people, between institutions, between countries. If you bring in the more recent databases of patents and the references they make to the journal literature, this extends to connections between science and technology.

Take just two examples described at the SPRU conference. Ben Martin and colleagues from Sussex are looking at the use large companies in Britain and Japan make of outside researchers. Despite the obvious constraints of commercial life, many companies publish widely in the open literature. ICI in Britain, for example, produces more than 400 papers a year, 60 per cent more than it did a decade ago, and more than many good-sized universities. Martin’s study of the papers published by British companies reveals that an increasing proportion are written in collaboration with outside authors, most commonly from universities. More than 40 per cent of BP’s papers involve outside co-authors, for example, as do 34 per cent of ICI’s papers. And the early findings indicate that the largest proportion, just over 30 per cent, of these corporate-academic links are made in Britain.

The project still has a way to go. There are plans to extend the analysis to patents, and their citations to research, to compare the findings with similar work on Japanese companies, and to put the raw numbers in context with case studies and interviews with industrial research managers. The result should be a very detailed picture of the interconnections between corporate R&D and academic science.

The advent of patent databases has also been extensively exploited by the American researcher Francis Narin, one of the pioneers of the analysis of scientific publications in policy research. Narin, who founded the private company CHI Research over 20 years ago, is now looking at the patterns of citations to the academic literature found in patents approved by the US authorities. Looking at trends between 1975 and 1989, he finds that patents citating scientific papers are becoming more common. They are generally subject-specific – that is, patents for drugs and medicines cite papers in the life sciences, and so on. The time-lag between scientific publication and patent citation is decreasing, and has virtually disappeared in some fields.

All this is evidence for increasingly rapid interchange between science and technology, and evidence that the interchange is fruitful. The science-intensive areas of patenting, including electronics and biotechnology, are growing more rapidly than others, such as heavy engineering. The citation of contemporary science in patents would once have been used to support the idea that ‘lags’ in a linear scheme of discovery-then-application were reducing. But the finding works just as well to support the contention that the kind of applications of science that are sought shape the science that is being done. The genes of the pig are being mapped today, for example, not because the creature is scientifically more interesting than, say, the tapir, but because pig business is big business. When pig papers start turning into patents, it will be because that was the intention all along.

What work like Narin’s also suggests – and you can get very similar results using the database built up from the European Patent Office – is that there are still strong national ties in science and technology. This chimes with Pavitt’s observations about the behaviour of large companies. Although knowledge knows no national boundaries, patents filed in the US which originate in Britain are significantly more likely to cite scientific papers by British authors than Britain’s share of the world’s papers would suggest (see Diagram). And the same is true for other countries.

As results like this mount up, it becomes plainer that a national science base is a vital resource. But this general truth is still not underpinned by much in the way of specific prescriptions for different fields. The policy makers’ ideal of a clearer understanding of the relations between funding and the growth of knowledge remains a tall order. Perhaps one day some of the detailed questions which vex decision makers will be posed in bibliometric terms. For example, if you are becoming increasingly selective in your funding system, it would be very nice to know whether the best science, which you want to keep, needs the results and insights of all the middle-ranking researchers to keep it going. Can you just have the cream, or does it need some milk to sit on? The answer is almost certainly that you can make do with just the cream, but no one knows how much cream, or whether it differs in different fields, or at different times, nor even whether the separation can be reliably performed.

Spending on science, as on most other things, consists largely of buying what you bought last year, plus a little bit extra if you are lucky. So can the new analyses help pinpoint areas which no longer produce any cream at all? Some areas must be pruned to make space for new opportunities, but what indicators can you use?

So far, so frustrating. The demise of the linear model implies the need to make choices in science, and to manage the whole innovation process. But there are few clues about how a planned selection should be made, either to maximise efficiency in the simple sense of getting the best science for the money, or in the larger task of serving economy and society. Science policy is still much more art than science.

Even so, the demise of linear thinking should be a welcome challenge to the collective imagination. For the linear model was, at its heart, a determinist scheme. Science set the agenda, developing according to its own internal logic. Technology then appeared as a more or less inevitable consequence of the boffins’ unguided discoveries. It seemed the rest of the world had little choice in the matter.

Now it can be seen that a whole variety of people are involved in the interactions between science and technology. Product planners, designers, marketing experts, consumers, regulators, forecasters and pressure groups all intervene, influencing what innovation theorists call the ‘selection environment’ which produces particular technologies from a spectrum of possibilities. The choices they make penetrate science, fixing the direction of inquiry if not its results. One implication of the decline in linear thinking is that, through the sum of all these choices, we eventually get the technology, and perhaps even the science, we deserve. It is a conclusion Bernal would have approved of.

(November 1991)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s