Final version received November 1984

 

In this paper we present the results of a study on the potentialities of bibliometric (publication and citation) data as tools for university research policy. In this study bibliomettic indicators were calculated for all research groups in the Faculty of Medicine and the Faculty of Mathematics and Natural Sciences at the University of Leiden. Bibliometric results were discussed with a number of researchers from the
two faculties involved.

Our main conclusion is that the use of bibliometric data for evaluation purposes carries a number of problems, both with respect to data collection and handling, and with respect to the interpretation of bibliometric results. However, most of these problems can be overcome. When used properly, bibliometric indicators can provide a monitoring device for university research-management and science policy. They enable research
policy-makers to ask relevant questions of researchers on their scientific performance, in order to find explanations of the bibliometric results in terms of factors relevant to policy.

 

I. Introduction
The central issue of this study is the examination of the potentialities of quantitative, literature-based (i.e. bibliometric) indicators as tools for university research-policy. As a subject of extensive investigation we analysed the research performance of two large faculties ~ of the University of Leiden (Faculty of Medicine and the Faculty of Mathematics and Natural Sciences) for the period 1970-80. The study presented in this paper 2 'was actuated by the necessity for a large-scale project evaluation resulting from a drastic change in the allocation system at the University of Leiden. The academic staff allocation system for teaching and research at Leiden was originally based almost entirely on student numbers. In fact, in this old allocation system there was a 50 : 50 ratio between staff time spent in teaching and in research. As a direct consequence, disciplines with an increasing number of students also had an increasing research
capacity. A few years ago the allocation system at Leiden was changed radically to a research project grant system. Two major considerations led to this change. First, the need was felt to finance scientific research more explicitly on the basis of scientific quality. Second, it was felt that the research capacity of a discipline should be protected against the consequences of a continuous decline of student numbers (see Van Raan and Frankfort [1]). Essentially, in the new system a separate research financing channel has been created. This was achieved by a very considerable reduction of a proportionality factor which in the.....