Search results

This paper presents an ongoing research project investigating the use of bibliometrics for research evaluation and resource allocation purposes, and to investigate the effects of these evaluation models on individual, organizational and political levels. A main focus has been on bibliometrics based evaluation practices. The use of bibliometrics for local resource allocation at Swedish universities was analyzed (Hammarfelt et.al, 2016; Hammarfelt & Åström, 2015), finding a great degree of variation both in how and where bibliometrics is used; and while the introduction of a bibliometrics based national model for resource allocation in Sweden in 2009 probably had an effect on the increasing use of bibliometrics in local allocation models, there are also great variation in terms of bibliometric indicators and allocation models being in use. But bibliometrics is not just being used in evaluation and resource allocation models at a systemic level. Another important part of the use of bibliometrics in research evaluation practices is how it is being used by academic scholars and scientists serving as external assessors of candidates for academic positions or of research funding proposals. This practice has been studied through analyses of application assessment reports in economics, history, and biomedical research at Swedish universities (Hammarfelt, forthcoming; Hammarfelt & Rushforth, in press). The other main focus of the project thus far has been on the underlying technologies, or the infrastructure of bibliometrics and research evaluation systems. A main aspect of the 'infrastructure' theme is the challenges of understanding complexities and interactions of different technical or auxiliary systems, and their role in evaluation processes. This has been investigated empirically in a minor study of the role of knowledge organization systems in bibliometric analysis and evaluation (Åström et.al, 2016). Another empirical study of infrastructures under development is a study of current research information systems (CRIS). From a theoretical perspective, a conceptual framework has been tentatively developed, describing infrastructures, not so much as specific technical systems, but as systems of relations between hardware, software and people (Åström, 2016). This was operationalized by defining: 'hardware' as databases, software for bibliometric analyses, and so on; 'software' as the evaluation systems, the bibliometric indicators, and the allocation models; and 'people' as the stakeholders involved in evaluation practices. The second aim of the project is to investigate effects of the use of bibliometric methods for research evaluation, on the research system, and on and the academic researchers. This can be seen from two different points of view. One is how academic researchers themselves use bibliometric methods when evaluating their peers assessing applications for research grants and academic positions. The other is effects along the lines of scholars and scientists changing for instance publication behavior to adjust to evaluation systems and resource allocation models. While the former point of view might be more contextually related to traditions within research fields, the other might be more related to local or national research policies. The interplay between these perspectives is of high theoretical interest for the project to develop further, but has to a certain extent been investigated within a master's thesis supervised within the project (Nästesjö, 2016); continued in a PhD-project, analyzing the relation between evaluation practices and academic socialization. The master's thesis, suggest that disciplinary differences, career stage, and academic age are important factors for understanding the relation between research evaluation systems and research practices, where there are potential conflicts between disciplinary norms and political practices, conflicts that affect the behavior of individual scholars, as well as disciplinary norms.

The research area of Digitalization and Privacy, as defined and delimited within the context of this knowledge review, (mainly through the selection of search strings), is in a phase of development. The present study has been delimited to articles in English that have been published in scientific peer-reviewed journals. During the past ten years, the number of scientific articles per year has increased more than five times. For example, a search of the database WEB OF SCIENCE for the year 2006 renders 13 matches, while an identical search for the year 2014 renders 72 matches. In the present systematic knowledge review, two types of investigations have been conducted. First, a bibliometric analysis that aims to produce a comprehensive overview of the current state of the research in the area at a statistically analytical level. Second, a systematic literature study that has identified relevant scientific articles, analyzed their content and categorized them. The bibliometric analysis demonstrates that research on digitalization and privacy is quite strictly divided, mainly between three scientific fields. In other words, communication between the various fields (i.e., intertextual references and citations of each field's research) is somewhat limited. The research fields can be described as: (a) a technical field that is largely concerned with systems development, (b) a legal field that focuses on issues regarding legislated protection of privacy, and (c) a social sciences and behavioral sciences oriented area that includes informatics, psychology, sociology, political science and marketing and management research and more. This systematic literature review, based on close reading of all included articles, shows a lack of clear, mutually shared, conceptual terminology and common understandings of methodologies within the various scientific disciplines. However, there are a number of areas (or focuses of research) that recur frequently. The five dominant areas are: (a) technology, (b) legislation, (c) the state, (d) theory, and (e) working life. Further, the research identifies different approaches to digitalization and privacy. First, as a problem (or, perhaps, a challenge) that can be managed using new, improved and more privacy-sensitive technology. Secondly, as an opportunity to work towards achieving good values such as improved health, through practical applications of potentially sensitive data. Thirdly, as a threat to citizens and employees. And, finally, as a relationship of ex-change between usefulness and risks, for example, with regards to state needs for information in order to prevent threats and protect the citizens rights of privacy. It is also strikingly clear that there is insufficient knowledge of the relation-ship between digital surveillance and potential behavioral changes in society. Various studies highlight that a lack of respect for privacy risks leading to reduced Internet use and reduced political involvement (at least on the Internet). However, at this point in time there is no empirical evidence to support that this is the case.