Suchergebnisse

Bill Starbuck reflects on the challenges associated with social science research - the limits of rationality, the lack of reliability in many research findings and the social shaping of research agendas, cultures and judgements. He discusses some of his own research projects and various methodological debates.

This essay proposes ways to improve editorial evaluations of manuscripts and to make published research more reliable and trustworthy. It points to troublesome properties of current editorial practices and suggests that editorial evaluations could become more reliable by making more allowance for reviewers' human limitations. The essay also identifies some troublesome properties of prevalent methodology, such as statistical significance tests, HARKing, and p-Hacking, and proposes editorial policies to mitigate these detrimental behaviors.

Changes in societies and communication technologies are forcing universities and related post-secondary institutions to change quite significantly. Some of these institutions face threats to their existence; many institutions will have to adopt new structures and find new rationales to attract students and faculty. Universities have long relied on academic publishers to provide evaluations of research quality via editorial review, but the publishing industry has gone through tremendous changes and no longer resembles the industry that once provided these evaluations. Neither publishers nor universities have found a clear path toward future relationships. A central issue for researchers is their inability to agree about the quality of research. This disagreement arises partly from the complexity of research and research reports and partly from humans' limitations, and it creates great ambiguity for editorial reviews and personnel reviews. Recognizing the unreliability of evaluations, however, can free researchers to take more control over their professional lives and can make science work better. (HRK / Abstract übernommen).

Research provides some observations about learning from events that appear to be rare or quite unusual. All learning has uncertain consequences, but learning from rare events is especially problematic. Learners see many idiosyncrasies and exogenous interference, tendencies that suppress learning on an organizational scale. Rare events also rouse uncertainty and bring on reactions to uncertainty such as wishful thinking, reliance on prior beliefs, biased probabilities, a search for more data, cautious action, and playing to audiences. The most important contingencies affecting these reactions are the content and strength of prior beliefs: people are unlikely to learn if they think they have nothing to learn. Although learning from rare events is statistically unusual, and effective learning from rare events is rare, both individuals and organizations can benefit significantly from active efforts to learn from rare events.

Articles in high-prestige journals receive more citations and more applause than articles in less-prestigious journals, but how much more do these articles contribute to knowledge? This article uses a statistical theory of review processes to draw inferences about differences value between articles in more-prestigious versus less-prestigious journals. This analysis indicates that there is much overlap in articles in different prestige strata. Indeed, theory implies that about half of the articles published are not among the best ones submitted to those journals, and some of the manuscripts that belong in the highest-value 20% have the misfortune to elicit rejections from as many as five journals. Some social science departments and business schools strongly emphasize publication in prestigious journals. Although one can draw inferences about an author's average manuscript from the percentage in top-tier journals, the confidence limits for such inferences are wide. A focus on prestigious journals may benefit the most prestigious departments or schools but add randomness to the decisions of departments or schools that are not at the very top. Such a focus may also impede the development of knowledge when mediocre research receives the endorsement of high visibility.