Order and entropy in scientific publishing. Some remarks on the proposed rules of evaluating scientific output

Abstract

The new evaluation rules proposed by the Polish Ministry of Science and Higher Education in July 2018 are set to cure some of the ailments of the existing system, notably the “punktoza” phenomenon (i.e. publishing for volume, not scientific quality). However, it should be pointed out that the method of fixing old “bugs” might in fact create some new ones. In this article I discuss three elements of the proposed regulations, namely: the principle of “inheritance of prestige”, treatment of chapters in edited volumes, and possible variants of ministerial registry of academic publishers. To address those issues empirically I use an existing dataset covering citation of books in 2009–2013 (Torres-Salinas et al. 2014). While the new evaluation rules apply relatively high value to chapters in edited volumes, they in fact have disproportionately low scientific impact. What is more, the correlation between citation of books and chapters in edited volumes is very low, casting doubt on the assumed “inheritance of publishers' prestige”. Finally, there seems to be a high risk that the registry of publishers will not reproduce the exponential distribution observed in the actual structure of scientific impact (and apparently sought by the new system), thereby jeopardizing validity of such evaluation.