Bibliographic citations in the evaluation of the scientific activity: meaning, consequences and an alternative conceptual framework
DOI:
https://doi.org/10.31055/1851.2372.v55.n3.28723Keywords:
Bibliographic citation, evaluation of science quality, conceptual framework, metricsAbstract
Background and aims: In the evaluation of scientific activity, metrics are applied based on the number of bibliographic citations received by articles from a journal or from a scientist. The objectives of this work are to establish the meaning of a bibliographic citation, discuss the consequences of the use of metrics based on bibliographic citations as a synonym of scientific quality including its effect in the field of Botany, and propose an alternative conceptual framework for evaluating the scientific activity.
Results: A series of factors prevent the statistical support of these metrics: size of the potential audience, the variation in publication and citation practices between the different disciplines, the long-tail statistical distribution of citations, the journal and language of publication, and that an observed statistical regularity will tend to collapse once pressure is put on it when used for control purposes. The consequences of using citations to evaluate are: inhibition of creativity, reification of the scientific achievement, assess the product based on the number of its consumers, the journals with the highest number of citations become a power factor, devaluation of topics with local or regional value, science becomes an industrialized activity, metrics replace judgment, and generating bibliographic citations is part of the objectives of a scientific article.
Conclusions: The origin and validity of these metrics are the consequence of a market society, that is, a way of life organized on the basis of market reasoning and morality and where human and social relations are mere consumer relations.
References
AAD, G., B. ABBOTT, J. ABDALLAH (y 5151 autores más). 2015. Combined measurement of the Higgs Boson mass in pp collisions at √s = 7 and 8 TeV with the ATLAS and CMS experiments. Phys. Rev. Lett. 114: 191803-1–191803-33. http://doi.org/10.1103/PhysRevLett.114.191803
ADLER, R., J. EWING & P. TAYLOR. 2009. Citation statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). Stat. Sci. 24: 1-14. http://doi.org/10.2307/20697661
ALBERTS, B. 2013. Impact factor distortions. Science 340: 787. http://doi.org/10.1126/science.124031
BORNMANN, L. & H.-D. DANIEL. 2008. What do citation counts measure? A review of studies on citing behavior. J. Doc. 64: 45-80.
http://doi.org/10.1108/00220410810844150
BORNMANN, L., R. MUTZ, S. E. HUG & H.-D. DANIEL. 2011. A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants. J. Informetr. 5: 346-359. https://doi.org/10.1016/j.joi.2011.01.006
BROOKS, T. A. 1985. Private acts and public objects: An investigation of citer motivations. J. Am. Soc. Inf. Sci. Tec. 36: 223-229. https://doi.org/10.1002/asi.4630360402
CAMPBELL, P. 2008. Escape from the impact factor. ESEP 8: 5-7 https://doi.org/10.3354/esep00078
CLAYTON, T., C. T. LAMB, S. L. GILBERT & A. T. FORD. 2018. Tweet success? Scientific communication correlates with increased citations in Ecology and Conservation. Peer J 6: e4564. https://doi.org/10.7717/peerj.4564
COLE, S., J. R. COLE & G. A. SIMON. 1981. Chance and consensus in peer review. Science 214: 881-886. https://doi.org/10.1126/science.7302566
COZZENS, S. E. 1985. Comparing the sciences - citation context analysis of papers from neuropharmacology and the sociology of science. Soc. Stud. Sci. 15: 127-153. https://doi.org/10.1177/030631285015001005
COZZENS, S. E. 1989. What do citations count? The rhetoric-first model. Scientometrics 15: 437-447. https://doi.org/10.1007/BF02017064
CRISCI, J. V. 2006. One-dimensional systematist: Perils in a time of steady progress. Syst. Bot. 31: 217-221. https://doi.org/10.1600/036364406775971859
CRISCI, J. V. 2008. La barbarie del “especialismo” en un tiempo de extinciones. Anales Acad. Nac. Agron. Veterin. Buenos Aires 62: 97-107.
CRISCI, J. V. & M. J. APODACA. 2017. Los rankings globales de universidades y su función disciplinaria. Revista Mus. La Plata 2: 12-18.
https://doi.org/10.24215/25456377e039
CRISCI, J. V., M. J. APOCADA & L. KATINAS. 2019. El fin de la Botánica. Revista Mus. La Plata 4: 41-50. https://doi.org/10.24215/25456377e067
CRONIN, B. 1981. Agreement and divergence on referencing practice. J. Inf. Sci. 3: 27-33. https://doi.org/10.1177/016555158100300104
CRONIN, B. 1984. The citation process. The role and significance of citations in scientific communication. Taylor Graham, Oxford, UK.
DORA (San Fransico Declaration on Research Assessment). 2020. [consultado el 9 de mayo de 2020]. Accesible en: http://www.ascb.org/dora
FINCH, T., N. O’HANLON & S. P. DUDLEY. 2017. Tweeting birds: Online mentions predict future citations in ornithology. R. Soc. Open Sci. 4:171371. https://doi.org/10.1098/rsos.171371
FISTER JR., I., I. FISTER & M. PERC. 2016. Toward the discovery of citation cartels in citation networks. Front. Phys. 4(49): 1-5. https://doi.org/10.3389/fphy.2016.00049
FOUCAULT, M. 1976. Vigilar y castigar. Nacimiento de la prisión. Siglo Veintiuno Editores, Buenos Aires, Argentina.
GARFIELD, E. 2006. The history and meaning of the journal impact factor. JAMA 295: 90-93. https://doi.org/10.1001/jama.295.1.90
GONZÁLEZ-PEREIRA, B., V. P. GUERRERO-BOTE & F. MOYA-ANEGÓN. 2010. A new approach to the metric of journals’ scientific prestige: The SJR indicator. J. Informetr. 4: 379-391. https://doi.org/10.1016/j.joi.2010.03.002
GOODHART, C. A. E. 1976. Monetary relationships: A View from threadneedle street. In: RESERVE BANK OF AUSTRALIA (ed.), Papers in Monetary Economics, vol. 1, Revised version of six papers presented at the Conference in Monetary Economics, Sydney, July 1975, in various pagings. Sidney, Australia.
GOULD, S. J. 1981. The mismeasure of man. W. W. Norton & Co., New York.
GRAMSCI, A. 1934. Cuadernos de la cárcel, tomo 6, Cuaderno 22 (V), Americanismo y fordismo. Traducción al español de la edición crítica del Instituto Gramsci a cargo de Valentino Garretana, 2000, Ediciones Era, Puebla, México.
GRAMSCI, A. 1952. Il materialismo storico e la filosofía di Benedetto Croce. Primera edición. Enaudi Editor, Torino, Italia.
HICKS, D., P. WOUTERS, L. WALTMAN, S. DE RIJCKE & I. RAFOLS. 2015. The Leiden manifesto for research metrics. Nature 520: 429-431. https://doi.org/10.1038/520429a
HIRSCH, J. E. 2005. An index to quantify an individual’s scientific research output. Proc. Natl. Acad. Sci. U. S. A. 102: 16569-16572. https://doi.org/10.1073/pnas.0507655102
HORROBIN, D. F. 1990. The philosophical basis of peer review and the suppression of innovation. JAMA 263: 1438-1441.
https://doi.org/10.1001/jama.1990.03440100162024
KUHN, T. S. 1970. The structure of scientific revolutions. Enlarged 2nd ed. University of Chicago Press, Chicago.
LAWRENCE, P. A. 2007. The mismeasurement of science. Curr. Biol. 17: R583-R585.
https://doi.org/10.1016/j.cub.2007.06.014
LEE, J.-S. M. 2019. How to use Twitter to further your research career. Nature blogs, https://www.nature.com/articles/d41586-019-00535-w
LUKÁCS, G. 1971. History and class consciousness. The MIT Press, Cambridge, Massachusetts.
LUUKKONEN, T. 1990. Citations in the rhetorical, reward and communications systems of science. PhD thesis, Acta Universitatis Tamperensis, ser A, vol. 285, University of Tampere, Tampere, Finland.
LUUKKONEN, T. 1997. Why has Latour’s theory of citations been ignored by the bibliometric
community? Discussion of sociological interpretations of citation analysis. Scientometrics 38: 27-37. https://doi.org/10.1007/BF02461121
MACDONALD, S. & J. KAM. 2007. Aardvark et al.: Quality journals and gamesmanship in management studies. J. Inform. Sci. 33: 702-717.
https://doi.org/10.1177/0165551507077419
MACROBERTS, M. H. & B. R. MACROBERTS. 1986. Quantitative measures of communication in science: A study of the formal level. Soc. Stud. Sci. 16: 151-172. https://doi.org/10.1177/030631286016001008
MACROBERTS, M. H. & B. R. MACROBERTS. 1987. Measurement in the face of universal uncertainty: A reply to Stigler. Soc. Stud. Sci. 17: 335. https://doi.org/10.1177/030631287017002008
MADDOX, J. 1998. What remains to be discovered. Free Press, New York.
MARCUSE, H. 1964. One-dimensional man: Studies in the ideology of advanced industrial society. 1st edition. Beacon Press, Boston.
MOED, H. F. 2005. Citation analysis in research evaluation. Springer, Dordrecht.
MOXHAM, H. & J. ANDERSON. 1992. Peer review. A view from the inside. Sci. Technol. Policy 5: 7-15.
NATURE EDITORIAL. 2012. Alternative metrics. Nature Materials 11: 907.
https://doi.org/10.1038/nmat3485
PRIEM, J. 2014. Altmetrics. In: CRONIN, B. & C. R. SUGIMOTO (eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact, pp. 263-288. 1st edition. The MIT Press, Cambridge, Massachusetts.
ROUS, P. 1910. A transmissible avian neoplasm (sarcoma of the common fowl). J. Exp. Med. 12: 696-705. https://doi.org/10.1084/jem.12.5.696
SIEBERT, H. 2001. Der Kobra-Effekt. Wie man Irrwege der Wirtschaftspolitik vermeidet. Deutsche Verlags-Anstalt, Munich.
THE SHENZHEN DECLARATION DRAFTING COMMITTEE. 2017. The Shenzhen Declaration on Plant Sciences–Uniting plant sciences and society to build a green, sustainable Earth. PhytoKeys 86: 3-7. https://doi.org/10.3897/phytokeys.86.20859
TODD, P. A. & R. J. LADLE. 2008. Hidden dangers of a ‘citation culture’. ESEP 8: 13-16.
https://doi.org/10.3354/esep00091
VAN NOORDEN, R., B. MAHER & R. NUZZO. 2014. The top 100 papers. Nature 514: 550-553.
https://doi.org/10.1038/514550a
WEINGART, P. 2005. Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics 62: 117-131.
https://doi.org/10.1007/s11192-005-0007-7
WOUTERS, P. 1999. The citation culture. Doctoral thesis, University of Amsterdam, The Netherlands.
Downloads
Published
Issue
Section
License
Copyright (c) 2020 Jorge Crisci, Liliana Katinas
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Provides immediate and free OPEN ACCESS to its content under the principle of making research freely available to the public, which fosters a greater exchange of global knowledge, allowing authors to maintain their copyright without restrictions.
Material published in Bol. Soc. Argent. Bot. is distributed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.
