Abstract
Citations to published work are gaining increasing prominence in evaluations of the research performance of scientists. Considering the importance accorded to gender issues in South African science, it is surprising that (to our knowledge) no research has as yet ascertained the extent of sex differences in citations to the published work of scientists in this country. Our literature study shows that studies that have been conducted elsewhere tend to neglect in their analyses important gender-related and other factors, such as the sex composition of multi-authored papers and the extent of foreign co-authorship. Against this background, we illustrate the difficulties inherent in measuring the quality aspect of sex-specific research performance by means of an analysis of a dataset of articles (n = 229) that were published between 1990 and 2002 in the field of invasion ecology and in journals included in the Thomson Reuters Web of Science. Each article has at least one South African author address. The results indicate that foreign co-authorship is a better correlate of high citations than the sex of South African authors, and this is true irrespective of whether the annual citation rate or window period is used, whether or not self-citations are excluded, and whether or not the number of authors is controlled for by calculating fractional counts. The paper highlights these and other considerations that are relevant for future gender-focused bibliometric research, both in South Africa and beyond.
Similar content being viewed by others
Notes
Also referred to as “normal counts” (Trifunac 2006, p. 1070).
In the latter case, however, the sex differences were small enough to lead Over and Moore (1980, p. 415) to conclude that “[t]he men and women were […] equally likely to employ authorship patterns that enhance individual visibility”.
Already in the early 1980s, ISI recognised the need “to treat all authors in every article as though they were listed first” (Garfield 1981, p. 269) and started developing a system that would allow this. In 1978, Garfield (1981) was the first to use these so-called “all-author data”. He does mention that “[d]eveloping all-author lists is considerably more complex than conducting first-author studies” (p. 269). However, his findings (the average author received 1,178 citations to papers on which he or she appeared as first author and 2,633 as co-author) clearly demonstrate “the importance of all-author data” (p. 275).
In some countries, it is possible to differentiate between the sexes based on the surname alone. For instance, in Poland more than sixty percent of all surnames can be determined as belonging to a man (those which end with “-ski”, “-cki” or “-owy”) or a woman (those ending with “-ska”, “-cka” or “-owa”) (Webster 2001). In Iceland, women’s surnames typically end in “dottir”, whereas the names of men end in “son”, which allowed Lewison (2001) to conduct one of the most extensive and detailed analyses of gender differences in citations thus far. Lewison (2001, p. 42) rightly comments that, “It would be very difficult to extend such a study to other countries unless there were a complete list of the country’s researchers available with their sexes so that comparisons could be made”.
See Boshoff (2005) for South African statistics in this regard.
Prof David Richardson, currently Deputy Director of the CIB and a Professor in the Department of Botany and Zoology at Stellenbosch University.
The South African Department of Science and Technology (DST) and National Research Foundation (NRF)—the two bodies responsible for the funding and administration of the Centres of Excellence.
Examples include invasive/invasion (management); alien/plant invasions; naturalized/non-indigenous; indigenous; native; exotic; biological invasions, alien species, invasive alien species and invasion biology/ecology. In subsequent searches, the net was thrown wider, as broader terms and their derivatives (e.g., biological diversity and biological control) were used to identify papers that may have been overlooked by the more obvious key terms. In case of doubt, the contents of papers were scanned in order to decide whether they should be included or not.
References
Aksnes, D. W., Rorstad, K., Piro, F., & Sivertsen, G. (2011). Are female researchers less cited? A large-scale study of Norwegian scientists. Journal of the American Society for Information Science and Technology, 62(4), 628–636.
Beaver, D. B. (1986). Collaboration and teamwork in physics. Czechoslovak Journal of Physics, 1, 14–18.
Bordons, M., Morillo, F., Fernández, M. T., & Gómez, I. (2003). One step further in the production of bibliometric indicators at the micro level: Differences by gender and professional category of scientists. Scientometrics, 57(2), 159–173.
Bornmann, L. (2010). Towards an ideal method of measuring research performance: Some comments to the Opthof and Leydesdorff (2010) paper. Journal of Informetrics, 4, 441–443.
Borrego, Á., Barrios, M., Villarroya, A., & Ollé, C. (2010). Scientific output and impact of postdoctoral scientists: A gender perspective. Scientometrics, 83, 93–101.
Borsuk, R. M., Budden, A. E., Leimu, R., Aarssen, L. W., & Lortie, C. J. (2009). The influence of author gender, national language and number of authors on citation rate in ecology. The Open Ecology Journal, 2, 25–28.
Boshoff, N. (2005). The representation of women academics in higher education in South Africa: Progress in the pipeline? South African Journal of Higher Education, 19(2), 359–377.
Cole, J. R. (1979). Fair science: Women in the scientific community. New York: Free Press.
Cole, J. R., & Cole, S. (1973). Social stratification in science. Chicago: University of Chicago Press.
Cole, J. R., & Zuckerman, H. (1984). The productivity puzzle: Persistence and change in patterns of publication of men and women scientists. Advances in Motivation and Achievement, 2, 217–258.
Corley, E. A., & Sabharwal, M. (2010). Scholarly collaboration and productivity patterns in public administration: Analysing recent trends. Public Administration, 88(3), 627–648.
Craig, I. D., Plume, A. M., McVeigh, M. E., Pringle, J., & Amin, M. (2007). Do open access articles have greater citation impact? A critical review of the literature. Journal of Informetrics, 1(3), 239–248.
Creamer, E. G. (1998). Assessing faculty publication productivity: Issues of equity. ASHE-ERIC Higher Education Report (Vol. 26, No. 2). Washington, DC: Association for the Study of Higher Education.
Cronin, B., & Overfelt, K. (1994). Citation-based auditing of academic performance. Journal of the American Society for Information Science, 45(2), 61–72.
Davenport, E., & Snyder, H. (1995). Who cites women? Whom do women cite? An exploration of gender and scholarly citation in sociology. Journal of Documentation, 51(4), 404–410.
Davis, D. E., & Astin, H. S. (1987). Reputational standing in academe. The Journal of Higher Education, 58, 261–275.
Demetrulias, D. M. (1986). Gender differences in co-authorship. Journal of Educational Equity and Leadership, 6(2), 119–127.
Drake, J., Mooney, H. A., Di Castri, F., Groves, R., Kruger, F. J., Rejmánek, M., et al. (Eds.). (1989). Biological invasions: A global perspective. Chichester: Wiley.
Ferber, M. A. (1986). Citations: Are they an objective measure of scholarly merit? Signs: Journal of Women in Culture and Society, 11(2), 381–389.
Flores, L. Y., Rooney, S. C., Heppner, P. P., Browne, L. D., & Wei, M.-F. (1999). Trend analyses of major contributions in The Counseling Psychologist cited from 1986 to 1996: Impact and implications. The Counseling Psychologist, 27(1), 73–95.
Fox, M. F. (1991). Gender, environmental milieu, and productivity in science. In H. Zuckerman, J. R. Cole, & J. T. Bruer (Eds.), The outer circle: Women in the scientific community (pp. 188–204). New York: Norton.
García-Pérez, M. A. (2009). The Hirsch h index in a non-mainstream area: Methodology of the behavioral sciences in Spain. The Spanish Journal of Psychology, 12(2), 833–849.
Garfield, E. (1981). The 1,000 contemporary scientists most-cited 1965–1978 (Part I: The basic list and introduction). Current Contents, 41, 5–14. [Reprinted in: Garfield, E. (1981–1982). Essays of an information scientist, 5, 269–278].
Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research output and impact: A study of Mexican researchers. Research Policy, 36, 1035–1051.
Hammel, E. A., Mason, C., Prater, A., & Lundy, R. (1995). Gender and the academic career in North American Anthropology: Differentiating intramarket from extramarket bias. Current Anthropology, 36(2), 366–380.
Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185.
Helmreich, R. L., Spence, J. T., Beane, W. E., Lucker, G. W., & Matthews, K. A. (1980). Making it in academic psychology: Personality correlates of attainment. Journal of Personality and Social Psychology, 39(5), 896–908.
Housri, N., Cheung, M. C., Koniaris, L. G., & Zimmers, T. A. (2008). Scientific impact of women in academic surgery. Journal of Surgical Research, 148, 13–16.
Hunter, L. A., & Leahey, E. (2010). Parenting and research productivity: New evidence and methods. Social Studies of Science, 40(3), 433–451.
Hutson, S. R. (2002). Gendered citation practices in American Antiquity and other archaeology journals. American Antiquity, 67(2), 331–342.
Hutson, S. R. (2006). Self-citation in archaeology: Age, gender, prestige, and the self. Journal of Archaeological Method and Theory, 13(1), 1–19.
Irvine, J., & Martin, B. R. (1986). Women in radio astronomy—Shooting stars? In J. Harding (Ed.), Perspectives on gender and science (pp. 80–103). London: The Falmer Press.
Kelly, C. D., & Jennions, M. D. (2006). The h index and career assessment by numbers. Trends in Ecology & Evolution, 21(4), 167–170.
Kolpin, V. W., & Singell, L. D., Jr. (1996). The gender composition and scholarly performance of economics departments: A test for employment discrimination. Industrial and Labor Relations Review, 49(3), 408–423.
Krampen, G. (2008). The evaluation of university departments and their scientists: Some general considerations with reference to exemplary bibliometric publication and citation analyses for a Department of Psychology. Scientometrics, 76(1), 3–21.
Leahey, E. (2007). Not by productivity alone: How visibility and specialization contribute to academic earnings. American Sociological Review, 72(4), 533–561.
Leimu, R., & Koricheva, J. (2005). What determines the citation frequency of ecological papers? Trends in Ecology & Evolution, 20(1), 28–32.
Leta, J., & Lewison, G. (2003). The contribution of women in Brazilian science: A case study in astronomy, immunology and oceanography. Scientometrics, 57(3), 339–353.
Lewison, G. (2001). The quantity and quality of female researchers: A bibliometric study of Iceland. Scientometrics, 52(1), 29–43.
Lindsey, D. (1980). Production and citation measures in the sociology of science: The problem of multiple authorship. Social Studies of Science, 10(2), 145–162.
Long, J. S. (1992). Measures of sex differences in scientific productivity. Social Forces, 71(1), 159–178.
Lutz, C. (1990). The erasure of women’s writing in sociocultural anthropology. American Ethnologist, 17, 611–627.
MacRoberts, M. H., & MacRoberts, B. R. (1987). Problems of citation analysis: a critical review. Journal of the American Society for Information Science, 40, 342–349.
Mauleón, E., & Bordons, M. (2006). Productivity, impact and publication habits by gender in the area of materials science. Scientometrics, 66(1), 199–218.
Mauleón, E., Bordons, M., & Oppenheim, C. (2008). The effect of gender on research staff success in life sciences in the Spanish National Research Council. Research Evaluation, 17(3), 213–225.
Mcelhinny, B., Hols, M., Holtzkener, J., Unger, S., & Hicks, C. (2003). Gender, publication and citation in sociolinguistics and linguistic anthropology: The construction of a scholarly canon. Language in Society, 32, 299–328.
Montpetit, E., Blais, A., & Foucault, M. (2008). What does it take for a Canadian political scientist to be cited? Social Science Quarterly, 89(3), 802–816.
National Advisory Council on Innovation (NACI). (2008). Creating the future South African national system of innovation: Gender, race and SET sector issues. Report by the Science, Engineering and Technology for Women (SET4W), NACI Permanent Sub-Committee. http://www.nacinnovation.biz/about-naci/naci-sub-committees/set4women/set4w-downloads. Accessed 9 March 2011.
Over, R., & Moore, D. (1980). Research productivity and impact of men and women in psychology departments of Australian universities, 1975–1977. Australian Psychologist, 15(3), 413–418.
Peñas, C. S., & Willet, P. (2006). Gender differences in publication and citation counts in librarianship and information science research. Journal of Information Science, 32(5), 480–485.
Persell, C. H. (1983). Gender, rewards and research in education. Psychology of Women Quarterly, 8(1), 33–47.
Powell, A., Hassan, T. M., Dainty, A. R. J., & Carter, C. (2009). Exploring gender differences in construction research: A European perspective. Construction Management and Economics, 27, 803–807.
Prozesky, H. E. (2006). Gender differences in the journal publication productivity of South African academic authors. South African Review of Sociology, 37(2), 87–112.
Pyšek, P., Richardson, D. M., & Jarošik, V. (2006). Who cites who in the invasion zoo: Insights from an analysis of the most highly cited papers in invasion ecology. Preslia, 78, 437–468.
Pyšek, P., Richardson, D. M., Pergl, J., Jarošík, V., Sixtová, Z., & Weber, E. (2008). Geographical and taxonomical biases in invasion ecology. Trends in Ecology & Evolution, 23, 237–244.
Qiu, H., & Chen, Y.-F. (2009). Bibliometric analysis of biological invasions research during the period of 1991 to 2007. Scientometrics, 81(3), 601–610.
Reese-Evans, L. (2010). Gender and citation in two LIS e-journals: A bibliometric analysis of LIBRES and Information Research. Library and Information Science Research Electronic Journal, 20(1), 1–19.
Reskin, B. F. (1978). Scientific productivity, sex and location in the institution of science. American Journal of Sociology, 83(5), 1235–1243.
Richardson, D. M., & Van Wilgen, B. W. (2004). Invasive alien plants in South Africa: How well do we understand the ecological impacts? South African Journal of Science, 100(Jan/Feb), 45–52.
Rousseau, R. (1992). Letter to the editor: Why I am not cited or why are multi-authored papers more cited than others? Journal of Documentation, 48, 79–80.
Sandström, U. (2009). Combining curriculum vitae and bibliometric analysis: Mobility, gender and research performance. Research Evaluation, 18(2), 135–142.
Sonnert, G. (1995). What makes a good scientist? Determinants of peer evaluation among biologists. Social Studies of Science, 25(1), 35–55.
Sonnert, G., & Holton, G. J. (1995). Gender differences in science careers: The Project Access Study. New Brunswick, NJ: Rutgers University Press.
Sooryamoorthy, R. (2009). Collaboration and publication: How collaborative are scientists in South Africa? Scientometrics, 80(2), 419–439.
Stack, S. (1994). An analysis of the impacts of books and journal articles. International Review of Modern Sociology, 24(2), 119–125.
Symonds, M. R. E., Gemmell, N. J., Braisher, T. L., Gorringe, K. L., & Elgar, M. A. (2006). Gender differences in publication output: Towards an unbiased metric of research performance. PLoS ONE, 1(2), e127.
Teghtsoonian, M. (1974). Distribution of sex by authors and editors of psychological journals, 1970–1972: Are there enough women editors? American Psychologist, 29, 262–269.
Toutkoushian, R. K. (1994). Using citations to measure sex discrimination in faculty scales. The Review of Higher Education, 18(1), 61–82.
Tregenza, T. (2002). Gender bias in the refereeing process? Trends in Ecology & Evolution, 17(8), 349–350.
Trifunac, M. D. (2006). A note on publication and citation rates of female academics in earthquake engineering. Soil Dynamics and Earthquake Engineering, 26, 1063–1075.
Trimble, V. (1985). Some notes on patterns in citations of papers by American astronomers. Quarterly Journal of the Royal Astronomical Society, 26, 40–50.
Trimble, V. (1993). Patterns in citations of papers by American astronomers. Quarterly Journal of the Royal Astronomical Society, 34, 235–250.
United Nations Educational, Scientific and Cultural Organization (UNESCO). (2007). Science, technology and gender: An international report (executive summary). Science and technology for development series. Paris: UNESCO Publishing. http://unesdoc.unesco.org/images/0015/001540/154027e.pdf. Accessed 9 March 2011.
Van Raan, A. F. J. (1998). The influence of international collaboration on the impact of research results. Scientometrics, 42(3), 423–428.
Walters, G. D. (2006). Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact. Scientometrics, 69(3), 499–510.
Walters, C. G., Fry, E. H., & Chaisson, B. D. (1990). Women scholars: Closing the publication gap. Research in Higher Education, 31(4), 355–367.
Ward, K. B., Gast, J., & Grant, L. (1992). Visibility and dissemination of women’s and men’s sociological scholarship. Social Problems, 39(3), 291–298.
Ward, K. B., & Grant, L. (1991). Gender and publishing in sociology. Gender & Society, 5(2), 207–223.
Ward, K. B., & Grant, L. (1996). Gender and academic publishing. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. xi, pp. 172–212). New York: Agathon Press.
Webster, B. M. (2001). Polish women in science: A bibliometric analysis of Polish science and its publications, 1980–1999. Research Evaluation, 10(3), 185–194.
Wennerås, C., & Wold, A. (1997). Nepotism and sexism in peer-review. Nature, 387, 341–343.
Williamson, M. (1996). Biological invasions. London: Chapman and Hall.
Acknowledgments
The authors thank CREST for the use of SA Knowledgebase, and the former database manager, Derick van Niekerk, for running the keyword searches. Acknowledgements are due to the Director (Steven Chown) and Deputy Director (David Richardson) of the CIB for their assistance in identifying the sex of some of the authors in the dataset, and in identifying relevant keywords. David Richardson is also thanked for highlighting the need to control for the time available for accumulation of citations, and his and Dan Simberloff’s insightful comments on an earlier draft of this paper, as well as the comments of two anonymous reviewers, are much appreciated. Research assistance for this project was funded from the first author’s CIB core team member grant. Finally, we are highly indebted to Charline Mouton, Marion van Dorssen and Christopher Mechnig for their assistance in collecting data for this project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Prozesky, H., Boshoff, N. Bibliometrics as a tool for measuring gender-specific research performance: an example from South African invasion ecology. Scientometrics 90, 383–406 (2012). https://doi.org/10.1007/s11192-011-0478-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-011-0478-7