In their paper, Marvin Stecker, Paul Balluff, Fabienne Lind, Celina Dinhopl, Annie Waldherr, and Hajo G. Boomgaarden analyse 406 journal articles for mentions of software tools. They find that

1. Software mentions showed no clear trends over time or differences between journals/ disciplines. However, articles discussing validation of their methodology were more likely to mention the software they used.
2. Tools with greater visibility (e.g., a DOI, CRAN listing, repository) were cited more often. Surprisingly, frequent updates to the codebase didn’t increase mentions.
3. They tried to, but couldn't settle the debate over whether R or Python is more popular in the social sciences (even split).

This research highlights that crediting software increases the transparency of research results and also gives visibility and well-deserved credits to the developers and maintainers of software packages who make our work possible.

The Abstract:

The use of computational methods for text analysis has been rapidly gaining a foothold in computational social science. Yet, little is known about the reporting practices of the software tools utilised in this field. This research note investigates the factors influencing the likelihood of software tools being mentioned in social science journal articles. To this end, we reviewed 406 journal articles, representing all computational text analysis articles published in top social science journals between 2016 and 2020, and identified all software tools reported. We explore both article-level and tool-level characteristics and investigate their association with the mentioning of software tools. Our findings indicate a consistent pattern of tool reporting across time and disciplines, while authors who detail methodological validation within their papers are more inclined to mention the software tools employed. Furthermore, we observe that software tools with greater accessibility tend to receive more frequent mentions, while the maintenance status of the tools does not significantly impact their frequency of being mentioned. We discuss the implications of these results for the further development of computational text analysis in social science and especially communication research.
Find the full study here: https://www.aup-online.com/content/journals/10.5117/CCR2024.1.6.STEC
Cite Article: Stecker, M., Balluff, P., Lind, F., Dinhopl, C., Waldherr, A., & Boomgaarden, H. G. (2024). Tools of the Trade–When Are Software Tools Mentioned in Computational Text Analysis Research?. Computational Communication Research, 6(1), 1. https://www.aup-online.com/content/journals/10.5117/CCR2024.1.6.STEC