Typology of Carrots, Sticks, and Sermons: Tracing Its Influence on the Discipline of Public Policy

Background: The 1998 book Carrots, Sticks & Sermons: Policy Instruments and Their Evaluationintroduced a seminal typology, categorizing policy tools by their mechanisms of influence. Despite its potential to shape policy theory and practice, its broader disciplinary impact and citation patterns remain underexplored. This article evaluates the book’s legacy in public policy literature while testing bibliometric methods, including emerging generative artificial intelligence (GAI) tools for research synthesis.

Purpose: The study aims to (1) assess the influence of the book on the discourse in the public policy literature and (2) explore the utility and limits of AI-driven tools for tracing scholarly impact. It addresses four areas: the book’s citation trends, its presence in major disciplinary texts, citation intent, and GAI’s ability to reference the work accurately.Setting:Not applicable.

Intervention: Not applicable.

Research Design:A mixed-methods approach consisted of: (1) quantitative bibliometrics (citation counts, trends, and keyword analysis); (2) qualitative content analysis by human researchers (major disciplinary handbooks/textbooks and articles); and (3) generative AI semantic search and analysis.

Data Collection and Analysis: The quantitative analysis covered a corpus of interdisciplinary public policy literature from 1998 to 2024, extracted from citation databases (Scopus, Web of Science, Google Scholar). Qualitative analysis focused on 20 major handbooks and textbooks from public policy discipline (occurrence of references to book and policy instruments typology) and coding of 20 highly cited articles (classification of citation types in terms of intent). Work with GAI focused on semantic search and analysis with various AI platforms.

Findings: The findings underscore the typology’s enduring relevance while advocating for methodological rigor in impact assessments and GAI-aided research. The book has achieved “classic” status in public policy, evidenced by 1,369 citations on Google Scholar, 924 citations on Scopus, and 627 citations on Web of Science, as well as presence in 15 out of 20 major disciplinary handbooks/textbooks. Its typology is foundational but often superficially referenced; only 7 of 20 top articles applied it directly. GAI tools also referred to the book, although they frequently misattributed or diluted the source, highlighting risks in automated literature reviews. We propose updating the typology of policy instruments to include behavioral tools (e.g., nudges) and stress the need for human verification of GAI outputs.