The DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Clear Sanctions, Vague Rewards: How China's Social Credit System Currently Defines "Good" and "Bad" Behavior
Abstract
China's Social Credit System (SCS, 社会信用体系 or shehui xinyong tixi) is expected to become the first digitally-implemented nationwide scoring system with the purpose to rate the behavior of citizens, companies, and other entities. Thereby, in the SCS, "good" behavior can result in material rewards and reputational gain while "bad" behavior can lead to exclusion from material resources and reputational loss. Crucially, for the implementation of the SCS, society must be able to distinguish between behaviors that result in reward and those that lead to sanction. In this paper, we conduct the first transparency analysis of two central administrative information platforms of the SCS to understand how the SCS currently defines "good" and "bad" behavior. We analyze 194,829 behavioral records and 942 reports on citizens' behaviors published on the official Beijing SCS website and the national SCS platform "Credit China", respectively. By applying a mixed-method approach, we demonstrate that there is a considerable asymmetry between information provided by the so-called Redlist (information on "good" behavior) and the Blacklist (information on "bad" behavior). At the current stage of the SCS implementation, the majority of explanations on blacklisted behaviors includes a detailed description of the causal relation between inadequate behavior and its sanction. On the other hand, explanations on redlisted behavior, which comprise positive norms fostering value internalization and integration, are less transparent. Finally, this first SCS transparency analysis suggests that socio-technical systems applying a scoring mechanism might use different degrees of transparency to achieve particular behavioral engineering goals.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, having a low or high score has very real material consequences for profiled individuals, formalised in the system of respective rewards and punishments (Ohlberg et al. 2017; Engelmann et al. 2019).”
From Sovereignty, privacy, and ethics in blockchain-based identity management systems (2021) · cites this paper as Engelmann et al. 2019
In the citing paper
“Moreover, governments can use AI applications for automated propaganda and disinformation campaigns (Lazer etal. 2018), social control (Engelmann et al. 2019), surveillance (Helbing 2019), face recognition or sentiment analysis (Introna and Wood 2004), social sorting (Lyon 2003), or improved interrogation techniques (McAllister 2017).”
From The Ethics of AI Ethics: An Evaluation of Guidelines (2020) · cites this paper as Engelmann et al. 2019
In the citing paper
“In each case, attending to end-use provides a richer design space and new opportunities for improvements and safeguards.”
From Toward situated interventions for algorithmic equity: lessons from the field (2020) · cites this paper as [19]
Citations
Cites
Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions
Ryan, R. M., & Deci, E. L. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.1999.1020 · Cited by 18,666 · cited 1×
“This phenomenon, termed “crowding-out effect”, results in fewer people engaging in this behavior since the consequences of failing to act can simply be compensated by financial means [2, 9, 26].”
BackgroundIncentives and Prosocial Behavior
Bénabou, R., & Tirole, J. (2006) · American Economic Review · doi:10.1257/aer.96.5.1652 · Cited by 3,357
The Global Findex Database 2017: Measuring Financial Inclusion and the Fintech Revolution
Demirguc-Kunt, A., Klapper, L., Singer, D., Ansar, S., & Hess, J. (2018) · doi:10.1596/978-1-4648-1259-0 · Cited by 2,664
Exploring the Space of Topic Coherence Measures
Röder, M., Both, A., & Hinneburg, A. (2015) · Proceedings of the Eighth ACM International Conference on Web Search and Data Mining · doi:10.1145/2684822.2685324 · Cited by 2,224
Pay Enough or Don't Pay at All*
Gneezy, U., & Rustichini, A. (2000) · Quarterly Journal of Economics · doi:10.1162/003355300554917 · Cited by 2,175
Gneezy, U., & Rustichini, A. (2000) · The Journal of Legal Studies · doi:10.1086/468061 · Cited by 1,816
When and Why Incentives (Don't) Work to Modify Behavior
Gneezy, U., Meier, S., & Rey-Biel, P. (2011) · Journal of Economic Perspectives · doi:10.1257/jep.25.4.191 · Cited by 1,735
LDAvis: A method for visualizing and interpreting topics
Sievert, C., & Shirley, K. (2014) · Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces · doi:10.3115/v1/w14-3110 · Cited by 1,366
Wachter, S., Mittelstadt, B., & Floridi, L. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx005 · Cited by 1,308
Privacy and rationality in individual decision making
Acquisti, A., & Grossklags, J. (2005) · IEEE Security and Privacy Magazine · doi:10.1109/msp.2005.22 · Cited by 1,123
Meaningful information and the right to explanation
Selbst, A. D., & Powles, J. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx022 · Cited by 516
Termite: visualization techniques for assessing textual topic models
Chuang, J., Manning, C. D., & Heer, J. (2012) · Proceedings of the International Working Conference on Advanced Visual Interfaces · doi:10.1145/2254556.2254572 · Cited by 420
The ethics of information transparency
Turilli, M., & Floridi, L. (2009) · Ethics and Information Technology · doi:10.1007/s10676-009-9187-9 · Cited by 419
China’s social credit systems and public opinion: Explaining high levels of approvalPreprint
Kostka, G. (2019) · New Media & Society · doi:10.2139/ssrn.3215138 · Cited by 267
Privacy and Data Privacy Issues in Contemporary China
Yao-huai, L. (2005) · Ethics and Information Technology · doi:10.1007/s10676-005-0456-y · Cited by 176
(2008) · Cited by 171
Ethical pluralism and global information ethics
Ess, C. (2006) · Ethics and Information Technology · doi:10.1007/s10676-006-9113-3 · Cited by 133
Chen, Y., & Cheung, A. S. Y. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2992537 · Cited by 129
Dao, Harmony and Personhood: Towards a Confucian Ethics of Technology
Wong, P. (2011) · Philosophy & Technology · doi:10.1007/s13347-011-0021-z · Cited by 105
Morality: An Anthropological Perspective
Laidlaw, J. C. (2009) · Ethnos · doi:10.1080/00141840903311859 · Cited by 99
Hoofnagle, C. J. (2003) · University of North Carolina School of Law Scholarship Repository (University of North Carolina Hospitals) · Cited by 85
A comparative study of online privacy regulations in the U.S. and China
Wu, Y., Lau, T., Atkin, D. J., & Lin, C. A. (2011) · Telecommunications Policy · doi:10.1016/j.telpol.2011.05.002 · Cited by 69
Marczewski, A. (2017) · XRDS Crossroads The ACM Magazine for Students · doi:10.1145/3123756 · Cited by 69
The gamification of trust: the case of China’s “social credit”
Ramadan, Z. B. (2017) · Marketing Intelligence & Planning · doi:10.1108/mip-06-2017-0100 · Cited by 69
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5154 · Cited by 12
Evolving Legal Frameworks for Protecting the Right to Internet Privacy in China
Xu, J.-H. (2015) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780190201265.003.0010 · Cited by 9
Research works printed but not linked: 6
- Luciano Floridi . 2013. The Ethics of Information . Oxford University Press . Luciano Floridi. 2013. The Ethics of Information. Oxford University Press.
- Bernard Gert and Joshua Gert . 2017. The definition of morality . In The Stanford Encyclopedia of Philosophy, Edward N . Zalta (Ed.). Metaphysics Research Lab, Stanford University . Bernard Gert and Joshua Gert. 2017. The definition of morality. In The Stanford Encyclopedia of Philosophy, Edward N. Zalta (Ed.). Metaphysics Research Lab, Stanford University.
- Jianzhou Liu . 2011 . Building the Social Credit System: The content, the model, and the trajectory (Shehui Xinyong Tixi Jianshe: Neihan, Moshi yu Lujing Xuanze; in Chinese) . Journal of the Party School of the Central Committee of the C.P.C. 15 , 3 (2011), 50 -- 53 . Jianzhou Liu. 2011. Building the Social Credit System: The content, the model, and the trajectory (Shehui Xinyong Tixi Jianshe: Neihan, Moshi yu Lujing Xuanze; in Chinese). Journal of the Party School of the Central Committee of the C.P.C. 15, 3 (2011), 50--53.
- Mareike Ohlberg , Shazeda Ahmed , and Bertram Lang . 2018. Central Planning , Local Experiments : The Complex Implementation of China's Social Credit System . https:/www.merics.org/en/microsite/china-monitor/central-planning-local-experiments. Mareike Ohlberg, Shazeda Ahmed, and Bertram Lang. 2018. Central Planning, Local Experiments: The Complex Implementation of China's Social Credit System. https:/www.merics.org/en/microsite/china-monitor/central-planning-local-experiments.
- State Council . 2014. Notice of the State Council on Issuing the Outline of the Plan for Building a Social Credit System (2014-2020) ; (in Chinese) . http://www.gov.cn/zhengce/content/ 2014 -06/27/content_8913.htm. State Council. 2014. Notice of the State Council on Issuing the Outline of the Plan for Building a Social Credit System (2014-2020); (in Chinese). http://www.gov.cn/zhengce/content/2014-06/27/content_8913.htm.
- Zhiling Zhao and Feng Ding . 2007 . Shanghai, Zhejiang, Shenzhen Social Credit System models, problems and revelation (Shanghai, Zhejiang, Shenzhen Shehui Xinyong Tixi Jianshe Moshi, Wenti yu Qishi) (in Chinese) . Wei Shi 10 (2007), 70 -- 73 . Zhiling Zhao and Feng Ding. 2007. Shanghai, Zhejiang, Shenzhen Social Credit System models, problems and revelation (Shanghai, Zhejiang, Shenzhen Shehui Xinyong Tixi Jianshe Moshi, Wenti yu Qishi) (in Chinese). Wei Shi 10 (2007), 70--73.
Cited by
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
“2018), social control (Engelmann et al. 2019), surveillance (Helbing 2019), face recognition or sentiment analysis (Introna and Wood 2004), social sorting (Lyon 2003), or improved interrogation techniques (McAllister 2017).”
BackgroundThe Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
“2018), social control (Engelmann et al. 2019), surveillance (Helbing 2019), face recognition or sentiment analysis (Introna and Wood 2004), social sorting (Lyon 2003), or improved interrogation techniques (McAllister 2017).”
BackgroundThe Chinese approach to artificial intelligence: an analysis of policy, ethics, and regulation
Roberts, H., Cowls, J., Morley, J., Taddeo, M., Wang, V., & Floridi, L. (2021) · AI & SOCIETY · doi:10.1007/s00146-020-00992-2 · Cited by 573
“In terms of the former, national blacklists are comprised of those who have broken existing laws and regulations, with a clear rationale for inclusion provided (Engelmann et al. 2019).”
BackgroundSix Human-Centered Artificial Intelligence Grand Challenges
Ozmen Garibay, O., Winslow, B., Andolina, S., Antona, M., Bodenschatz, A., Coursaris, C., Falco, G., Fiore, S. M., Garibay, I., Grieman, K., et al. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2153320 · Cited by 501
The Forgotten Margins of AI Ethics
Birhane, A., Ruane, E., Laurent, T., Brown, M. S., Flowers, J., Ventresque, A., & Dancy, C. L. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533157 · Cited by 222
Metadata
- Authors
- Engelmann, S., Chen, M., Fischer, F., Kao, C.-Y., & Grossklags, J.
- Affiliations
- Technical University of Munich (DE)
- Fraunhofer Institute for Applied and Integrated Security (DE)
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 69-78
- OpenAlex
- W2910291565
- Cited by
- 98
- References
- 27
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 12 · 2023: 5 · 2022: 9 · 2021: 35 · 2020: 16 · 2019: 3
- FWCI
- 108.55
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Publikationsdatenbank der Fraunhofer-Gesellschaft (Fraunhofer-Gesellschaft) (submittedVersion)
- Fraunhofer-Publica (Fraunhofer-Gesellschaft) (submittedVersion)
- Topics
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Keywords
- social credit system, behavioral scoring, transparency, rewards and sanctions, red list, blacklists, behavioral engineering, socio-technical systems, reputation, value internalization, civic behavior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.