CoChat

Improve Reputation Evaluation of Crowdsourcing Participants Using Multidimensional Index and Machine Learning Techniques

Open access
Huang, Yanrong, Chen, Min · 2019 · IEEE Access

The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.

Abstract

Building a scientific and reasonable reputation evaluation mechanism for crowdsourcing participants is an effective way to solve the problem of transaction fraud, to establish the trust of traders and ensure the quality of task completion. Under the big data environment, machine learning methods have been applied in the domain of e-commerce of physical goods to improve the traditional reputation evaluation methods, and achieved good results. However, few studies have applied machine learning methods to crowdsourcing, a form of service e-commerce, to evaluate the reputation of participants. This paper proposes a reputation evaluation model (i.e. LDA-RF) for crowdsourcing participants of Random Forest based on Linear Discriminant Analysis. The model consists of five steps: firstly, building a multidimensional reputation evaluation index system for crowdsourcing participants, collecting real data sets, and preprocessing data; secondly, data dimensionality reduction methods, including Linear Discriminant Analysis, Principal Component Analysis, Mean Impact Value method and ReliefF feature selection method, are used to eliminate redundant variables; thirdly, data normalization; fourthly, with selected feature subset, five machine learning techniques, Random Forest, Decision Tree, Back propagation Neural Network, Radial Basis Function Neural Network and Support Vector Machine are used to train the model; Fifthly, the validity of the model is tested by four evaluation measures: 10 fold cross validation, confusion matrix, Kruskal-wallis test and dispersion degree. The results show that the LDA-RF model on accuracy, F1-measure, generalization ability and robustness are better than those of other models, and it has better performance and effectiveness. This study represents a new contribution to establish reputation evaluation of crowdsourcing participants under big data environment.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2019.2933147

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Figures

FIGURE 1 Flowchart of reputation evaluation of crowdsourcing participants.
FIGURE 1 Flowchart of reputation evaluation of crowdsourcing participants.CC-BY · From the PDF
TABLE 4. Performance measure results for reputation evaluation model of crowdsourcing participants.
TABLE 4. Performance measure results for reputation evaluation model of crowdsourcing participants.CC-BY · From the PDF
FIGURE 4 Accuracy of 10 fold cross validation of crowdsourcing participants' reputation evaluation models based on PCA and MIV data dimension reduction methods.
FIGURE 4 Accuracy of 10 fold cross validation of crowdsourcing participants' reputation evaluation models based on PCA and MIV data dimension reduction methods.CC-BY · From the PDF
FIGURE 5 Kruskal-wallis test results of accuracy for reputation evaluation models of crowdsourcing participants.
FIGURE 5 Kruskal-wallis test results of accuracy for reputation evaluation models of crowdsourcing participants.CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Image 6
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 25 references

Research works printed but not linked: 11

  1. broomhead, 1988, Multivariable functional interpolation and adaptive networks, Complex Syst, 2, 321
  2. liao, 2014, Research on C2C seller selection model based on the evaluation of implicit credit factors, Journal of Yunnan University of Finance and Economics, 30, 134
  3. wang, 2014, B2C enterprise credit evaluation based on improved TOPSIS method, Soft Science, 28, 21
  4. fu, 2016, Network supplier reputation evaluation model based on big data, Journal of Central University of Finance & Economics, 8, 74
  5. rong, 2018, The study of credit evaluation of suppliers on cross-border E-commerce platform based on big data, Statist Inf Forum, 33, 100
  6. yan, 2017, Reputation model of crowdsourcing workers based on activity degree, J Comput Appl, 37, 2039
  7. li, 2012, Multidimensional reputation evaluation model for C2C E-commerce, Chinese Journal of Management, 9, 204
  8. howe, 2006, The rise of crowdsourcing, Wired Mag, 14, 1
  9. ruan, 2015, Crowdsourcing quality control based on reputation model of Dempster–Shafer theory, J Comput Appl, 35, 2380
  10. wang, 2016, Online incentive mechanism based on reputation for mobile crowdsourcing system, J Comput Appl, 36, 2121
  11. rui, 2016, Reputation-based incentive mechanism in crowdsourcing, J Electron Inf Technol, 38, 1808

Cited by

See all 25

Metadata

Authors
Huang, Y., & Chen, M.
Affiliations
Wuhan University (CN)
Venue
IEEE Access
Published
2019
Volume
7
Pages
118055-118067
ISSN
2169-3536 (ISSN-L)
OpenAlex
W2964583479
Cited by
25
References
24
Citations by year
2025: 4 · 2024: 2 · 2023: 5 · 2022: 7 · 2021: 2 · 2020: 5
FWCI
3.80
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Natural Science Foundation of China (91118003, 61373039)
Topics
  • Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
  • Physical Sciences › Computer Science › Computer Networks and Communications › Access Control and Trust
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Evaluation and Optimization Models
Keywords
reputation measurement, crowdsourcing, transaction fraud, trust, linear discriminant analysis, random forest, feature selection, dimensionality reduction

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Improve Reputation Evaluation of Crowdsourcing Participants Using Multidimensional Index and Machine Learning Techniques. https://app.cochat.ai/papers/10.1109/access.2019.2933147
The paper
Huang, Y., & Chen, M. (2019). Improve Reputation Evaluation of Crowdsourcing Participants Using Multidimensional Index and Machine Learning Techniques. IEEE Access. https://doi.org/10.1109/access.2019.2933147

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.