The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
SCARFF : A scalable framework for streaming credit card fraud detection with spark
Open accessAbstract
The expansion of the electronic commerce, together with an increasing confidence of customers in electronic payments, makes of fraud detection a critical factor. Detecting frauds in (nearly) real time setting demands the design and the implementation of scalable learning techniques able to ingest and analyse massive amounts of streaming data. Recent advances in analytics and the availability of open source solutions for Big Data storage and processing open new perspectives to the fraud detection field. In this paper we present a SCAlable Real-time Fraud Finder (SCARFF) which integrates Big Data tools (Kafka, Spark and Cassandra) with a machine learning approach which deals with imbalance, nonstationarity and feedback latency. Experimental results on a massive dataset of real credit card transactions show that this framework is scalable, efficient and accurate over a big stream of transactions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
(2010) · Cited by 4,411
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing
Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J. H., McCauley, M., Franklin, M. J. M., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 3,622
Cassandra: a decentralized structured storage system
Lakshman, A., & Malik, P. (2010) · ACM SIGOPS Operating Systems Review · doi:10.1145/1773912.1773922 · Cited by 2,664
Learning from imbalanced data: open challenges and future directions
Krawczyk, B. (2016) · Progress in Artificial Intelligence · doi:10.1007/s13748-016-0094-0 · Cited by 2,459
Apache Hadoop YARN: yet another resource negotiator
Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., et al. (2013) · Proceedings of the 4th annual Symposium on Cloud Computing · doi:10.1145/2523616.2523633 · Cited by 1,848
Data mining for credit card fraud: a comparative study
Bhattacharyya, S., Jha, S., Tharakunnel, K., & Westland, J. C. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.08.008 · Cited by 1,108 · cited 1×
“In particular, the classification relies on Random Forests [42, 43], which have been shown to be particularly effective in fraud detection problems [44, 45, 46].”
MethodEnsemble learning for data stream analysis: A survey
Krawczyk, B., Minku, L. L., Gama, J., Stefanowski, J., & Woźniak, M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.004 · Cited by 1,050
MLlib: Machine Learning in Apache Spark
Meng, X., Bradley, J. K., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D., Freeman, J., Tsai, D., Amde, M., Owen, S., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.06807 · Cited by 940
A survey of multiple classifier systems as hybrid systems
Woźniak, M., Graña, M., & Corchado, E. (2014) · Information Fusion · doi:10.1016/j.inffus.2013.04.006 · Cited by 912
2014 International Joint Conference on Neural Networks (IJCNN)
(2014) · doi:10.1109/ijcnn32435.2014 · Cited by 685
Credit Card Fraud Detection: A Realistic Modeling and a Novel Learning Strategy
Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2736643 · Cited by 682
Self-citationCalibrating Probability with Undersampling for Unbalanced Classification
Pozzolo, A. D., Caelen, O., Johnson, R. A., & Bontempi, G. (2015) · 2015 IEEE Symposium Series on Computational Intelligence · doi:10.1109/ssci.2015.33 · Cited by 667 · cited 1×
“Most of these works address some specific issues of fraud detection, notably class imbalance [6, 7, 8] (the percentage of fraudulent transactions is usually very small), concept drift [9, 10, 11, 12, 13, 14] (the distribution of fraudulent transactions might change in time) and stream processing [15, 16].”
BackgroundSelf-citationA Survey on Ensemble Learning for Data Stream Classification
Gomes, H. M., Barddal, J. P., Enembreck, F., & Bifet, A. (2018) · ACM Computing Surveys · doi:10.1145/3054925 · Cited by 579
MillWheel: fault-tolerant stream processing at internet scale
Akidau, T., Balikov, A., Bekiroğlu, K., Chernyak, S., Haberman, J., Lax, R., McVeety, S., Mills, D., Nordstrom, P., & Whittle, S. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536222.2536229 · Cited by 536
Learned lessons in credit card fraud detection from a practitioner perspective
Dal Pozzolo, A., Caelen, O., Le Borgne, Y.-A., Waterschoot, S., & Bontempi, G. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.02.026 · Cited by 530
Self-citationAn Overview of the Scala Programming Language
Odersky, M., Altherr, P., Cremet, V., Emir, B., Maneth, S., Micheloud, S., Mihaylov, N., Schinz, M., Stenman, E., & Zenger, M. (2004) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 518
A survey on data preprocessing for data stream mining: Current status and future directions
Ramírez-Gallego, S., Krawczyk, B., García, S., Woźniak, M., & Herrera, F. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.078 · Cited by 502
Decision forest: Twenty years of research
Rokach, L. (2016) · Information Fusion · doi:10.1016/j.inffus.2015.06.005 · Cited by 472
Feature engineering strategies for credit card fraud detection
Correa Bahnsen, A., Aouada, D., Stojanovic, A., & Ottersten, B. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.12.030 · Cited by 451
Credit card fraud detection with a neural-network
Ghosh, & Reilly (1994) · Proceedings of the Twenty-Seventh Hawaii International Conference on System Sciences HICSS-94 · doi:10.1109/hicss.1994.323314 · Cited by 433
Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters
Zaharia, M., Das, T., Li, H., Shenker, S. J., & Stoica, I. G. (2012) · Cited by 425
A cost-sensitive decision tree approach for fraud detection
Sahin, Y., Bulkan, S., & Duman, E. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.021 · Cited by 381
Van Vlasselaer, V., Bravo, C., Caelen, O., Eliassi-Rad, T., Akoglu, L., Snoeck, M., & Baesens, B. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.04.013 · Cited by 368
Self-citationTransaction aggregation as a strategy for credit card fraud detection
Whitrow, C., Hand, D. J., Juszczak, P., Weston, D., & Adams, N. M. (2009) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-008-0116-z · Cited by 337
On the use of MapReduce for imbalanced big data using Random Forest
del Río, S., López, V., Benítez, J. M., & Herrera, F. (2014) · Information Sciences · doi:10.1016/j.ins.2014.03.043 · Cited by 328
Gama, J., & Brazdil, P. (2000) · Machine Learning · doi:10.1023/a:1007652114878 · Cited by 281
Association rules applied to credit card fraud detection
Sánchez, D., Vila, M. A., Cerda, L., & Serrano, J. M. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.02.001 · Cited by 248
AI^2: Training a Big Data Machine to Defend
Veeramachaneni, K., Arnaldo, I., Korrapati, V., Bassias, C., & Li, K. (2016) · 2016 IEEE 2nd International Conference on Big Data Security on Cloud (BigDataSecurity), IEEE International Conference on High Performance and Smart Computing (HPSC), and IEEE International Conference on Intelligent Data and Security (IDS) · doi:10.1109/bigdatasecurity-hpsc-ids.2016.79 · Cited by 245
Example-dependent cost-sensitive decision trees
Correa Bahnsen, A., Aouada, D., & Ottersten, B. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.04.042 · Cited by 223
A case study of applying boosting naive bayes to claim fraud diagnosis
Viaene, S., Derrig, R. A., & Dedene, G. (2004) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2004.1277822 · Cited by 149
Just-In-Time Classifiers for Recurrent Concepts
Alippi, C., Boracchi, G., & Roveri, M. (2013) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2239309 · Cited by 144
Credit card fraud detection and concept-drift adaptation with delayed supervised information
Pozzolo, A. D., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2015) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2015.7280527 · Cited by 135
Self-citationWhen is Undersampling Effective in Unbalanced Classification Tasks?
Pozzolo, A. D., Caelen, O., & Bontempi, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-23528-8_13 · Cited by 135
Self-citationTriguero, I., del Río, S., López, V., Bacardit, J., Benítez, J. M., & Herrera, F. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.05.027 · Cited by 129
Novelty detection in data streams
Faria, E. R., Gonçalves, I. J. C. R., de Carvalho, A. C. P. L. F., & da Gama, J. M. P. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-015-9444-8 · Cited by 111
Fast Perceptron Decision Tree Learning from Evolving Data Streams
Bifet, A., Holmes, G., Pfahringer, B., & Frank, E. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13672-6_30 · Cited by 95
2014 IEEE International Congress on Big Data
(2014) · doi:10.1109/bigdatacongress33106.2014 · Cited by 93
Scalable Distributed Stream Join Processing
Lin, Q., Ooi, B. C., Wang, Z., & Cui, Y. J. (2015) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2723372.2746485 · Cited by 92
One-class classifiers with incremental learning and forgetting for data streams with concept drift
Krawczyk, B., & Woźniak, M. (2014) · Soft Computing · doi:10.1007/s00500-014-1492-5 · Cited by 91
Big data based fraud risk management at Alibaba
Chen, J.-D., Tao, Y., Wang, H., & Chen, T. (2015) · The Journal of Finance and Data Science · doi:10.1016/j.jfds.2015.03.001 · Cited by 79
AnyNovel: detection of novel concepts in evolving data streams
Abdallah, Z. S., Gaber, M. M., Srinivasan, B., & Krishnaswamy, S. P. (2016) · Evolving Systems · doi:10.1007/s12530-016-9147-7 · Cited by 57
Countering the concept-drift problems in big data by an incrementally optimized stream mining model
Yang, H., & Fong, S. J. (2014) · Journal of Systems and Software · doi:10.1016/j.jss.2014.07.010 · Cited by 30
Research works printed but not linked: 3
- Krawczyk, 2015, Incremental weighted one-class classifier for mining stationary data streams, J. Comput, Sci., 9, 19, 10.1016/j.jocs.2015.04.024 · doi:10.1016/j.jocs.2015.04.024
- Phulari, 2016, Pattern analysis and fraud detection using hadoop framework, Int. J. Eng. Sci.Innov. Technol. (IJESIT), 5, 92
- Fan, 2011, Detection of rare items with target, Stat. Interface, 4, 11, 10.4310/SII.2011.v4.n1.a2 · doi:10.4310/sii.2011.v4.n1.a2
Cited by
Machine Learning Operations (MLOps): Overview, Definition, and Architecture
Kreuzberger, D., Kühl, N., & Hirschl, S. (2023) · IEEE Access · doi:10.1109/access.2023.3262138 · Cited by 766
“into the Spark-based feature engineering pipeline [54].”
Combining unsupervised and supervised learning in credit card fraud detection
Carcillo, F., Le Borgne, Y.-A., Caelen, O., Kessaci, Y., Oblé, F., & Bontempi, G. (2021) · Information Sciences · doi:10.1016/j.ins.2019.05.042 · Cited by 478
Self-citationTaha, A. A., & Malebary, S. J. (2020) · IEEE Access · doi:10.1109/access.2020.2971354 · Cited by 443
The state of the art and taxonomy of big data analytics: view from new big data framework
Mohamed, A., Najafabadi, M. K., Wah, Y. B., Zaman, E. A. K., & Maskat, R. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09685-9 · Cited by 265
Metadata
- Authors
- Carcillo, F., Dal Pozzolo, A., Le Borgne, Y.-A., Caelen, O., Mazzer, Y., & Bontempi, G.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2018-05
- Volume
- 41
- Pages
- 182-194
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W2751596245
- Cited by
- 267
- References
- 48
- FWCI
- 9.84
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Funders
- INNOVIRIS
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.