The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Variability in architectural simulations of multi-threaded workloads
Abstract
Multi-threaded commercial workloads implement many important Internet services. Consequently, these workloads are increasingly used to evaluate the performance of uniprocessor and multiprocessor system designs. This paper identifies performance variability as a potentially major challenge for architectural simulation studies using these workloads. Variability refers to the differences between multiple estimates of a workload's performance. Time variability occurs when a workload exhibits different characteristics during different phases of a single run. Space variability occurs when small variations in timing cause runs starting from the same initial condition to follow widely different execution paths. Variability is a well-known phenomenon in real systems, but is nearly universally ignored in simulation experiments. In a central result of this paper we show that variability in multi-threaded commercial workloads can lead to incorrect architectural conclusions (e.g., 31% of the time in one experiment). We propose a methodology, based on multiple simulations and standard statistical techniques, to compensate for variability. Our methodology greatly reduces the probability of reaching incorrect conclusions, while enabling simulations to finish within reasonable time limits.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The SPLASH-2 programs: characterization and methodological considerations
Woo, S. C., Ohara, M., Torrie, E., Singh, J. P., & Gupta, A. (1995) · Proceedings of the 22nd annual international symposium on Computer architecture - ISCA '95 · doi:10.1145/223982.223990 · Cited by 3,624
Automatically characterizing large scale program behavior
Sherwood, T., Perelman, E., Hamerly, G., & Calder, B. (2002) · Proceedings of the 10th international conference on Architectural support for programming languages and operating systems · doi:10.1145/605397.605403 · Cited by 1,674
The SPLASH-2 programs: characterization and methodological considerations
Woo, S. C., Ohara, M., Torrie, E., Singh, J. P., & Gupta, A. (2002) · Proceedings 22nd Annual International Symposium on Computer Architecture · doi:10.1109/isca.1995.524546 · Cited by 1,601
Barroso, L. A., Kourosh Gharachorloo, Y., McNamara, R., Nowatzyk, A. G., Qadeer, S., Sano, B., Smith, S. F., Stets, R. J., & Verghese, B. (2000) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/339647.339696 · Cited by 517
Basic Block Distribution Analysis to Find Periodic Behavior and Simulation Points in Applications
Sherwood, T., Perelman, E., & Calder, B. (2001) · Cited by 480
Memory system characterization of commercial workloads
Barroso, L. A., Kourosh Gharachorloo, Y., & Bugnion, E. (1998) · doi:10.1145/279358.279363 · Cited by 377
Basic block distribution analysis to find periodic behavior and simulation points in applications
Sherwood, T., Perelman, E., & Calder, B. (2002) · Proceedings 2001 International Conference on Parallel Architectures and Compilation Techniques · doi:10.1109/pact.2001.953283 · Cited by 313
Gibson, J., Kunz, R., Ofelt, D., et al. (2000) · ACM SIGPLAN Notices · doi:10.1145/356989.356994 · Cited by 98
Evaluating Non-deterministic Multi-threaded Commercial Workloads
Alameldeen, A. R., Mauer, C. J., Xu, M. G., Harper, P. J., Martin, M. M. K., Sorin, D. J., Hill, M. D., & Wood, D. A. (2001) · Cited by 79
Self-citationMartin, M. M. K., Sorin, D. J., Hill, M. D., et al. (2004) · Proceedings - International Symposium on High-Performance Computer Architecture/Proceedings · doi:10.1109/hpca.2002.995715 · Cited by 71
Cited by
The PARSEC benchmark suite: characterization and architectural implications
Bienia, C., Kumar, S., Singh, J. P., & Li, K. (2008) · Proceedings of the 17th international conference on Parallel architectures and compilation techniques · doi:10.1145/1454115.1454128 · Cited by 3,444
Sniper: exploring the level of abstraction for scalable and accurate parallel multi-core simulation
Carlson, T. E., Heirman, W., & Eeckhout, L. (2011) · Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis · doi:10.1145/2063384.2063454 · Cited by 840
Cooperative Caching for Chip Multiprocessors
Chang, J., & Sohi, G. S. (2006) · ACM SIGARCH Computer Architecture News · doi:10.1145/1150019.1136509 · Cited by 355
Producing wrong data without doing anything obviously wrong!
Mytkowicz, T., Diwan, A., Hauswirth, M., & Sweeney, P. F. (2009) · Proceedings of the 14th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1508244.1508275 · Cited by 303
Metadata
- Authors
- Alarneldeen, A. R., & Wood, D. A.
- Venue
- The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings.
- Publisher
- IEEE Comput. Soc
- Published
- 2003-08-27
- Pages
- 7-18
- OpenAlex
- W2141386638
- Cited by
- 260
- References
- 11
- FWCI
- 13.40
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Variability in architectural simulations of multi-threaded workloads. https://app.cochat.ai/papers/10.1109/hpca.2003.1183520
- The paper
- Alarneldeen, A. R., & Wood, D. A. (2003). Variability in architectural simulations of multi-threaded workloads. The Ninth International Symposium on High-Performance Computer Architecture, 2003. HPCA-9 2003. Proceedings. https://doi.org/10.1109/hpca.2003.1183520
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.