Skip to main content
Journal of Internet Services and Applications
Download PDF
  • Special Issue: Middleware
  • Open access
  • Published:

Sizing multi-tier systems with temporal dependence: benchmarks and analytic models

Journal of Internet Services and Applications volume 1, pages 117–134 (2010)Cite this article

Abstract

Temporal dependence, as a synonym for burstiness, is often found in workloads (i.e., arrival flows and/or service times) in enterprise systems that use the multi-tier paradigm. Despite the fact that burstiness has deleterious effects on performance, existing modeling and benchmarking techniques do not provide an effective capacity planning for multi-tier systems with temporal dependence. In this paper, we first present strong evidence that existing models cannot capture bursty conditions and accurately predict performance. Therefore, we propose a simple and effective sizing methodology to integrate workload burstiness into models and benchmarking tools used in system sizing. This modeling methodology is based on the index of dispersion which jointly captures variability and burstiness of the service process in a single number. We report experimentation on a real testbed that validates the accuracy of our modeling technique by showing that experimental and model prediction results are in excellent agreement under both bursty and non-bursty workloads. To further support the capacity planning process under burstiness, we propose an enhanced benchmarking technique that can emulate workload burstiness in systems. We find that most existing benchmarks, like the standard TPC-W benchmark, are designed to assess system performance only under non-bursty conditions. In this work, we rectify this deficiency by introducing a new module into existing benchmarks, which allows to inject burstiness into the arrival stream in a controllable and reproducible manner by using the index of dispersion as a single turnable knob. This approach enables a better understanding of system performance degradation due to burstiness and makes a strong case for the usefulness of the proposed benchmark enhancement for capacity planning of enterprise systems.

References

  1. Almeida V, Bestavros A, Crovella M, de Oliveira A (1996) Characterizing reference locality in the WWW. In: IEEE conference on parallel and distributed information systems, Dec 1996

  2. Almeida V, Arlitt M, Rolia J (2002) Analyzing a web-based system’s performance measures at multiple timescales. ACM Perform Eval Rev 30(2):3–9

    Article  Google Scholar 

  3. Andersen AT, Nielsen BF (1998) A Markovian approach for modeling packet traffic with long-range dependence. IEEE J Sel Areas Commun 16(5):719–732

    Article  Google Scholar 

  4. Andersen AT, Nielsen BF (2002) On the use of second-order descriptors to predict queueing behavior of MAPs. Nav Res Logist 49(4):391–409

    Article  MATH  MathSciNet  Google Scholar 

  5. Arlitt M, Jin T (1999) Workload characterization of the 1998 World Cup website. Technical Report HPL-1999-35R1

  6. Arlitt M, Williamson C (1996) Web server workload characterization: the search for invariants. In: Proc of ACM SIGMETRICS, pp 126–137

  7. Arlitt M, Friedrich R, Jin T (1999) Workload characterization of a web proxy in a cable environment. ACM Perform Eval Rev 27(2):25–36

    Article  Google Scholar 

  8. Balbo G, Serazzi G (1996) Asymptotic analysis of multiclass closed queueing networks: common bottlenecks. Perform Eval 26(1):51–72

    Article  MATH  Google Scholar 

  9. Banga G, Druschel P (1999) Measuring the capacity of a web server under realistic loads. WWW 2(1–2):69–83

    Article  Google Scholar 

  10. Barford P, Crovella M (1998) Generating representative web workloads for network and server performance evaluation. ACM Perform Eval Rev 26(1):151–160

    Article  Google Scholar 

  11. Casale G, Zhang E, Smirni E (2007) Characterization of moments and autocorrelation in MAPs. ACM Perform Eval Rev 35(1):27–29. Special issue on MAMA workshop

    Article  Google Scholar 

  12. Casale G, Mi N, Smirni E (2008) Bound analysis of closed queueing networks with workload burstiness. In: Proc of SIGMETRICS, pp 13–24

  13. Casale G, Mi N, Cherkasova L, Smirni E (2010) Dealing with burstiness in multi-tier applications: new models and their parameterization (under submission)

  14. Casale G, Mi N, Smirni E (2010) Model-driven system capacity planning under workload burstiness. IEEE Trans Comput 59(1):66–80

    Article  MathSciNet  Google Scholar 

  15. Cox DR, Lewis PAW (1966) The statistical analysis of series of events. Methuen, London

    Book  MATH  Google Scholar 

  16. Crovella M, Bestravos A (1996) Self-similarity in Word Wide Web traffic: evidence and possible causes. In: Proc of SIGMETRICS

  17. Fonseca R, Almeida V, Crovella M, Abrahao B (2003) On the intrinsic locality properties of web reference streams. Proc IEEE INFOCOM

  18. Garcia D, Garcia J (2003) TPC-W e-commerce benchmark evaluation. IEEE Comput 36:42–48

    Article  Google Scholar 

  19. Gusella R (1991) Characterizing the variability of arrival processes with indexes of dispersion. IEEE J Sel Areas Commun 19(2):203–211

    Article  Google Scholar 

  20. Jung J, Krishnamurthy B, Rabinovich M (2002) Flash crowds and denial of service attacks: characterization and implications for CDNs and websites. In: Proc of WWW, pp 293–304

  21. Kant K, Tewary V, Iyer R (2001) An internet traffic generator for server architecture evaluation. In: Proc of workshop computer architecture evaluation using commercial workloads

  22. Kobayashi H, Mark BL (2009) System modeling and analysis: foundations of system performance evaluation

  23. Krishnamurthy D, Rolia J (1998) Predicting the QoS of an electronic commerce server: those mean percentiles. ACM Sigmetrics Perform Eva Rev 26(3):16–22

    Article  Google Scholar 

  24. Li H (2010) Realistic workload modeling and its performance impacts in large-scale eScience grids. IEEE Trans Parallel Distrib Syst 21(4):1045–9219

    Article  Google Scholar 

  25. Liu Z, Niclausse N, Jalpa-Villanueva C (2001) Traffic model and performance evaluation of web servers. Perform Eval 46(2–3)

    Google Scholar 

  26. Liu Z, Wynter L, Xia CH, Zhang F (2006) Parameter inference of queueing models for it systems using end-to-end measurements. Perform Eval 63(1):36–60

    Article  Google Scholar 

  27. Menascé DA, Almeida VAF, Dowdy WL (1994) Capacity planning and performance modeling: from mainframes to client–server systems

  28. Menascé DA, Almeida VAF, Reidi R, Pelegrinelli. R. Fonesca F, Meira W Jr. (2000) In search of invariants in e-business workloads. In: Proc of ACM conf electronic commerce, pp 56–65

  29. Mi N, Zhang Q, Riska A, Smirni E, Riedel E (2007) Performance impacts of autocorrelated flows in multi-tiered systems. Perform Eval 64(9–12):1082–1101

    Google Scholar 

  30. Mi N, Casale G, Cherkasova L, Smirni E (2008) Burstiness in multi-tier applications: symptoms, causes, and new models. In: Proc of Middleware

  31. Mi N, Casale G, Cherkasova L, Smirni E (2009) Injecting realistic burstiness into a traditional client–server benchmark. In: Proc of ICAC

  32. Minh TN, Wolters L (2009) Modeling job arrival process with long range dependence and burstiness characteristics. In: Proc of int’l symp on cluster computing and the grid, pp 324–330

  33. Mosberger D, Jin T (1998) httperf: a tool for measuring web server performance. In: Proc of workshop internet server performance

  34. Neuts MF (1989) Structured stochastic matrices of M/G/1 type and their applications. Dekker, New York

    MATH  Google Scholar 

  35. Paxon V, Floyd S (1995) Wide area traffic: the failure of poisson modeling. IEEE/ACM Trans Netw 3(3):226–244

    Article  Google Scholar 

  36. Ranjan S, Rolia J, Fu H, Knightly E (2002) QoS-driven server migration for internet data center. In: Proc of IWQoS, pp 3–12

  37. Ranjan S, Rolia J, Fu H, Knightly F (2002) Qos-driven server migration for Internet data centers. In: Proc of IWQoS

  38. Reiser M, Lavenberg S (1980) Mean-value analysis of closed multichain queueing networks. J ACM 27(2):312–322

    Article  MathSciNet  Google Scholar 

  39. Riedi RH, Crouse MS, Ribeiro VJ, Baraniuk RG (1999) A multifractal wavelet model with application to network traffic. IEEE Trans Inf Theory 45(4):992–1018

    Article  MATH  MathSciNet  Google Scholar 

  40. Rolia J, Vetland V (1998) Correlating resource demand information with arm data for application services. In: Proc of WOSP, pp 219–230

  41. Slashdot effect, Wikipedia, Oct 13, 2008. http://en.wikipedia.org/wiki/Slashdot_effect

  42. Sriram K, Whitt W (1986) Characterizing superposition arrival processes in packet multiplexers for voice and data. IEEE J Sel Areas Commun 4(6):833–846

    Article  Google Scholar 

  43. Urgaonkar B, Pacifici G, Shenoy P, Spreitzer M, Tantawi A (2005) An analytical model for multi-tier internet services and its applications. In: Proc of ACM SIGMETRICS, pp 291–302

  44. Urgaonkar B, Shenoy P, Chandra A, Goyal P (2005) Dynamic provisioning of multi-tier internet applications. In: Proc of ICAC

  45. Villela D, Pradhan P, Rubenstein D (2002) Provisioning servers in the application tier for e-commerce systems. ACM Trans Internet Technol 7(1):7

    Article  Google Scholar 

  46. Williams A, Arlitt M, Williamson C, Barker K (2005) Web workload characterization: ten years later. Springer, New York

    Google Scholar 

  47. www.mercury.com/us/products/diagnostics. HP (Mercury) diagnostics

  48. Zhang Q, Cherkasova L, Mathews G, Greene W, Smirni E (2007) R-capriccio: a capacity planning and anomaly detection tool for enterprise services with live workloads. In: Proc of Middleware, pp 244–265

  49. Zhang Q, Cherkasova L, Smirni E (2007) A regression-based analytic model for dynamic resource provisioning of multi-tier applications. In: Proc of ICAC

Download references

Author information

Authors and Affiliations

  1. Northeastern University, Boston, MA, USA

    Ningfang Mi

  2. Imperial College London, London, UK

    Giuliano Casale

  3. HP Labs, Palo Alto, CA, USA

    Ludmila Cherkasova

  4. College of William and Mary, Williamsburg, VA, USA

    Evgenia Smirni

Authors
  1. Ningfang Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuliano Casale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ludmila Cherkasova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Evgenia Smirni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ningfang Mi.

Additional information

This work was partially supported by NSF grants CNS-0720699 and CCF-0811417, a gift from HP Labs, and the Imperial College JRF fellowship.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Mi, N., Casale, G., Cherkasova, L. et al. Sizing multi-tier systems with temporal dependence: benchmarks and analytic models. J Internet Serv Appl 1, 117–134 (2010). https://doi.org/10.1007/s13174-010-0012-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13174-010-0012-9

Keywords