Skip to main content
Journal of Internet Services and Applications
Download PDF

Web graph similarity for anomaly detection

Journal of Internet Services and Applications volume 1, pages 19–30 (2010)Cite this article

Abstract

Web graphs are approximate snapshots of the web, created by search engines. They are essential to monitor the evolution of the web and to compute global properties like PageRank values of web pages. Their continuous monitoring requires a notion of graph similarity to help measure the amount and significance of changes in the evolving web. As a result, these measurements provide means to validate how well search engines acquire content from the web. In this paper, we propose five similarity schemes: three of them we adapted from existing graph similarity measures, and two we adapted from well-known document and vector similarity methods (namely, the shingling method and random projection based method). We empirically evaluate and compare all five schemes using a sequence of web graphs from Yahoo!, and study if the schemes can identify anomalies that may occur due to hardware or other problems.

References

  1. Becchetti L, Castillo C (2006) The distribution of PageRank follows a power-law only for particular values of the damping factor. In: WWW. ACM, New York, pp 941–942

    Chapter  Google Scholar 

  2. Blondel V, Gajardo A, Heymans M, Senellart P, Dooren PV (2004) A measure of similarity between graph vertices: applications to synonym extraction and web searching. SIAM Rev 46(4):647–666

    MATH  MathSciNet  Article  Google Scholar 

  3. Boncz P, Kersten M (1999) MIL primitives for querying a fragmented world. VLDB J 8(2):101–119

    Article  Google Scholar 

  4. Borodin A, Roberts GO, Rosenthal JS, Tsaparas P (2005) Link analysis ranking: algorithms, theory, and experiments. ACM Trans Internet Technol 5(1):231–297

    Article  Google Scholar 

  5. Broder A, Glassman S, Manasse M, Zweig G (1997) Syntactic clustering of the web. In: WWW, pp 393–404

  6. Bunke H, Dickinson PJ, Kraetzl M, Wallis WD (2007) A graph-theoretic approach to enterprise network dynamics. Birkhäuser, Boston

    MATH  Google Scholar 

  7. Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A, Gruber RE (2006) Bigtable: a distributed storage system for structured data. In: OSDI. ACM, New York, pp 205–218

    Google Scholar 

  8. Charikar M (2002) Similarity estimation techniques from rounding algorithms. In: STOC. ACM, New York, pp 380–388

    Google Scholar 

  9. D’Alberto P, Dasdan A (2009) Non-parametric information-theoretic measures of one-dimensional distribution functions from continuous time series. In: SDM, pp 685–696

  10. Dasdan A, Papadimitriou P (2007) Methods and apparatus for computing graph similarity via sequence similarity. US Patent Application, No 20090164411

  11. Dasdan A, Papadimitriou P (2007) Methods and apparatus for computing graph similarity via signature similarity, US Patent Application, No 20090150381

  12. Eiron N, McCurley KS, Tomlin JA (2004) Ranking the web frontier. In: WWW. ACM, New York, pp 309–318

    Google Scholar 

  13. Fagin R, Kumar R, Sivakumar D (2003) Comparing top klists. SIAM J Discrete Math 17(1):134–160

    MATH  MathSciNet  Article  Google Scholar 

  14. Fan J, Yao Q (2005) Nonlinear time series: nonparametric and parametric methods. Springer series in statistics, 2nd edn. Springer, New York

    Google Scholar 

  15. Gibson D, Kumar R, Tomkins A (2005) Discovering large dense subgraphs in massive graphs. In: VLDB. ACM, New York, pp 721–732

    Google Scholar 

  16. Henzinger MR. Finding near-duplicate web pages: a large-scale evaluation of algorithms. In: SIGIR

  17. Jeh G, Widom J (2002) SimRank: a measure of structural-context similarity. In: KDD, Canada

  18. Melnik S, Garcia-Molina H, Rahm E (2002) Similarity flooding: a versatile graph matching algorithm and its applications to schema matching. In: ICDE

  19. Papadimitriou P, Dasdan A, Garcia-Molina H (2008) Web graph similarity for anomaly detection. Tech Rep 2008-1, Stanford Univ

  20. Papadopoulos A, Manolopoulos Y (1999) Structure-based similarity search with graph histograms. In: DEXA/IWOSS. IEEE, New York, pp 174–178

    Google Scholar 

  21. Robles-Kelley A, Hancock ER (2005) Graph edit distance from spectral seriation. IEEE Trans Pattern Anal Mach Intell 27(3):365–378

    Article  Google Scholar 

  22. Seidl T. References for graph similarity. URL: http://www.dbs.informatik.uni-muenchen.de/~seidl/graphs/

  23. Xue GR, Yang Q, Zeng HJ, Yu Y, Chen Z (2005) Exploiting the hierarchical structure for link analysis. In: SIGIR, New York, NY, USA, pp 186–193

  24. Zager L, Verghese G (2005) Graph similarity. URL: http://lees.mit.edu/lees/presentations/LauraZager.pdf

  25. Zhu P, Wilson RC (2005) A study of graph spectra for comparing graphs. In: MBVC

Download references

Author information

Authors and Affiliations

  1. Stanford University, Stanford, CA, 94305, USA

    Panagiotis Papadimitriou & Hector Garcia-Molina

  2. Yahoo! Inc., Sunnyvale, CA, 94089, USA

    Ali Dasdan

Authors
  1. Panagiotis Papadimitriou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Dasdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hector Garcia-Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panagiotis Papadimitriou.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Papadimitriou, P., Dasdan, A. & Garcia-Molina, H. Web graph similarity for anomaly detection. J Internet Serv Appl 1, 19–30 (2010). https://doi.org/10.1007/s13174-010-0003-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13174-010-0003-x

Keywords

  • Anomaly detection
  • Graph similarity
  • Locality sensitive hashing