Open Access

Web graph similarity for anomaly detection

  • Panagiotis Papadimitriou1Email author,
  • Ali Dasdan2 and
  • Hector Garcia-Molina1
Journal of Internet Services and Applications20101:3

https://doi.org/10.1007/s13174-010-0003-x

Received: 14 December 2009

Accepted: 27 January 2010

Published: 25 February 2010

Abstract

Web graphs are approximate snapshots of the web, created by search engines. They are essential to monitor the evolution of the web and to compute global properties like PageRank values of web pages. Their continuous monitoring requires a notion of graph similarity to help measure the amount and significance of changes in the evolving web. As a result, these measurements provide means to validate how well search engines acquire content from the web. In this paper, we propose five similarity schemes: three of them we adapted from existing graph similarity measures, and two we adapted from well-known document and vector similarity methods (namely, the shingling method and random projection based method). We empirically evaluate and compare all five schemes using a sequence of web graphs from Yahoo!, and study if the schemes can identify anomalies that may occur due to hardware or other problems.

Keywords

Anomaly detectionGraph similarityLocality sensitive hashing