The webgraph framework I
Paolo Boldi, Sebastiano Vigna · International World Wide Web Conference (WWW) · 2004
Studying web graphs is often difficult due to their large size. Recently,several proposals have been published about various techniques that allow tostore a web graph in memory in a limited space, exploiting the inner redundancies of the web. The WebGraph framework is a suite of codes, algorithms and tools that aims at making it easy to manipulate large web graphs. This papers presents the compression techniques used in WebGraph, which are centred around referentiation and intervalisation (which in turn are dual to each other). WebGraph can compress the WebBase graph (118 Mnodes, 1 Glinks)in as little as 3.08 bits per link, and its transposed version in as littleas 2.89 bits per link.
Cited by 1,201 works. doi:10.1145/988672.988752
Authors
References
- The PageRank Citation Ranking : Bringing Order to the Web
- Authoritative sources in a hyperlinked environment
- The Connectivity Server: fast access to linkage information on the Web
- WebBase: a repository of Web pages
- UbiCrawler: a scalable fully distributed Web crawler
- Towards compressing Web graphs
- Representing Web graphs
- The Link Database: fast access to graphs of the Web
- Compressing the graph structure of the Web
- Compact Representations of Separable Graphs