GES3

In recent years researchers at SICS, KTH, and around the world have studied a new class of distributed algorithms, based on distributed hash tables (DHTs). They allow the creation of structured overlay networks or peer-to-peer networks for organizing large-scale distributed applications. Unlike traditional server-centric solutions DHT-infrastructures are decentralized, scalable, self-organizing, and inherently robust in the face of failures, attacks and high peak loads. The results of this research, though for the most part based on large-scale simulations of distributed directory services (discovery), are extremely promising. So promising, that researchers are beginning to speculate that the next-generation large-scale distributed applications will be based on DHTs. Based on previous work on DHTs by SICS and KTH in other projects we now propose to take the next step, to build, experiment and validate real systems. The project will work on three different levels.

• First, we will develop a decentralized DHT-based infrastructure providing such basic services as discovery, name-based communication, publish/subscribe mechanisms, and group membership and communication services.
• Second, on top of the basic services we will build a grid-enabled (i.e. OGSA-compliant) application service platform, with services like storage management, searching and indexing, group services, and data-distribution. Finally, we will build a number of application prototypes to test and validate the infrastructure.

These potential applications are media distribution, network-based defence, and remote operation and maintenance of service gateway systems. The project consists of a consortium of two partners, SICS, KTH to provide critical mass and complementary expertise required to successfully achieve the goals. Outside of the project cooperation is intended with third partners focusing on the application domains.

1. design and implement a practical decentralized DHT-based infrastructure offering a number of basic services. This is a software component with a well-defined API intended for the system developers.
2. design and implement a package of application services on top of the basic infrastructure. These are services that are intended to be used directly by the application developer; ease of use and simplicity of the API is important
3. develop a few pilot applications that make use of the application services. These applications will, like the infrastructure that they are based upon, have the advantage of decentralization and self-organization.
4. 4. validate the applications and hence, indirectly, validate the application services and basic infrastructure, as regards robustness, load-balancing, performance, self-organization (i.e. adaptation as nodes join and leave the infrastructure).

The team at SICS/KTH is co-operating with Truve & Vasell Innovation AB and Generic Systems on the application side.

DKS middleware: A prototype has been developed in Java. The prototype is based on previous algorithmic work in other projects (PEPITO) and has been further refined (winter 2004-2005). The middleware supports scalable Internet-scale multicast, broadcast, name-based routing, and provides a simple Distributed Hash Table abstraction for data storage/retrieval. The middleware is currently being tested in one application together with partners (spring 2005). Further testing focussing on robustness and scalability is planned.

• PEPITO: A European project where much of the basic algorithmic work was done. www.sics.se/pepito
• DKS: A site devoted to the DKS middleware dks.sics.se
• P2P overview: A site collecting useful and interesting results on DHTs, and peer-to-peer computing in general www.sics.se/~sameh/p2pComputing.php
• DSLab: The distributed systems laboratory at SICS www.sics.se/dsl
• KTH: www.imit.kth.se

SICS:

• Per Brand
• Konstantin Popov
• Erik Klintskog
• Johanna Mannung

KTH

• Seif Haridi
• Ali Ghodsi
• Vladimir Vlasov