ProjectREST: Real-time, scalable, self-managing storage for Telecoms

REST: Real-time, scalable, self-managing storage for Telecoms

Subscriber databases are among the most high performance, highly available computer systems in the world, managing tens of millions of users and handling thousands of transactions per second, whilst being almost always available. High performance, reliable, real-time subscriber databases have long been a key building block for Telecom services, while in recent years, they have been adopted by Internet companies, often building in-house using open-source software, to deliver widely used services, including Amazon, Facebook, Wikipedia, Youtube and Twitter. The goal of REST project is to design and build a new architecture for a subscriber database based on a structured overlay network (SON) that will run over multiple data centres on common off-the-shelf (COTS) hardware. This research plan involves developing:

  • smart data partitioning algorithms for the SON to enable increased storage size and system throughput,
  • data-aware and network-aware consensus algorithms for transactions to improve availability in the face of network partitions,
  • a message passing component model to improve scalability for multi-core hardware,
  • self-managing features to reduce system administration costs.

In REST project, we develop a new SON architecture for subscriber systems that will support a higher number of transactions/second and store a larger amount of data storage per user than existing systems, while maintaining real-time guarantees on data read and write latencies.  REST is required to deliver a system that can adapt not just to node failures but also network partitions. REST will "intelligently" adapt its replication algorithms and their consistency models to network partitions. Similarly, intelligence will be required to build replication algorithms that learn about the frequency with which subscriber attributes are updated. REST will be "faster" than existing systems by virtue of the SON handling a higher number of subscriber operations in parallel, as well our multicore support, which will mean fewer machines are required to manage the same number of subscribers. Fewer machines will result in power savings that will increase proportionally to the number of cores in a system.  

Publications
Number of items: 6.

Arad, Cosmin (2013) Programming Model and Protocols for Reconfigurable Distributed Systems. Doctoral thesis, KTH Royal Institute of Technology.

Arad, Cosmin and Shafaat, Tallat M. and Haridi, Seif (2012) CATS: linearizability and partition tolerance in scalable and self-organizing key-value stores. [SICS Report]

Payberah, Amir H. (2011) Distributed Optimization of P2P Media Delivery Overlays. Licentiate thesis, KTH - Royal Institute of Technology.

Payberah, Amir H. and Dowling, Jim and Haridi, Seif (2011) GLive: The Gradient overlay as a market maker for mesh-based P2P live streaming. In: The 10th IEEE International Symposium on Parallel and Distributed Computing (ISPDC'11), July 2011, Cluj-Napoca, Romania.

Payberah, Amir H. and Dowling, Jim and Haridi, Seif (2011) Gozar: NAT-friendly Peer Sampling with One-Hop Distributed NAT Traversal. In: The 11th IFIP international conference on Distributed Applications and Interoperable Systems (DAIS'11), June 2011, Reykjavik, Iceland.

Terelius, Hakan and Shi, Guodong and Dowling, Jim and Payberah, Amir H. and Gattami, Ather and Johansson, Karl Henrik (2011) Converging an Overlay Network to a Gradient Topology. In: The 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC'11), December 2011, Orlando, Florida, USA.

This list was generated on Wed Dec 13 12:29:18 2017 CET.