Middleware

Background

Programming platforms for distributed applications or middleware may be seen as an integration of a programming language with a collection of distributed algorithms. Careful integration allows the programmer to design and develop distributed applications in a coherent programming environment. At the very least data structures created by normal programming constructs can be passed between machines directly. The programmer does not want to have to convert the data structure into a bit-stream representation to put into a message that at the other end needs to be parsed. Furthermore, it is usually advantageous to merge the programming language concepts with suitable algorithms so that the program concepts can be used across machines. An example is a distributed object that can be invoked on different machines in the same way that object is invoked when local.

Middleware for Language Distribution with Peer-to-Peer Properties

Exactly how programming platforms or middleware that cater to large-scale decentralized systems should look like is a new and totally open research question. Put another way, how can existing programming platforms be extended to handle peer-to-peer. Given that the necessary algorithms exist how are they naturally mapped into the programming language and system. One can have the goal to make the system powerful and hide from the programmer the complexity of middleware algorithms, but the programmer must be able to tune his application for his specific requirements and expected use patterns.

Middleware Design: Modularity

We have already stated that middleware may be seen as an integration of a programming language with a collection of distributed algorithms. Generally the integration between the two may be tight or loose. Tight integration makes for good efficiency and ease of use. Language constructs are extended to operate in a distributed scope and makes the system easy to use for programmers already familiar with centralized programming systems based on the same language. Loose integration makes the algorithms or distribution services based on them available to the programmer as libraries. The chief disadvantage of this is the multiplicity of models, one for programming local computation and one for coordinating between sites. This greatly complicates the task of developing distributed applications and typically introduces errors in the interface between the two. In addition this approach is not as efficient as the tight integration approach.

In the integrated approach different systems based on different languages will to large extent package and code the same algorithms. Of course, middleware systems differ in the distribution services they supply, the algorithms they use, but to the extent that they do offer the same service the same algorithms are encoded many times, once for each system. This does not make for good modularity and can be seen to violate accepted principles of software engineering.

One could see middleware as being composed of two main components, one encodes the language-independent distributed algorithms and the other the language-specific virtual machine (be it Java, C#, Oz or Erlang). The first component is potentially common to all middleware, though it should be seen as open-ended as new algorithms and services are added. However, in practice sophisticated middleware of today is not built along these lines as all, rather language-specific and language-independent aspects are mixed in the code. It would be an advance in middleware if the two-component architecture could be realized in a software-engineering sense where the same distribution service component can be coupled with different programming languages. Note that we can accept only a very small or negligible runtime cost for our two-component architecture as compared to the mixed approach. We strive to realize this architecture and have a preliminary design that we intend to verify by constructing two middleware systems based on different programming languages using the same distribution service component.

Middleware Design: Programming Constructs

Middleware contains a number of distribution services. These services are based on distributed algorithms appropriately packaged. Some services the programmer might never need to be aware of, they are handled by the middleware automatically. For instance, the middleware might support mobility of agents or objects. From the viewpoint of the programmer of one component if he has a reference to an agent he can just send message and if he has a reference to the object he just invokes it. And it works, irrespective of where or how the agent/object moved.

This is not always the case; there are some services that the programmer needs to be aware of in order to use them or tune them. These services in some form will be new programming constructs that have to co-exist with traditional program constructs (objects, procedures, classes, etc). How can all these concepts be presented as a coherent whole to the programmer? Are the interactions between different constructs clear, and if not, can we make them clear? Do two services interact so intricately making it so difficult to predict that we need to provide a new service that combines the two in such a way as to offer well-modeled clear-cut choices? Care needs to be taken, that a system that puts together a collection of features, however useful taken individually, taken as a whole form a useful system.

Middleware Extending To Small Devices

It is generally acknowledged that in the future we will see many applications involving both devices and ordinary machines. It will greatly ease the task of programming these applications if middleware can be developed that extends over both ordinary machines and devices. Currently only rudimentary middleware exists offering little more than a messaging abstraction. The two fundamental problems in adapting middleware for devices are 1) limited memory of devices and 2) the high probability of intermittent connectivity. The limited memory of devices shifts the emphasis from optimizing for speed, the traditional criteria for middleware for wired networks, to optimizing for compactness. Traditional distributed computing research hardly considers intermittent connectivity at all, focused as it has been on homogenous LAN-based systems. WAN-based systems do need to consider it to some degree, particular when it comes to peer-to-peer. One of the major challenges in designing middleware both for peer-to-peer computing over fixed networks as well as middleware for devices will be in dealing with intermittent connectivity (or transient failure)