Report
 
Prepared 
ETX/DN/SV J HalÚn,../SU R Karlsson,../SP M Nilsson
No.  
ETX/DN/SU-98:024
Approved 
ETX/DN/SU Bjarne Dńcker
Checked Date 
1998-11-02
Rev 
A
File

Performance Measurements of Threads in Java and Processes in Erlang

1 Introduction

Telecommunications applications require massive support for concurrency and asynchronous message passing in the underlying software technology. Java supports concurrency using threads and Erlang by using processes. Both languages support message passing.

This paper reports from a benchmark of these features in the two languages. The benchmark focuses on concurrency issues and does not include any computation.

2 The Benchmark

The following program has been written in Java and in Erlang:

The Java and Erlang source programs are appended to this report. It is quite possible that they could be improved.

3 Results - Pentium

3.1 Hardware, Software and OS

The following versions of the Java and Erlang systems were used

Erlang R4b (4.7.3)
Java JDK 1.1.6

Three Pentium computers were used

Pentium Pro 200 MHz 64 MByte Win95 (4.00.950a) / FreeBSD 2.2.7
Pentium MMX 233 MHz 64 MByte Win95 (4.00.950b)
Pentium II MMX 266 MHz128 MByte WinNT 4.00, SP3

The 200 MHz computer could not spawn more than 200 Java threads in Windows 95 so the 233 MHz computer was used instead. This might be due to the use of an older version of the OS.

3.2 Spawn Time

Time required to spawn (create) a new thread or process in micro seconds

Number of
processes
or threads
FreeBSD Windows 95 Windows NT
Erlang Java Erlang Java Erlang Java
Pro MMX
100      1100   ---- 1400
200      850   ---- 1150
500       1500 20 900
1000 16 270    2500 10 1240
2000 16        10 ----
5000 16        14 ----
10000 16        14 ----
20000 16        14 ----

Comments

3.3 Message Passing Time

Time required to send and receive a message in micro seconds

Number of
processes
or threads
FreeBSD Windows 95 Windows NT
Erlang Java Erlang Java Erlang Java
Pro MMX
1 3.96 9.40  2.50   3.01 2.03
2 7.65 28.8  57.1   3.90 24.73
5 6.89 28.4  58.6   4.05 27.54
10 6.75 28.4  58.4   4.28 27.94
20 6.74 28.8  59.8   4.38 29.34
50 6.90 30.3  62.6 54.4 4.98 29.94
100 6.93 34.3  65.2 56.8 5.07 33.05
200 7.09 34.7  68.0 59.3 5.10 35.95
500 8.10 43.8  - 60.6 5.16 37.55
600   53.8  -     38.35
700   67.3  -     38.66
800   86.2  -     38.85
900   120  -     43.46
950   41.5 or
134
 -     41.56
1000 9.07 36.5  - 63.3 5.97 39.26
1500   65  - 80.4 6.56 -
1600      - 93.2 6.56 -
1700      - - 6.71 -
2000 9.46 54 or
223
  - - 6.76 -
5000 9.45 -  - - 7.36 -
10000 9.52 -  - - 7.99 -
20000 9.57 -  - - 9.16 -

Comments

3.4 Thread/process Size

 Process or
thread size
Erlang1 K
Java - FreeBSD8 K
Java - Win95a 
Java - Win95b 

4 Results - SPARC

4.1 Hardware, Software and OS

The following versions of the Java and Erlang systems were used

Erlang R4b (4.7.3)
Java JDK 1.1

The following computer and OS was used

UltraSPARC 1 167 MHz 64 MByteSolaris SunOS 5.5.1

4.2 Spawn Time

The spawn time for Erlang was always 30 us but the threads spawn time on Solaris was alsways given as 0 us. Probably spawning of a thread on Solaris does not really spawn the thread, but only queues requests for thread spawning. In this case, the timings for the message passing are too high.

4.3 Message Passing Time

Time required to send and receive a message in micro seconds

Number of
processes
or threads
Erlang
Solaris
Java
Solaris
1 5.83 7.86
2 8.45 48.0
5 8.82 49.1
10 8.81 49.4
20 9.14 50.4
50 9.60 54.8
100 9.76 56.6
200 9.91 56.4
500 10.2 57.2
1.000 11.3 58.4
2.000 11.4 60-70
3.000   400-1.200
(paging)
5.000 12.0 -
10.000 12.4 -
20.000 12.5 -
30.000 12.3 -

Comment

4.4 Thread/process Size

 Process or
thread size
Erlang1 K
Java16 K

5 Evaluation

This is a small benchmark but the results can be of interest nevertheless. Erlang processes are more lightweight than Java threads (a few us compared with 1-2 ms on FreeBSD and Windows 95). Erlang allows spawning of a large number of processes, tenth of thousands. The number of threads that can be spawned in Java is of an order of magnitude lower.

Message passing between sleeping processes in Erlang is very fast. On the computers used it was in the vicinity of 10 us per message which includes sending the message and waking up the receiving process. It is also OS independent. Java is about 5 to 10 times slower.

6 Benchmark Programs

The benchmark programs are included in the hope that they could be executed on other computers and OS's. The benchmark programs could perhaps also be written more efficiently.

6.1 Program in Java

class Zog extends Thread {
  private int id;
  private Zog next;
  private boolean flag;
  private int message;

  Zog(int n) {
    id = n;
    flag = false;
  }

  public void link(Zog zog) {
    next = zog;
  }

  public void run() {
    try {
      do this.relay(); while (message > 0);
    } catch (InterruptedException e) {}
  }
  
  private synchronized void relay() throws InterruptedException {
    while (flag == false)
      wait();
    flag = false;
    next.send(message - 1);
  }

  public synchronized void send(int n) throws InterruptedException {
    message = n;
    flag = true;
    notify();
  }

  public static void main(String args[]) {
    int n = Integer.parseInt(args[0]);
    int m = Integer.parseInt(args[1]);
    Zog old;
    First first = new First(n, m);

    first.start();

    old = first;
    while (--n > 0) {
      Zog curr = new Zog(n);

      curr.link(old);
      curr.start();
      old = curr;
    }

    first.link(old);

    try {
      first.send(m);
    } catch (InterruptedException e) {}
  }
}

class First extends Zog {
  private int procs, msgs;
  private long t1, t2, t3;

  First(int n, int m) {
    super(n);
    procs = n;
    msgs = m;
  }

  public void run() {
    long run_time, init_time;
    double msg_time, spawn_time;

    t1 = System.currentTimeMillis();
    super.run();
    t3 = System.currentTimeMillis();

    init_time = t2 - t1;
    spawn_time = (1000.0 * init_time) / procs;
    System.out.println("init_time = " + init_time + " ms (" +
		       spawn_time + " us/proc) (" + procs + " procs)");

    run_time = t3 - t2;
    msg_time = (1000.0 * run_time) / msgs;
    System.out.println("run_time = " + run_time + " ms (" +
		       msg_time + " us/msg) (" + msgs + " msgs)");
  }

  public void link(Zog zog) {
    super.link(zog);
    t2 = System.currentTimeMillis();
  }
}

6.2 Program in Erlang

-module(zog).

%% This is a test program that first creates N processes (that are
%% "connected" in a ring) and then sends M messages in that ring.
%%
%% - September 1998
%% - roland


-export([start/0, start/1, start/2]).

-export([run/2, process/1]).			% Local exports - ouch

start() -> start(16000).

start(N) -> start(N, 1000000).

start(N, M) -> spawn(?MODULE, run, [N, M]).


run(N, M) when N < 1 ->
    io:format("Must be at least 1 process~n", []),
    0.0;
run(N, M) ->
    statistics(wall_clock),

    Pid = setup(N-1, self()),

    {_,T1} = statistics(wall_clock),
    io:format("Setup : ~w s", [T1/1000]),
    case N of
	1 -> io:format(" (0 spawns)~n", []);
	_ -> io:format(" (~w us per spawn) (~w spawns)~n",
		       [1000*T1/(N-1), N-1])
    end,
    statistics(wall_clock),

    Pid ! M,
    K = process(Pid),

    {_,T2} = statistics(wall_clock),
    Time = 1000*T2/(M+K),
    io:format("Run   : ~w s (~w us per msg) (~w msgs)~n",
	      [T2/1000, Time, (M+K)]),

    Time.

setup(0, OldPid) ->
    OldPid;
setup(N, OldPid) ->
    NewPid = spawn(?MODULE, process, [OldPid]),
    setup(N-1, NewPid).


process(Pid) ->
    receive
	M ->
	    Pid ! M-1,
	    if
		M < 0  -> -M;
		true   -> process(Pid)
	    end
    end.