Report

Performance Measurements of Threads in Java and Processes in Erlang

1 Introduction

Telecommunications applications require massive support for concurrency and asynchronous message passing in the underlying software technology. Java supports concurrency using threads and Erlang by using processes. Both languages support message passing.

This paper reports from a benchmark of these features in the two languages. The benchmark focuses on concurrency issues and does not include any computation.

2 The Benchmark

The following program has been written in Java and in Erlang:

• N threads/processes are spawned connected as a ring structure. Through this ring a message (an integer comprising 4 bytes) is circulated involving M message passings. Actual (real) time used is measured.

The Java and Erlang source programs are appended to this report. It is quite possible that they could be improved.

3 Results - Pentium

3.1 Hardware, Software and OS

The following versions of the Java and Erlang systems were used

Erlang	R4b (4.7.3)
Java	JDK 1.1.6

Three Pentium computers were used

Pentium Pro	200 MHz	64 MByte	Win95 (4.00.950a) / FreeBSD 2.2.7
Pentium MMX	233 MHz	64 MByte	Win95 (4.00.950b)
Pentium II MMX	266 MHz	128 MByte	WinNT 4.00, SP3

The 200 MHz computer could not spawn more than 200 Java threads in Windows 95 so the 233 MHz computer was used instead. This might be due to the use of an older version of the OS.

3.2 Spawn Time

Time required to spawn (create) a new thread or process in micro seconds

Number of processes or threads	FreeBSD		Windows 95			Windows NT
	Erlang	Java	Erlang	Java		Erlang	Java
				Pro	MMX
100				1100		----	1400
200				850		----	1150
500					1500	20	900
1000	16	270			2500	10	1240
2000	16					10	----
5000	16					14	----
10000	16					14	----
20000	16					14	----

Comments

• The spawn time for Erlang processes is always 16 us regardless of the number of processes spawned and the OS. The spawn time for Java threads is dependent of OS and also of the the number of threads.
• Erlang can spawn up to 20.000 processes on FreeBSD or Windows 95. Java can spawn 2.000 threads on FreeBSD and on Windows 95 200 or 1.600 threads depending on different versions of the OS.

3.3 Message Passing Time

Time required to send and receive a message in micro seconds

Number of processes or threads	FreeBSD		Windows 95			Windows NT
	Erlang	Java	Erlang	Java		Erlang	Java
				Pro	MMX
1	3.96	9.40		2.50		3.01	2.03
2	7.65	28.8		57.1		3.90	24.73
5	6.89	28.4		58.6		4.05	27.54
10	6.75	28.4		58.4		4.28	27.94
20	6.74	28.8		59.8		4.38	29.34
50	6.90	30.3		62.6	54.4	4.98	29.94
100	6.93	34.3		65.2	56.8	5.07	33.05
200	7.09	34.7		68.0	59.3	5.10	35.95
500	8.10	43.8		-	60.6	5.16	37.55
600		53.8		-			38.35
700		67.3		-			38.66
800		86.2		-			38.85
900		120		-			43.46
950		41.5 or 134		-			41.56
1000	9.07	36.5		-	63.3	5.97	39.26
1500		65		-	80.4	6.56	-
1600				-	93.2	6.56	-
1700				-	-	6.71	-
2000	9.46	54 or 223		-	-	6.76	-
5000	9.45	-		-	-	7.36	-
10000	9.52	-		-	-	7.99	-
20000	9.57	-		-	-	9.16	-

Comments

• The message passing time in Erlang is very stable around 7 to 10 us from 2 to 20.000 processes. When using only a few processes (threads) the message passing time in Java is about 4 times slower on FreeBSD and 8 times slower on Windows 95.
• Neither the FreeBSD nor the Windows 95 Java versions behaves well when the number of processes are increased. The FreeBSD Java implementation had a curious behaviour between 600 and 1.000 threads, where the time went up dramatically.

3.4 Thread/process Size

	Process or thread size
Erlang	1 K
Java - FreeBSD	8 K
Java - Win95a
Java - Win95b

4 Results - SPARC

4.1 Hardware, Software and OS

The following versions of the Java and Erlang systems were used

Erlang	R4b (4.7.3)
Java	JDK 1.1

The following computer and OS was used

UltraSPARC

1 167 MHz

64 MByte

Solaris SunOS 5.5.1

4.2 Spawn Time

The spawn time for Erlang was always 30 us but the threads spawn time on Solaris was alsways given as 0 us. Probably spawning of a thread on Solaris does not really spawn the thread, but only queues requests for thread spawning. In this case, the timings for the message passing are too high.

4.3 Message Passing Time

Time required to send and receive a message in micro seconds

Number of processes or threads	Erlang Solaris	Java Solaris
1	5.83	7.86
2	8.45	48.0
5	8.82	49.1
10	8.81	49.4
20	9.14	50.4
50	9.60	54.8
100	9.76	56.6
200	9.91	56.4
500	10.2	57.2
1.000	11.3	58.4
2.000	11.4	60-70
3.000		400-1.200 (paging)
5.000	12.0	-
10.000	12.4	-
20.000	12.5	-
30.000	12.3	-

Comment

• Both Java and Erlang behave quite well on SPARC Solaris. Erlang could spawn up to 30.000 processes and Java up to 3000 threads. Java is 6 times slower than Erlang. The speed for Java starts to decrease when approaching the maximum number of threads.

4.4 Thread/process Size

	Process or thread size
Erlang	1 K
Java	16 K

5 Evaluation

This is a small benchmark but the results can be of interest nevertheless. Erlang processes are more lightweight than Java threads (a few us compared with 1-2 ms on FreeBSD and Windows 95). Erlang allows spawning of a large number of processes, tenth of thousands. The number of threads that can be spawned in Java is of an order of magnitude lower.

Message passing between sleeping processes in Erlang is very fast. On the computers used it was in the vicinity of 10 us per message which includes sending the message and waking up the receiving process. It is also OS independent. Java is about 5 to 10 times slower.

6 Benchmark Programs

The benchmark programs are included in the hope that they could be executed on other computers and OS's. The benchmark programs could perhaps also be written more efficiently.

6.1 Program in Java

class Zog extends Thread {
  private int id;
  private Zog next;
  private boolean flag;
  private int message;

  Zog(int n) {
    id = n;
    flag = false;
  }

  public void link(Zog zog) {
    next = zog;
  }

  public void run() {
    try {
      do this.relay(); while (message > 0);
    } catch (InterruptedException e) {}
  }
  
  private synchronized void relay() throws InterruptedException {
    while (flag == false)
      wait();
    flag = false;
    next.send(message - 1);
  }

  public synchronized void send(int n) throws InterruptedException {
    message = n;
    flag = true;
    notify();
  }

  public static void main(String args[]) {
    int n = Integer.parseInt(args[0]);
    int m = Integer.parseInt(args[1]);
    Zog old;
    First first = new First(n, m);

    first.start();

    old = first;
    while (--n > 0) {
      Zog curr = new Zog(n);

      curr.link(old);
      curr.start();
      old = curr;
    }

    first.link(old);

    try {
      first.send(m);
    } catch (InterruptedException e) {}
  }
}

class First extends Zog {
  private int procs, msgs;
  private long t1, t2, t3;

  First(int n, int m) {
    super(n);
    procs = n;
    msgs = m;
  }

  public void run() {
    long run_time, init_time;
    double msg_time, spawn_time;

    t1 = System.currentTimeMillis();
    super.run();
    t3 = System.currentTimeMillis();

    init_time = t2 - t1;
    spawn_time = (1000.0 * init_time) / procs;
    System.out.println("init_time = " + init_time + " ms (" +
		       spawn_time + " us/proc) (" + procs + " procs)");

    run_time = t3 - t2;
    msg_time = (1000.0 * run_time) / msgs;
    System.out.println("run_time = " + run_time + " ms (" +
		       msg_time + " us/msg) (" + msgs + " msgs)");
  }

  public void link(Zog zog) {
    super.link(zog);
    t2 = System.currentTimeMillis();
  }
}

6.2 Program in Erlang

-module(zog).

%% This is a test program that first creates N processes (that are
%% "connected" in a ring) and then sends M messages in that ring.
%%
%% - September 1998
%% - roland


-export([start/0, start/1, start/2]).

-export([run/2, process/1]).			% Local exports - ouch

start() -> start(16000).

start(N) -> start(N, 1000000).

start(N, M) -> spawn(?MODULE, run, [N, M]).


run(N, M) when N < 1 ->
    io:format("Must be at least 1 process~n", []),
    0.0;
run(N, M) ->
    statistics(wall_clock),

    Pid = setup(N-1, self()),

    {_,T1} = statistics(wall_clock),
    io:format("Setup : ~w s", [T1/1000]),
    case N of
	1 -> io:format(" (0 spawns)~n", []);
	_ -> io:format(" (~w us per spawn) (~w spawns)~n",
		       [1000*T1/(N-1), N-1])
    end,
    statistics(wall_clock),

    Pid ! M,
    K = process(Pid),

    {_,T2} = statistics(wall_clock),
    Time = 1000*T2/(M+K),
    io:format("Run   : ~w s (~w us per msg) (~w msgs)~n",
	      [T2/1000, Time, (M+K)]),

    Time.

setup(0, OldPid) ->
    OldPid;
setup(N, OldPid) ->
    NewPid = spawn(?MODULE, process, [OldPid]),
    setup(N-1, NewPid).


process(Pid) ->
    receive
	M ->
	    Pid ! M-1,
	    if
		M < 0  -> -M;
		true   -> process(Pid)
	    end
    end.