Speed Comparison of Smalltalk, IronPython, C# and ActionScript

There was a question from a recent post about the relative speed of my Smalltalk implementation. To answer the question (and my own curiosity), I performed  a simple benchmark test on three versions of Smalltalk, IronPython, ActionScript and C#.

The test was run five times in each language and the lowest time was recorded. All tests were performed on the same machine (3 GHz Pentium, 1 GB memory, Windows Vista).

The results were:

Language	Milliseconds
C#	9
ActionScript	15
VisualWorks	112
IronPython	430
Squeak	1233
Smalltalk/DLR	4000

Some notes about the testing:

• C# – the test initially showed 2 milliseconds, probably because the compiler recognized that the computed values weren’t being used and “short-circuited” the test. After adding a statement at the end of the test to print out the values, the test took 9 milliseconds.
• ActionScript – I ran two versions of the ActionScript test. The first used typed variables and took 15 milliseconds to run. The second test used untyped variables and took 371 milliseconds to run.
• IronPython – the test was run in a browser using the DLRConsole demo.
• Smalltalk/DLR – this is my 2 week old Smalltalk compiler. Once I learn more about DLR optimization, I expect the speed to be similar to IronPython.
• C# and ActionScript use typed variables, whereas the other languages use untyped variables. Not surprisingly, the more information that a compiler has, the better it can optimize the code for speed.
• It is very easy to call C# code from DLR-based languages – an strategy for increasing speed is to recode portions of a program in C# and then call it from the dynamic code.

Subjectively, I have always found ActionScript 3.0 to be amazingly fast – the results, for this benchmark at least, bear that out. Also, IronPython is doing very well for an alpha release.

Anyway, it was an interesting experiment.

Below is the code used in the tests.

Smalltalk/DLR

"-- Local variables --"
| index x y z |
"-- Import DateTime --"
Vm import: 'System.DateTime'.
"-- Write header --"
Transcript clear.
Transcript show: 'Starting 1,000,000 iterations'; cr; cr.
Transcript show: DateTime Now; cr."– Iterations –"
index := 0.
[index < 1000000] whileTrue: [
  index := index + 1.
  x := 3.45 + index.
  y := 7.89 + index.
  z := (x*x)+(y*y) + index].
"– Results –"
Transcript show: DateTime Now; cr; cr.
Transcript show: index; cr.
Transcript show: x; cr.
Transcript show: y; cr.
Transcript show: z; cr.

Squeak / VisualWorks

| index x y z |

Time millisecondsToRun:[
index := 0.
[index < 1000000] whileTrue: [
  index := index + 1.
  x := 3.45 + index.
  y := 7.89 + index.
  z := (x*x)+(y*y) + index]]

ActionScript 1 (using Number type variables)

private function test1():Number {
  var index:int;
  var x:Number;
  var y:Number;
  var z:Number;
  var t1:Number;
  var t2:Number;
  t1 = (new Date()).getMilliseconds();
  for(index=0;index<1000000;index++) {
    x=2.34+index;
    y=4.56+index;
    z=(x*x)+(y*y)+index;
  }
  t2 = (new Date()).getMilliseconds();
  return t2-t1;
  }

ActionScript 2 (using ”type undefined” variables)

private function test2():Number{
  var index:int;
  var x:*;
  var y:*;
  var z:*;
  var t1:Number;
  var t2:Number;
  t1 = (new Date()).getMilliseconds();
  for(index=0;index<1000000;index++) {
    x=2.34+index;
    y=4.56+index;
    z=(x*x)+(y*y)+index;
  }
  t2 = (new Date()).getMilliseconds();
  return t2-t1;
  }

 IronPython

def test():
  t1 = DateTime.Now
  for index in range(1000000):
  x = 2.34+index
  y = 3.45+index
  z = (x*x)+(y*y)+index
  t2 = DateTime.Now
  print (t2-t1).Milliseconds

C#

private double Test() {
  double x = 0, y = 0, z = 0;
  DateTime t1 = DateTime.Now;
  for (int index = 0; index < 1000000; index++) {
    x = 2.34 + index;
    y = 3.45 + index;
    z = (x * x) + (y * y) + index;
  }
  DateTime t2 = DateTime.Now;
  Console.WriteLine(x + y + z);
  return (t2 - t1).Milliseconds;
}