From 9886f246413c74ed90b7c0b8cb5a488cbb047a7d Mon Sep 17 00:00:00 2001 From: Rawley Date: Mon, 19 Feb 2024 10:00:44 -0600 Subject: [PATCH] add table --- perl-oo-action-benchmark.html | 246 +++++++++++++++++++++------------- 1 file changed, 155 insertions(+), 91 deletions(-) diff --git a/perl-oo-action-benchmark.html b/perl-oo-action-benchmark.html index f0405c3..b8255f5 100644 --- a/perl-oo-action-benchmark.html +++ b/perl-oo-action-benchmark.html @@ -9,50 +9,114 @@ << back

Perl OO benchmarking

- I've spent a lot of time working in projects that use the Action pattern (also called the Command pattern). - Now, regardless of how i've land on OO vs Functional in the last 5 years, there has always been a question I have yet to answer: "How much does this abstraction cost, - and more importantly, how much does object instantiation actually cost?" (since the action pattern has you instantiating A LOT of objects) + I've spent a lot of time working in projects that use the Action pattern (also called the Command pattern). + Now, regardless of how i've land on OO vs Functional in the last 5 years, there has always been a question I have yet to answer: "How much does this abstraction cost, + and more importantly, how much does object instantiation actually cost?" (since the action pattern has you instantiating A LOT of objects)

- So I set out to benchmark, Perl OO, using the traditional Module OO, Corinna, Moo and Moose. But I thought it may also - provide some value to compare these findings to other languages, namely ones in the same sort of realm that Perl has lives in: Ruby and Raku (formerly Perl 6). + So I set out to benchmark, Perl OO, using the traditional Module OO, Corinna, Moo and Moose. But I thought it may also + provide some value to compare these findings to other languages, namely ones in the same sort of realm that Perl has lives in: Ruby and Raku (formerly Perl 6).

The benchmark

- The benchmark is fairly simple, since the goal here is to test the actual instantiation costs, so the method associated with the implemented action is very simple. - An Action object has two members, name and age. It's execute method returns a Map/Hash/Dictionary, with the following keys - { name => self.name, age => self.age }. Basically just accessing the instantiated objects age and name attributes, and associating them to - keys of the same name. + The benchmark is fairly simple, since the goal here is to test the actual instantiation costs, so the method associated with the implemented action is very simple. + An Action object has two members, name and age. It's execute method returns a Map/Hash/Dictionary, with the following keys + { name => self.name, age => self.age }. Basically just accessing the instantiated objects age and name attributes, and associating them to + keys of the same name.

- One-million Action objects will be instantiated in a O(n) loop, and executed, then destroyed all within the same scope. Like so: + One-million Action objects will be instantiated in a O(n) loop, and executed, then destroyed all within the same scope. Like so: 

-              
+                
 my $name = 'Tim';
 my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( name => $name, name => $age )->execute();
 }
-              
+

- This will be performed one-hundred times, then, the average of the times taken to do the one-million iterations will be the benchmark result. - It is also important to note that the initial load-time of the language run-times is included in the calculation, though they are mostly negligble especially - in the Perl case, and the actual instantiation takes the vast-majority of the time (close to 99% on average). + This will be performed one-hundred times, then, the average of the times taken to do the one-million iterations will be the benchmark result. + It is also important to note that the initial load-time of the language run-times is included in the calculation, though they are mostly negligble especially + in the Perl case, and the actual instantiation takes the vast-majority of the time (close to 99% on average).

- This benchmark was performed on an Intel i7 12700KF, using DDR5 memory. + This benchmark was performed on an Intel i7 12700KF, using DDR5 memory.

+

Summary Table

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
LanguagePerformance
+ Perl Module OO + + 454.627ms +
+ Perl Module OO + Types + + 505.276ms +
+ Moo + + 960.03ms +
+ Moo + Types + + 2308.226ms +
+ Moose + + 11420.455ms +
+ Moose + Types + + 13649.37ms +
Ruby167.627ms
Raku (MoarVM start-up subtracted)801.947ms

Times and implementations

Perl

Module OO

- Using the traditional Perl approach to OO using ref's and modules, is by far the fastest. + Using the traditional Perl approach to OO using ref's and modules, is by far the fastest.

Code:
-              
+                
 package Action;
 
 use strict;
@@ -81,26 +145,26 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( $name, $age )->execute();
 }
-              
+

- This implementation is alright, it's fast, but it doesn't provide any validations, which is what I assume most OO frameworks like Moo and Moose do, especially - if you use modules like Type::Tiny. + This implementation is alright, it's fast, but it doesn't provide any validations, which is what I assume most OO frameworks like Moo and Moose do, especially + if you use modules like Type::Tiny.

Time:
-              
+                
 PERL TIME : 454.627ms
-              
+

Module OO with some type checks

- Considering my assumptions about other OO libraries I decided to add a few checks to the Module OO implementation, - using Scalar::Util we can test if age is a number, and also ensure the defined-ness of our - attributes using the built-in defined. + Considering my assumptions about other OO libraries I decided to add a few checks to the Module OO implementation, + using Scalar::Util we can test if age is a number, and also ensure the defined-ness of our + attributes using the built-in defined. 

-              
+                
 package Action;
 
 use strict;
@@ -139,22 +203,22 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( $name, $age )->execute();
 }
-              
+

This only came at a net-cost of around 50ms extra on average, which is surprisingly fast.

-              
+                
 PERL + TYPES TIME : 505.276ms
-              
+
Code:

Corinna

- Corinna is a new OO system added to Perl 5.38 but still marked as experimental. It performs very well compared to the features it provides, - but it is definitely not something you'll see in the wild too often. + Corinna is a new OO system added to Perl 5.38 but still marked as experimental. It performs very well compared to the features it provides, + but it is definitely not something you'll see in the wild too often. 

-              
+                
 use feature 'class';
 no warnings;
 
@@ -172,29 +236,29 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     Action->new( name => $name, age => $age )->execute();
 }
-              
+

- I had to disable warnings on this because I wasn't sure what flags I needed to set in the feature or the experimental, - also the lack of documentation definitely hurt, it took me a while to figure out I had to use feature 'class';. For some reason I thought - the feature flag was corinna or something. + I had to disable warnings on this because I wasn't sure what flags I needed to set in the feature or the experimental, + also the lack of documentation definitely hurt, it took me a while to figure out I had to use feature 'class';. For some reason I thought + the feature flag was corinna or something.

Time:
-              
+                
 CORINNA TIME : 564.411ms
-              
+

Moo

- Moo is a "Minimalist Object Orientation" for Perl. It's what I've used the most, and it performs quite well for all of the - features it gives you. However, there is definitely a cost, one that a lot of developers ignore. I've done two implementations here, one with - Type::Tiny, which adds optional type-checking to attributes, and one without. The difference was quite compelling! + Moo is a "Minimalist Object Orientation" for Perl. It's what I've used the most, and it performs quite well for all of the + features it gives you. However, there is definitely a cost, one that a lot of developers ignore. I've done two implementations here, one with + Type::Tiny, which adds optional type-checking to attributes, and one without. The difference was quite compelling!

Code:
Without Type::Tiny: 
-              
+                
 package Action;
 
 use Moo;
@@ -227,11 +291,11 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( name => $name, age => $age )->execute();
 }
-              
+
With Type::Tiny: 
-              
+                
 package Action;
 
 use Moo;
@@ -267,32 +331,32 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( name => $name, age => $age )->execute();
 }
-              
+

- To my shock, Type::Tiny degrades performance by close to 300%. However, the more I looked into Type::Tiny, the more it made sense. - Type::Tiny, to enforce types uses a lot of complex meta-programming, and ref checks, that add up to a ton of extra operations. Perhaps it would be - worth using B::Concise or B::Deparse to see how many more operations this actually adds, but thats something for another day. + To my shock, Type::Tiny degrades performance by close to 300%. However, the more I looked into Type::Tiny, the more it made sense. + Type::Tiny, to enforce types uses a lot of complex meta-programming, and ref checks, that add up to a ton of extra operations. Perhaps it would be + worth using B::Concise or B::Deparse to see how many more operations this actually adds, but thats something for another day.

Time:
-              
+                
 MOO TIME         : 960.03ms
 MOO + TYPES TIME : 2308.226ms
-              
+

Moose

- Moose is a heavyweight, industrial purpose OO framework for Perl, at which Moo derives from. It is big, and heavy, and unfortunately, - really slow for tasks like this. After starting this benchmark, I thought that the slowness was coming from loading the module between runs, but after timing the average load time, - the time spent was negligble (31ms at most). + Moose is a heavyweight, industrial purpose OO framework for Perl, at which Moo derives from. It is big, and heavy, and unfortunately, + really slow for tasks like this. After starting this benchmark, I thought that the slowness was coming from loading the module between runs, but after timing the average load time, + the time spent was negligble (31ms at most).

- Like in the Moo benchmark, I did two implementations, one using Type::Tiny, and one not. + Like in the Moo benchmark, I did two implementations, one using Type::Tiny, and one not.

Code:
-              
+                
 package Action;
 
 use Moose;
@@ -325,11 +389,11 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( name => $name, age => $age )->execute();
 }
-              
+
With Type::Tiny: 
-              
+                
 package Action;
 
 use Moose;
@@ -365,29 +429,29 @@ my $age  = 12;
 for ( 1 .. 1000000 ) {
     my $user = Action->new( name => $name, age => $age )->execute();
 }
-              
+

- Just like Moo with types, Type::Tiny hits the final Moose result by around 2000ms overall. + Just like Moo with types, Type::Tiny hits the final Moose result by around 2000ms overall.

Time:
-              
+                
 MOOSE TIME         : 11420.455ms
 MOOSE + TYPES TIME : 13649.37ms
-              
+

Ruby

- I also did a quick Ruby implementation, since its a "true" Object-Oriented language, I assumed it would have - a lot of optimizations built-in for this sort of work, and based on its runtime, that seems to be the case. + I also did a quick Ruby implementation, since its a "true" Object-Oriented language, I assumed it would have + a lot of optimizations built-in for this sort of work, and based on its runtime, that seems to be the case.

- Also, note, this was on Ruby 3.1.2, not Ruby 3.3 with YJIT, so this could theoretically be faster. + Also, note, this was on Ruby 3.1.2, not Ruby 3.3 with YJIT, so this could theoretically be faster.

Code:
-              
+                
 class Action
   def initialize(age, name)
     @age = age
@@ -404,23 +468,23 @@ name = 'Tim'
 1000000.times do
   user = Action.new(age, name).execute()
 end
-              
+
Time:
-              
+                
 RUBY TIME : 167.627ms
-              
+

Raku (formerly Perl 6)

- Finally, we have Raku, which unfortunately doesn't do too well. To make this a little more competitive, - I subtracted the run-time of MoarVM from the end-result. Though, I think Raku's implementation is the simplest to - understand in terms of langauge design, so it gets a +1 from me for that. + Finally, we have Raku, which unfortunately doesn't do too well. To make this a little more competitive, + I subtracted the run-time of MoarVM from the end-result. Though, I think Raku's implementation is the simplest to + understand in terms of langauge design, so it gets a +1 from me for that.

Code:
-              
+                
 class Action {
     has $.name;
     has $.age;
@@ -436,39 +500,39 @@ for 1..1000000 -> $ {
     my $user = Action.new(name => $name, age => $age).execute();
 }
 
-              
+
Time (with MoarVM startup time removed):
-              
+                
 RAKU TIME : 801.947ms
-              
+

So what?

- Object instantiation has a cost. A lot of people ignore this, especially when using heavyweight libraries like Moose. - You can put this into practical terms simply, using basic Perl module OO you effectively double the number of objects of this type you can create and execute a - method on given any time-frame compared to Moo without types, and twenty-two times more objects versus Moose without types. - But these abstractions exist for reasons, and they are used for a reason. + Object instantiation has a cost. A lot of people ignore this, especially when using heavyweight libraries like Moose. + You can put this into practical terms simply, using basic Perl module OO you effectively double the number of objects of this type you can create and execute a + method on given any time-frame compared to Moo without types, and twenty-two times more objects versus Moose without types. + But these abstractions exist for reasons, and they are used for a reason.

- Moo and Moose have their places, especially in web-applications where the number of instantiations is low, and the expensive - operations happen during side-effects like database operations. - This also goes without mentioning some of the great features modules like Moo and Moose provide. - It is valuable to know that if you need something to perform well, Moo and Moose - may incur overhead you hadn't previously considered. This is compounded when enforcing types. - For hot-code that will be hit repeatedly within a short amount of time, this can drastically effect performance, think of things like a landing page, or a search-bar API. + Moo and Moose have their places, especially in web-applications where the number of instantiations is low, and the expensive + operations happen during side-effects like database operations. + This also goes without mentioning some of the great features modules like Moo and Moose provide. + It is valuable to know that if you need something to perform well, Moo and Moose + may incur overhead you hadn't previously considered. This is compounded when enforcing types. + For hot-code that will be hit repeatedly within a short amount of time, this can drastically effect performance, think of things like a landing page, or a search-bar API.

- Corinna, is a very nice prospect, it offers a lot of great things from Moo and Moose but seems to - exalt far less of a performance penalty. It will definitely be on my radar in the future, especially when the documentation improves. + Corinna, is a very nice prospect, it offers a lot of great things from Moo and Moose but seems to + exalt far less of a performance penalty. It will definitely be on my radar in the future, especially when the documentation improves.

- Another approach is to simply use functions, which of course will perform the best. But, it's hard to argue to do this unless performance is the number one goal, - especially on codebases that follow this pattern. + Another approach is to simply use functions, which of course will perform the best. But, it's hard to argue to do this unless performance is the number one goal, + especially on codebases that follow this pattern.

- You can view the Source code. + You can view the Source code.