ivan/site
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add table

Browse Source
This commit is contained in:
Rawley
2024-02-19 10:00:44 -06:00
parent 1c28590d20
commit 9886f24641
1 changed files with 155 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files

246
perl-oo-action-benchmark.html
View File

@@ -9,50 +9,114 @@
<a href="/">&lt;&lt; back</a> <a href="/">&lt;&lt; back</a>
<h1>Perl OO benchmarking</h1> <h1>Perl OO benchmarking</h1>
<p> <p>
I've spent a lot of time working in projects that use the <a href="https://en.wikipedia.org/wiki/Command_pattern"><code>Action pattern</code></a> (also called the <code>Command pattern</code>). I've spent a lot of time working in projects that use the <a href="https://en.wikipedia.org/wiki/Command_pattern"><code>Action pattern</code></a> (also called the <code>Command pattern</code>).
Now, regardless of how i've land on OO vs Functional in the last 5 years, there has always been a question I have yet to answer: "How much does this abstraction cost, Now, regardless of how i've land on OO vs Functional in the last 5 years, there has always been a question I have yet to answer: "How much does this abstraction cost,
and more importantly, how much does object instantiation actually cost?" (since the action pattern has you instantiating A LOT of objects) and more importantly, how much does object instantiation actually cost?" (since the action pattern has you instantiating A LOT of objects)
</p> </p>
<p> <p>
So I set out to benchmark, Perl OO, using the traditional Module OO, <code>Corinna</code>, <code>Moo</code> and <code>Moose</code>. But I thought it may also So I set out to benchmark, Perl OO, using the traditional Module OO, <code>Corinna</code>, <code>Moo</code> and <code>Moose</code>. But I thought it may also
provide some value to compare these findings to other languages, namely ones in the same sort of realm that Perl has lives in: <code>Ruby</code> and <code>Raku (formerly Perl 6)</code>. provide some value to compare these findings to other languages, namely ones in the same sort of realm that Perl has lives in: <code>Ruby</code> and <code>Raku (formerly Perl 6)</code>.
</p> </p>
<h2 id="theBenchmark">The benchmark</h2> <h2 id="theBenchmark">The benchmark</h2>
<p> <p>
The benchmark is fairly simple, since the goal here is to test the actual instantiation costs, so the method associated with the implemented action is very simple. The benchmark is fairly simple, since the goal here is to test the actual instantiation costs, so the method associated with the implemented action is very simple.
An <code>Action</code> object has two members, <code>name</code> and <code>age</code>. It's <code>execute</code> method returns a Map/Hash/Dictionary, with the following keys An <code>Action</code> object has two members, <code>name</code> and <code>age</code>. It's <code>execute</code> method returns a Map/Hash/Dictionary, with the following keys
<code>{ name => self.name, age => self.age }</code>. Basically just accessing the instantiated objects <code>age</code> and <code>name</code> attributes, and associating them to <code>{ name => self.name, age => self.age }</code>. Basically just accessing the instantiated objects <code>age</code> and <code>name</code> attributes, and associating them to
keys of the same name. keys of the same name.
</p> </p>
<p> <p>
One-million <code>Action</code> objects will be instantiated in a O(n) loop, and executed, then destroyed all within the same scope. Like so: One-million <code>Action</code> objects will be instantiated in a O(n) loop, and executed, then destroyed all within the same scope. Like so:
</p> </p>
<pre> <pre>
<code> <code>
my $name = 'Tim'; my $name = 'Tim';
my $age = 12; my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( name => $name, name => $age )->execute(); my $user = Action->new( name => $name, name => $age )->execute();
} }
</code> </code>
</pre> </pre>
<p> <p>
This will be performed one-hundred times, then, the average of the times taken to do the one-million iterations will be the benchmark result. This will be performed one-hundred times, then, the average of the times taken to do the one-million iterations will be the benchmark result.
It is also important to note that the initial load-time of the language run-times is included in the calculation, though they are mostly negligble especially It is also important to note that the initial load-time of the language run-times is included in the calculation, though they are mostly negligble especially
in the Perl case, and the actual instantiation takes the vast-majority of the time (close to 99% on average). in the Perl case, and the actual instantiation takes the vast-majority of the time (close to 99% on average).
</p> </p>
<p> <p>
This benchmark was performed on an Intel i7 12700KF, using DDR5 memory. This benchmark was performed on an Intel i7 12700KF, using DDR5 memory.
</p> </p>
<h3>Summary Table</h3>
<table style="border: 1px solid black;">
<tr>
<th style="text-align: left;">Language</th>
<th style="text-align: left;">Performance</th>
</tr>
<tr>
<td>
Perl Module OO
</td>
<td>
454.627ms
</td>
</tr>
<tr>
<td>
Perl Module OO + Types
</td>
<td>
505.276ms
</td>
</tr>
<tr>
<td>
Moo
</td>
<td>
960.03ms
</td>
</tr>
<tr>
<td>
Moo + Types
</td>
<td>
2308.226ms
</td>
</tr>
<tr>
<td>
Moose
</td>
<td>
11420.455ms
</td>
</tr>
<tr>
<td>
Moose + Types
</td>
<td>
13649.37ms
</td>
</tr>
<tr>
<td>Ruby</td>
<td>167.627ms</td>
</tr>
<tr>
<td>Raku (MoarVM start-up subtracted)</td>
<td>801.947ms</td>
</tr>
</table>
<h2 id="timesAndImplementations">Times and implementations</h2> <h2 id="timesAndImplementations">Times and implementations</h2>
<h3 id="perlImpls">Perl</h3> <h3 id="perlImpls">Perl</h3>
<h4 id="perlModuleOO">Module OO</h4> <h4 id="perlModuleOO">Module OO</h4>
<p> <p>
Using the traditional Perl approach to OO using ref's and modules, is by far the fastest. Using the traditional Perl approach to OO using ref's and modules, is by far the fastest.
</p> </p>
<h5>Code:</h5> <h5>Code:</h5>
<pre> <pre>
<code> <code>
package Action; package Action;
use strict; use strict;
@@ -81,26 +145,26 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( $name, $age )->execute(); my $user = Action->new( $name, $age )->execute();
} }
</code> </code>
</pre> </pre>
<p> <p>
This implementation is alright, it's fast, but it doesn't provide any validations, which is what I assume most OO frameworks like <code>Moo</code> and <code>Moose</code> do, especially This implementation is alright, it's fast, but it doesn't provide any validations, which is what I assume most OO frameworks like <code>Moo</code> and <code>Moose</code> do, especially
if you use modules like <code>Type::Tiny</code>. if you use modules like <code>Type::Tiny</code>.
</p> </p>
<h5>Time:</h5> <h5>Time:</h5>
<pre> <pre>
<code> <code>
PERL TIME : 454.627ms PERL TIME : 454.627ms
</code> </code>
</pre> </pre>
<h4 id="perlModuleOOTypes">Module OO with some type checks</h4> <h4 id="perlModuleOOTypes">Module OO with some type checks</h4>
<p> <p>
Considering my assumptions about other OO libraries I decided to add a few checks to the Module OO implementation, Considering my assumptions about other OO libraries I decided to add a few checks to the Module OO implementation,
using <code>Scalar::Util</code> we can test if <code>age</code> is a number, and also ensure the defined-ness of our using <code>Scalar::Util</code> we can test if <code>age</code> is a number, and also ensure the defined-ness of our
attributes using the built-in <code>defined</code>. attributes using the built-in <code>defined</code>.
</p> </p>
<pre> <pre>
<code> <code>
package Action; package Action;
use strict; use strict;
@@ -139,22 +203,22 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( $name, $age )->execute(); my $user = Action->new( $name, $age )->execute();
} }
</code> </code>
</pre> </pre>
<p>This only came at a net-cost of around 50ms extra on average, which is surprisingly fast.</p> <p>This only came at a net-cost of around 50ms extra on average, which is surprisingly fast.</p>
<pre> <pre>
<code> <code>
PERL + TYPES TIME : 505.276ms PERL + TYPES TIME : 505.276ms
</code> </code>
</pre> </pre>
<h5>Code:</h5> <h5>Code:</h5>
<h4 id="perlCorinnaModuleOO">Corinna</h4> <h4 id="perlCorinnaModuleOO">Corinna</h4>
<p> <p>
Corinna is a new OO system added to Perl 5.38 but still marked as experimental. It performs very well compared to the features it provides, Corinna is a new OO system added to Perl 5.38 but still marked as experimental. It performs very well compared to the features it provides,
but it is definitely not something you'll see in the wild too often. but it is definitely not something you'll see in the wild too often.
</p> </p>
<pre> <pre>
<code> <code>
use feature 'class'; use feature 'class';
no warnings; no warnings;
@@ -172,29 +236,29 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
Action->new( name => $name, age => $age )->execute(); Action->new( name => $name, age => $age )->execute();
} }
</code> </code>
</pre> </pre>
<p> <p>
I had to disable warnings on this because I wasn't sure what flags I needed to set in the <code>feature</code> or the <code>experimental</code>, I had to disable warnings on this because I wasn't sure what flags I needed to set in the <code>feature</code> or the <code>experimental</code>,
also the lack of documentation definitely hurt, it took me a while to figure out I had to <code>use feature 'class';</code>. For some reason I thought also the lack of documentation definitely hurt, it took me a while to figure out I had to <code>use feature 'class';</code>. For some reason I thought
the feature flag was <code>corinna</code> or something. the feature flag was <code>corinna</code> or something.
</p> </p>
<h5>Time:</h5> <h5>Time:</h5>
<pre> <pre>
<code> <code>
CORINNA TIME : 564.411ms CORINNA TIME : 564.411ms
</code> </code>
</pre> </pre>
<h4 id="perlMoo">Moo</h4> <h4 id="perlMoo">Moo</h4>
<p> <p>
<code>Moo</code> is a "Minimalist Object Orientation" for Perl. It's what I've used the most, and it performs quite well for all of the <code>Moo</code> is a "Minimalist Object Orientation" for Perl. It's what I've used the most, and it performs quite well for all of the
features it gives you. However, there is definitely a cost, one that a lot of developers ignore. I've done two implementations here, one with features it gives you. However, there is definitely a cost, one that a lot of developers ignore. I've done two implementations here, one with
<code>Type::Tiny</code>, which adds optional type-checking to attributes, and one without. The difference was quite compelling! <code>Type::Tiny</code>, which adds optional type-checking to attributes, and one without. The difference was quite compelling!
</p> </p>
<h5>Code:</h5> <h5>Code:</h5>
Without <code>Type::Tiny</code>: Without <code>Type::Tiny</code>:
<pre> <pre>
<code> <code>
package Action; package Action;
use Moo; use Moo;
@@ -227,11 +291,11 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( name => $name, age => $age )->execute(); my $user = Action->new( name => $name, age => $age )->execute();
} }
</code> </code>
</pre> </pre>
With <code>Type::Tiny</code>: With <code>Type::Tiny</code>:
<pre> <pre>
<code> <code>
package Action; package Action;
use Moo; use Moo;
@@ -267,32 +331,32 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( name => $name, age => $age )->execute(); my $user = Action->new( name => $name, age => $age )->execute();
} }
</code> </code>
</pre> </pre>
<p> <p>
To my shock, <code>Type::Tiny</code> degrades performance by close to 300%. However, the more I looked into <code>Type::Tiny</code>, the more it made sense. To my shock, <code>Type::Tiny</code> degrades performance by close to 300%. However, the more I looked into <code>Type::Tiny</code>, the more it made sense.
<code>Type::Tiny</code>, to enforce types uses a lot of complex meta-programming, and ref checks, that add up to a ton of extra operations. Perhaps it would be <code>Type::Tiny</code>, to enforce types uses a lot of complex meta-programming, and ref checks, that add up to a ton of extra operations. Perhaps it would be
worth using <code>B::Concise</code> or <code>B::Deparse</code> to see how many more operations this actually adds, but thats something for another day. worth using <code>B::Concise</code> or <code>B::Deparse</code> to see how many more operations this actually adds, but thats something for another day.
</p> </p>
<h5>Time:</h5> <h5>Time:</h5>
<pre> <pre>
<code> <code>
MOO TIME : 960.03ms MOO TIME : 960.03ms
MOO + TYPES TIME : 2308.226ms MOO + TYPES TIME : 2308.226ms
</code> </code>
</pre> </pre>
<h4 id="perlMoose">Moose</h4> <h4 id="perlMoose">Moose</h4>
<p> <p>
<code>Moose</code> is a heavyweight, industrial purpose OO framework for Perl, at which <code>Moo</code> derives from. It is big, and heavy, and unfortunately, <code>Moose</code> is a heavyweight, industrial purpose OO framework for Perl, at which <code>Moo</code> derives from. It is big, and heavy, and unfortunately,
really slow for tasks like this. After starting this benchmark, I thought that the slowness was coming from loading the module between runs, but after timing the average load time, really slow for tasks like this. After starting this benchmark, I thought that the slowness was coming from loading the module between runs, but after timing the average load time,
the time spent was negligble (31ms at most). the time spent was negligble (31ms at most).
</p> </p>
<p> <p>
Like in the <code>Moo</code> benchmark, I did two implementations, one using <code>Type::Tiny</code>, and one not. Like in the <code>Moo</code> benchmark, I did two implementations, one using <code>Type::Tiny</code>, and one not.
</p> </p>
<h5>Code:</h5> <h5>Code:</h5>
<pre> <pre>
<code> <code>
package Action; package Action;
use Moose; use Moose;
@@ -325,11 +389,11 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( name => $name, age => $age )->execute(); my $user = Action->new( name => $name, age => $age )->execute();
} }
</code> </code>
</pre> </pre>
With <code>Type::Tiny</code>: With <code>Type::Tiny</code>:
<pre> <pre>
<code> <code>
package Action; package Action;
use Moose; use Moose;
@@ -365,29 +429,29 @@ my $age = 12;
for ( 1 .. 1000000 ) { for ( 1 .. 1000000 ) {
my $user = Action->new( name => $name, age => $age )->execute(); my $user = Action->new( name => $name, age => $age )->execute();
} }
</code> </code>
</pre> </pre>
<p> <p>
Just like <code>Moo</code> with types, <code>Type::Tiny</code> hits the final <code>Moose</code> result by around 2000ms overall. Just like <code>Moo</code> with types, <code>Type::Tiny</code> hits the final <code>Moose</code> result by around 2000ms overall.
</p> </p>
<h5>Time:</h5> <h5>Time:</h5>
<pre> <pre>
<code> <code>
MOOSE TIME : 11420.455ms MOOSE TIME : 11420.455ms
MOOSE + TYPES TIME : 13649.37ms MOOSE + TYPES TIME : 13649.37ms
</code> </code>
</pre> </pre>
<h3 id="rubyImpl">Ruby</h3> <h3 id="rubyImpl">Ruby</h3>
<p> <p>
I also did a quick Ruby implementation, since its a "true" Object-Oriented language, I assumed it would have I also did a quick Ruby implementation, since its a "true" Object-Oriented language, I assumed it would have
a lot of optimizations built-in for this sort of work, and based on its runtime, that seems to be the case. a lot of optimizations built-in for this sort of work, and based on its runtime, that seems to be the case.
</p> </p>
<p> <p>
Also, note, this was on Ruby 3.1.2, not Ruby 3.3 with YJIT, so this could theoretically be faster. Also, note, this was on Ruby 3.1.2, not Ruby 3.3 with YJIT, so this could theoretically be faster.
</p> </p>
<h5>Code:</h5> <h5>Code:</h5>
<pre> <pre>
<code> <code>
class Action class Action
def initialize(age, name) def initialize(age, name)
@age = age @age = age
@@ -404,23 +468,23 @@ name = 'Tim'
1000000.times do 1000000.times do
user = Action.new(age, name).execute() user = Action.new(age, name).execute()
end end
</code> </code>
</pre> </pre>
<h5>Time:</h5> <h5>Time:</h5>
<pre> <pre>
<code> <code>
RUBY TIME : 167.627ms RUBY TIME : 167.627ms
</code> </code>
</pre> </pre>
<h3 id="rakuImpl">Raku (formerly Perl 6)</h3> <h3 id="rakuImpl">Raku (formerly Perl 6)</h3>
<p> <p>
Finally, we have Raku, which unfortunately doesn't do too well. To make this a little more competitive, Finally, we have Raku, which unfortunately doesn't do too well. To make this a little more competitive,
I subtracted the run-time of MoarVM from the end-result. Though, I think Raku's implementation is the simplest to I subtracted the run-time of MoarVM from the end-result. Though, I think Raku's implementation is the simplest to
understand in terms of langauge design, so it gets a +1 from me for that. understand in terms of langauge design, so it gets a +1 from me for that.
</p> </p>
<h5>Code:</h5> <h5>Code:</h5>
<pre> <pre>
<code> <code>
class Action { class Action {
has $.name; has $.name;
has $.age; has $.age;
@@ -436,39 +500,39 @@ for 1..1000000 -> $ {
my $user = Action.new(name => $name, age => $age).execute(); my $user = Action.new(name => $name, age => $age).execute();
} }
</code> </code>
</pre> </pre>
<h5>Time (with MoarVM startup time removed):</h5> <h5>Time (with MoarVM startup time removed):</h5>
<pre> <pre>
<code> <code>
RAKU TIME : 801.947ms RAKU TIME : 801.947ms
</code> </code>
</pre> </pre>
<h2>So what?</h2> <h2>So what?</h2>
<p> <p>
Object instantiation has a cost. A lot of people ignore this, especially when using heavyweight libraries like <code>Moose</code>. Object instantiation has a cost. A lot of people ignore this, especially when using heavyweight libraries like <code>Moose</code>.
You can put this into practical terms simply, using basic Perl module OO you effectively double the number of objects of this type you can create and execute a You can put this into practical terms simply, using basic Perl module OO you effectively double the number of objects of this type you can create and execute a
method on given any time-frame compared to <code>Moo</code> without types, and twenty-two times more objects versus <code>Moose</code> without types. method on given any time-frame compared to <code>Moo</code> without types, and twenty-two times more objects versus <code>Moose</code> without types.
But these abstractions exist for reasons, and they are used for a reason. But these abstractions exist for reasons, and they are used for a reason.
</p> </p>
<p> <p>
<code>Moo</code> and <code>Moose</code> have their places, especially in web-applications where the number of instantiations is low, and the expensive <code>Moo</code> and <code>Moose</code> have their places, especially in web-applications where the number of instantiations is low, and the expensive
operations happen during side-effects like database operations. operations happen during side-effects like database operations.
This also goes without mentioning some of the great features modules like <code>Moo</code> and <code>Moose</code> provide. This also goes without mentioning some of the great features modules like <code>Moo</code> and <code>Moose</code> provide.
It is valuable to know that if you need something to perform well, <code>Moo</code> and <code>Moose</code> It is valuable to know that if you need something to perform well, <code>Moo</code> and <code>Moose</code>
may incur overhead you hadn't previously considered. This is compounded when enforcing types. may incur overhead you hadn't previously considered. This is compounded when enforcing types.
For hot-code that will be hit repeatedly within a short amount of time, this can drastically effect performance, think of things like a landing page, or a search-bar API. For hot-code that will be hit repeatedly within a short amount of time, this can drastically effect performance, think of things like a landing page, or a search-bar API.
</p> </p>
<p> <p>
<code>Corinna</code>, is a very nice prospect, it offers a lot of great things from <code>Moo</code> and <code>Moose</code> but seems to <code>Corinna</code>, is a very nice prospect, it offers a lot of great things from <code>Moo</code> and <code>Moose</code> but seems to
exalt far less of a performance penalty. It will definitely be on my radar in the future, especially when the documentation improves. exalt far less of a performance penalty. It will definitely be on my radar in the future, especially when the documentation improves.
</p> </p>
<p> <p>
Another approach is to simply use functions, which of course will perform the best. But, it's hard to argue to do this unless performance is the number one goal, Another approach is to simply use functions, which of course will perform the best. But, it's hard to argue to do this unless performance is the number one goal,
especially on codebases that follow this pattern. especially on codebases that follow this pattern.
</p> </p>
<div> <div>
You can view the <a href="https://github.com/rawleyfowler/perl-oo-action-benchmark">Source code</a>. You can view the <a href="https://github.com/rawleyfowler/perl-oo-action-benchmark">Source code</a>.
</div> </div>
</div> </div>
</body> </body>