Update benchmarks

BENCHMARK.md

@@ -0,0 +1,302 @@
+Benchmark
+=========
+
+Environment
+-----------
+
+- OS: Ubuntu-15.10
+- CPU: Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz (x2)
+- Erlang: OTP-18.3.1 ([package link](https://packages.erlang-solutions.com/erlang/esl-erlang/FLAVOUR_1_general/esl-erlang_18.3-1~ubuntu~wily_amd64.deb))
+- Benchmark: [The benchmark of poison(v2.1.0)](https://github.com/devinus/poison#benchmarking)
+
+Benchmarks
+----------
+
+To execute following commands, the (Elixir)[http://elixir-lang.org/] is required.
+
+```sh
+###
+### Downloads poison
+###
+$ git clone https://github.com/devinus/poison
+$ cd poison
+$ git checkout 2.1.0
+
+
+###
+### Benchmark: Non HiPE
+###
+$ patch -p1 < non_hipe.patch
+$ mix deps.get
+$ MIX_ENV=bench mix compile
+$ MIX_ENV=bench mix bench
+Settings:
+duration:      1.0 s
+... abbrev ...
+Finished in 89.84 seconds
+
+## EncoderBench
+structs (Poison)               500000   6.35 µs/op
+maps (jiffy)                   500000   7.35 µs/op
+maps (jsone)                   100000   10.00 µs/op
+maps (Poison)                  100000   13.26 µs/op
+structs (Jazz)                 100000   14.18 µs/op
+structs (JSX)                  100000   17.83 µs/op
+maps (Jazz)                    100000   18.97 µs/op
+maps (JSX)                      50000   30.03 µs/op
+strings (jiffy)                 10000   103.77 µs/op
+lists (Poison)                  10000   114.34 µs/op
+string escaping (jiffy)         10000   141.46 µs/op
+lists (jsone)                   10000   151.70 µs/op
+lists (Jazz)                    10000   197.31 µs/op
+lists (jiffy)                   10000   214.93 µs/op
+strings (JSX)                   10000   237.08 µs/op
+lists (JSX)                      5000   337.25 µs/op
+strings (Poison)                 5000   350.49 µs/op
+jiffy                            5000   404.95 µs/op
+jiffy (pretty)                   5000   415.19 µs/op
+strings (Jazz)                   5000   445.60 µs/op
+strings (jsone)                  5000   550.78 µs/op
+string escaping (jsone)          2000   934.83 µs/op
+Poison                           1000   1322.36 µs/op
+string escaping (JSX)            1000   1371.11 µs/op
+jsone                            1000   1379.04 µs/op
+string escaping (Poison)         1000   1387.10 µs/op
+Poison (pretty)                  1000   1453.36 µs/op
+string escaping (Jazz)           1000   1568.84 µs/op
+jsone (pretty)                   1000   1734.20 µs/op
+Jazz                             1000   1818.49 µs/op
+JSX                              1000   2027.41 µs/op
+Jazz (pretty)                    1000   2031.73 µs/op
+JSX (pretty)                      500   5223.72 µs/op
+
+## ParserBench
+UTF-8 unescaping (jiffy)        50000   59.63 µs/op
+UTF-8 unescaping (Poison)       10000   219.79 µs/op
+UTF-8 unescaping (JSX)          10000   243.64 µs/op
+UTF-8 unescaping (jsone)         5000   342.50 µs/op
+jiffy                            5000   522.25 µs/op
+jsone                            1000   1217.44 µs/op
+Poison                           1000   1223.37 µs/op
+JSX                              1000   1630.77 µs/op
+
+
+###
+### Benchmark: HiPE
+###
+$ patch -p1 < hipe.patch
+$ mix clean --deps && find . -name '*.beam' | xargs rm
+$ ERL_COMPILER_OPTIONS="native" MIX_ENV=bench mix compile
+$ MIX_ENV=bench mix bench
+Settings:
+duration:      1.0 s
+... abbrev ...
+Finished in 103.42 seconds
+
+## EncoderBench
+maps (jsone)                   500000   5.58 µs/op
+maps (jiffy)                   500000   7.12 µs/op
+structs (Poison)               500000   7.66 µs/op
+maps (Poison)                  100000   10.71 µs/op
+structs (JSX)                  100000   14.68 µs/op
+structs (Jazz)                 100000   19.85 µs/op
+maps (Jazz)                    100000   21.08 µs/op
+maps (JSX)                     100000   25.38 µs/op
+lists (jsone)                   50000   64.76 µs/op
+lists (Poison)                  20000   76.80 µs/op
+strings (jiffy)                 10000   109.11 µs/op
+string escaping (jiffy)         10000   140.35 µs/op
+strings (Poison)                10000   140.58 µs/op
+strings (JSX)                   10000   179.35 µs/op
+lists (jiffy)                   10000   198.89 µs/op
+lists (Jazz)                    10000   217.64 µs/op
+lists (JSX)                     10000   220.85 µs/op
+strings (jsone)                  5000   323.63 µs/op
+strings (Jazz)                   5000   363.87 µs/op
+jiffy                            5000   388.06 µs/op
+jiffy (pretty)                   5000   404.84 µs/op
+string escaping (jsone)          5000   481.52 µs/op
+jsone                            5000   634.06 µs/op
+string escaping (JSX)            5000   699.05 µs/op
+Poison                           2000   744.13 µs/op
+Poison (pretty)                  2000   802.29 µs/op
+jsone (pretty)                   2000   956.29 µs/op
+string escaping (Poison)         1000   1222.81 µs/op
+string escaping (Jazz)           1000   1312.68 µs/op
+Jazz                             1000   1572.41 µs/op
+JSX                              1000   1667.37 µs/op
+Jazz (pretty)                    1000   1795.51 µs/op
+JSX (pretty)                      500   4434.13 µs/op
+
+## ParserBench
+UTF-8 unescaping (jiffy)        50000   59.95 µs/op
+UTF-8 unescaping (jsone)        20000   83.79 µs/op
+UTF-8 unescaping (Poison)       10000   109.70 µs/op
+UTF-8 unescaping (JSX)          10000   159.35 µs/op
+jiffy                            5000   504.77 µs/op
+jsone                            5000   527.77 µs/op
+Poison                           5000   686.90 µs/op
+JSX                              1000   1320.74 µs/op
+```
+
+Libraries Version
+-----------------
+
+mix.lock:
+```elixir
+%{"benchfella": {:hex, :benchfella, "0.3.1"},
+  "earmark": {:hex, :earmark, "0.2.1"},
+  "ex_doc": {:hex, :ex_doc, "0.11.4"},
+  "exjsx": {:git, "https://github.com/talentdeficit/exjsx.git", "53db5d995b1b070e3883381e9acb195969137f67", []},
+  "jazz": {:git, "https://github.com/meh/jazz.git", "49f335492aca5516495199dd81dd18b845ebaa69", []},
+  "jiffy": {:git, "https://github.com/davisp/jiffy.git", "6303ff98aaa3fce625038c8b7af2aa8b802f4742", []},
+  "jsone": {:hex, :jsone, "1.2.2"},
+  "jsx": {:hex, :jsx, "2.6.2"}}
+```
+
+Patches
+-------
+
+non_hipe.patch:
+```patch
+diff --git a/bench/encoder_bench.exs b/bench/encoder_bench.exs
+index cf40dcd..3453a2d 100644
+--- a/bench/encoder_bench.exs
++++ b/bench/encoder_bench.exs
+@@ -10,6 +10,10 @@ defmodule EncoderBench do
+     :jiffy.encode(list)
+   end
+
++  bench "lists (jsone)", [list: gen_list] do
++    :jsone.encode(list)
++  end
++
+   bench "lists (JSX)", [list: gen_list] do
+     JSX.encode!(list)
+   end
+@@ -27,6 +31,10 @@ defmodule EncoderBench do
+     :jiffy.encode(map)
+   end
+
++  bench "maps (jsone)", [map: gen_map] do
++    :jsone.encode(map)
++  end
++
+   bench "maps (JSX)", [map: gen_map] do
+     JSX.encode!(map)
+   end
+@@ -44,6 +52,10 @@ defmodule EncoderBench do
+     :jiffy.encode(string)
+   end
+
++  bench "strings (jsone)", [string: gen_string] do
++    :jsone.encode(string, [:native_utf8])
++  end
++
+   bench "strings (JSX)", [string: gen_string] do
+     JSX.encode!(string)
+   end
+@@ -61,6 +73,10 @@ defmodule EncoderBench do
+     :jiffy.encode(string, [:uescape])
+   end
+
++  bench "string escaping (jsone)", [string: gen_string] do
++    :jsone.encode(string)
++  end
++
+   bench "string escaping (JSX)", [string: gen_string] do
+     JSX.encode!(string, [:uescape])
+   end
+@@ -90,6 +106,10 @@ defmodule EncoderBench do
+     :jiffy.encode(data)
+   end
+
++  bench "jsone", [data: gen_data] do
++    :jsone.encode(data)
++  end
++
+   bench "JSX", [data: gen_data] do
+     JSX.encode!(data)
+   end
+@@ -106,6 +126,10 @@ defmodule EncoderBench do
+     :jiffy.encode(data, [:pretty])
+   end
+
++  bench "jsone (pretty)", [data: gen_data] do
++    :jsone.encode(data, [{:indent, 1}, {:space, 2}])
++  end
++
+   bench "JSX (pretty)", [data: gen_data] do
+     JSX.encode!(data) |> JSX.prettify!
+   end
+diff --git a/bench/parser_bench.exs b/bench/parser_bench.exs
+index a71fbff..114f226 100644
+--- a/bench/parser_bench.exs
++++ b/bench/parser_bench.exs
+@@ -12,6 +12,10 @@ defmodule ParserBench do
+     :jiffy.decode(json, [:return_maps])
+   end
+
++  bench "jsone", [json: gen_json] do
++    :jsone.decode(json)
++  end
++
+   bench "JSX", [json: gen_json] do
+     JSX.decode!(json, [:strict])
+   end
+@@ -25,6 +29,10 @@ defmodule ParserBench do
+     :jiffy.decode(utf8)
+   end
+
++  bench "UTF-8 unescaping (jsone)", [utf8: gen_utf8] do
++    :jsone.decode(utf8)
++  end
++
+   bench "UTF-8 unescaping (JSX)", [utf8: gen_utf8] do
+     JSX.decode!(utf8, [:strict])
+   end
+diff --git a/config/config.exs b/config/config.exs
+index 1240a78..d176e11 100644
+--- a/config/config.exs
++++ b/config/config.exs
+@@ -15,7 +15,8 @@ use Mix.Config
+ #       format: "$time $metadata[$level] $message\n"
+
+ config :poison,
+-  native: :erlang.system_info(:hipe_architecture) != :undefined
++#  native: :erlang.system_info(:hipe_architecture) != :undefined
++  native: false
+
+ # It is also possible to import configuration files, relative to this
+ # directory. For example, you can emulate configuration per environment
+diff --git a/mix.exs b/mix.exs
+index 89a8c10..ac2f53e 100644
+--- a/mix.exs
++++ b/mix.exs
+@@ -33,6 +33,7 @@ defmodule Poison.Mixfile do
+     [{:earmark, "~> 0.2", only: :docs},
+      {:ex_doc, "~> 0.11", only: :docs},
+      {:benchfella, "~> 0.3", only: :bench},
++     {:jsone, "~> 1.2.2", only: :bench},
+      {:jiffy, github: "davisp/jiffy", only: :bench},
+      {:exjsx, github: "talentdeficit/exjsx", only: :bench},
+      {:jazz, github: "meh/jazz", only: :bench}]
+```
+
+hipe.patch:
+```patch
+diff --git a/config/config.exs b/config/config.exs
+index 013831e..1240a78 100644
+--- a/config/config.exs
++++ b/config/config.exs
+@@ -15,8 +15,7 @@ use Mix.Config
+#       format: "$time $metadata[$level] $message\n"
+
+config :poison,
+-#  native: :erlang.system_info(:hipe_architecture) != :undefined
+-  native: false
++  native: :erlang.system_info(:hipe_architecture) != :undefined
+
+# It is also possible to import configuration files, relative to this
+# directory. For example, you can emulate configuration per environment
+```

README.md

@@ -183,50 +183,51 @@ See [EDoc Document](doc/jsone.md)
 Benchmark
 ---------
 
-### Environment/Method
-
-- OS: CentOS 6.5
-- CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz (x8)
-- Erlang/OTP: R17.1
-- Benchmark Tool: [erl_json_test](https://github.com/si14/erl_json_test/tree/7ae5a254943ce3e5d9e4d5eb9cd2e86b92ce8e83)
-- CompileOption: [native, {hipe, [o3]}]
-
-### Target
-
-- [jiffy-0.11.3](https://github.com/davisp/jiffy/tree/0.11.3) (NIF)
-- [jsone-v0.2.3](https://github.com/sile/jsone/tree/0.2.3-hipe) (HiPE)
-- [jsonerl](https://github.com/lambder/jsonerl/tree/9720df66052dfc66c9935d954061eda56f81a6f2) (HiPE)
-- [jsonx](https://github.com/iskra/jsonx/tree/9c95948c6835827ed61a9506ae4a9aba61acf335) (NIF)
-- [jsx-v2.0.1](https://github.com/talentdeficit/jsx/tree/v2.0.1) (HiPE)
-- [mochijson2](https://github.com/bjnortier/mochijson2/tree/3663fb01fd98958181adc2d1300c7bfa553e1434) (HiPE)
-- [yawsjson2](https://github.com/spawnproc/yawsjson2/tree/863b7476b4bf7615b578316670c3bd7f04e0048f) (HiPE)
-
-### Decode Result
-
-- column: module name
-- row: data size (bytes)
-- cell: elapsed time (micro seconds)
-
-|                   | jiffy | jsone | jsonerl | jsonx |  jsx  | mochijson2 | yawsjson2 |
-|------------------:|------:|------:|--------:|------:|------:|-----------:|----------:|
-| 559 (1x)          | 12    | 11    | 61      | 7     | 50    | 28         | 37        |
-| 1583 (3x)         | 24    | 25    | 66      | 15    | 134   | 68         | 84        |
-| 4637 (9x)         | 65    | 86    | 178     | 36    | 410   | 186        | 311       |
-| 13914 (27x)       | 189   | 271   | 533     | 109   | 1466  | 550        | 582       |
-| 41542 (81x)       | 525   | 813   | 1578    | 299   | 4684  | 1599       | 1939      |
-| 124726 (243x)     | 1549  | 2406  | 4709    | 852   | 14562 | 4799       | 6123      |
-
-### Encode Result
-
-- column: module name
-- row: data size (bytes)
-- cell: elapsed time (micro seconds)
-
-|                   | jiffy | jsone | jsonerl | jsonx |  jsx  | mochijson2 | yawsjson2 |
-|------------------:|------:|------:|--------:|------:|------:|-----------:|----------:|
-| 559 (1x)          | 14    | 19    | 21      | 8     | 83    | 19         | 15        |
-| 1583 (3x)         | 29    | 49    | 65      | 14    | 228   | 61         | 42        |
-| 4637 (9x)         | 77    | 133   | 229     | 36    | 638   | 225        | 161       |
-| 13914 (27x)       | 215   | 393   | 737     | 101   | 1993  | 664        | 435       |
-| 41542 (81x)       | 621   | 1172  | 2058    | 300   | 6237  | 2310       | 1192      |
-| 124726 (243x)     | 1830  | 3968  | 5842    | 828   | 17032 | 6979       | 5266      |
+The results of [poison](https://github.com/devinus/poison) benchmarking.
+
+See the [BENCHMARK.md](BENCHMARK.md) file for more information.
+
+### EncoderBench Result
+
+| Non HiPE         | jiffy        | jsone             | poison        | jazz          | jsx           |
+|:-----------------|-------------:|------------------:|--------------:|--------------:|--------------:|
+| maps             |   7.35 μs/op |   10.00 μs/op (2) |   13.26 μs/op |   18.97 μs/op |   30.03 μs/op |
+| lists            | 214.93 μs/op |  151.70 μs/op (2) |  114.34 μs/op |  197.31 μs/op |  337.25 μs/op |
+| strings*         | 103.77 μs/op |  550.78 μs/op (5) |  350.49 μs/op |  445.60 μs/op |  237.08 μs/op |
+| string escaping* | 141.46 μs/op |  934.83 μs/op (2) | 1387.10 μs/op | 1568.84 μs/op | 1371.11 μs/op |
+| large value**    | 404.95 μs/op | 1379.04 μs/op (3) | 1322.36 μs/op | 1818.49 μs/op | 2027.41 μs/op |
+| pretty print**   | 415.19 μs/op | 1734.20 μs/op (3) | 1453.36 μs/op | 2031.73 μs/op | 5223.72 μs/op |
+
+| HiPE             | jiffy        | jsone             | poison        | jazz          | jsx           |
+|:-----------------|-------------:|------------------:|--------------:|--------------:|--------------:|
+| maps             |   7.12 μs/op |    5.58 μs/op (1) |   10.71 μs/op |   21.08 μs/op |   25.38 μs/op |
+| lists            | 198.89 μs/op |   64.76 μs/op (1) |   76.80 μs/op |  217.64 μs/op |  220.85 μs/op |
+| strings*         | 109.11 μs/op |  323.63 μs/op (4) |  140.58 μs/op |  363.87 μs/op |  179.35 μs/op |
+| string escaping* | 140.35 μs/op |  481.52 μs/op (2) | 1222.81 μs/op | 1312.68 μs/op |  699.05 μs/op |
+| large value**    | 388.06 μs/op |  634.06 μs/op (2) |  744.13 μs/op | 1572.41 μs/op | 1667.37 μs/op |
+| pretty print**   | 404.84 μs/op |  956.29 μs/op (3) |  802.29 μs/op | 1795.51 μs/op | 4434.13 μs/op |
+
+* binary representation of [UTF-8-demo.txt](https://github.com/devinus/poison/blob/2.1.0/bench/data/UTF-8-demo.txt)
+** [generated.json](https://github.com/devinus/poison/blob/2.1.0/bench/data/generated.json)
+
+### ParserBench Result
+
+| Non HiPE           | jiffy        | jsone             | poison        | jsx           |
+|:-------------------|-------------:|------------------:|--------------:|--------------:|
+| json value*        | 522.25 μs/op | 1217.44 μs/op (2) | 1223.37 μs/op | 1630.77 μs/op |
+| UTF-8 unescaping** |  59.63 μs/op |  342.50 μs/op (4) |  219.79 μs/op |  243.64 μs/op |
+
+| HiPE               | jiffy        | jsone             | poison        | jsx           |
+|:-------------------|-------------:|------------------:|--------------:|--------------:|
+| json value*        | 504.77 μs/op |  527.77 μs/op (2) |  686.90 μs/op | 1320.74 μs/op |
+| UTF-8 unescaping** |  59.95 μs/op |   83.79 μs/op (2) |  109.70 μs/op |  159.35 μs/op |
+
+* [generated.json](https://github.com/devinus/poison/blob/2.1.0/bench/data/generated.json)
+** [UTF-8-demo.txt](https://github.com/devinus/poison/blob/2.1.0/bench/data/UTF-8-demo.txt)
+
+License
+-------
+
+This library is released under the MIT License.
+
+See the [COPYING](COPYING) file for full license information.