Skip to content

Commit 3227e05

Browse files
authored
formatter_json: use JSON as fallback parser instead of Yajl for performance (#4835)
**Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: Recently, Ruby's json has incredible performance improvements. It might be faster than oj gem. So, I think json is a suitable as fallback. This is similar with #4813 * Before * It spent 70.708872503 sec to handle 10 GB file * After * It spent 54.428017716 sec to handle 10 GB file * config ``` <source> @type tail path "#{File.expand_path('~/tmp/fluentd/access*.log')}" pos_file "#{File.expand_path('~/tmp/fluentd/access.log.pos')}" tag log read_from_head true <parse> @type none </parse> </source> <match **> @type file path "#{File.expand_path('~/tmp/fluentd/output/log')}" # use formatter_json format json </match> ``` **Docs Changes**: **Release Note**: --------- Signed-off-by: Shizuo Fujita <[email protected]>
1 parent 644c921 commit 3227e05

File tree

2 files changed

+4
-4
lines changed

2 files changed

+4
-4
lines changed

lib/fluent/plugin/formatter_json.rb

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,11 +34,11 @@ def configure(conf)
3434
if Fluent::OjOptions.available?
3535
@dump_proc = Oj.method(:dump)
3636
else
37-
log.info "Oj isn't installed, fallback to Yajl as json parser"
38-
@dump_proc = Yajl.method(:dump)
37+
log.info "Oj isn't installed, fallback to JSON as json parser"
38+
@dump_proc = JSON.method(:generate)
3939
end
4040
else
41-
@dump_proc = Yajl.method(:dump)
41+
@dump_proc = JSON.method(:generate)
4242
end
4343

4444
# format json is used on various highload environment, so re-define method to skip if check

test/plugin/test_out_stdout.rb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -80,7 +80,7 @@ def create_driver(conf = CONFIG)
8080

8181
if data == 'yajl'
8282
# NOTE: Float::NAN is not jsonable
83-
assert_raise(Yajl::EncodeError) { d.feed('test', time, {'test' => Float::NAN}) }
83+
assert_raise(JSON::GeneratorError) { d.feed('test', time, {'test' => Float::NAN}) }
8484
else
8585
out = capture_log { d.feed('test', time, {'test' => Float::NAN}) }
8686
assert_equal "#{Time.at(time).localtime.strftime(TIME_FORMAT)} test: {\"test\":NaN}\n", out

0 commit comments

Comments
 (0)