GSets cannot be created until other items are

[Bob-The-Marauder]

If you have a brand new cluster with no data whatsoever, gsets cannot be created. This can currently be worked around by creating a set first and then the gset. Example underneath where I fail multiple times to create a gset, successfully create a set and then create my gset with the exact same command that was failing earlier:-

[root@localhost ~]# riak-admin bucket-type create gsets '{"props":{"datatype":"gset"}}'
RPC to 'riak@127.0.0.1' failed: {'EXIT',
                                 {badarg,
                                  [{erlang,binary_to_existing_atom,
                                    [<<"gset">>,utf8],
                                    []},
                                   {riak_kv_wm_utils,erlify_bucket_prop,1,
                                    [{file,"src/riak_kv_wm_utils.erl"},
                                     {line,416}]},
                                   {riak_kv_console,
                                    '-bucket_type_create/2-lc$^0/1-0-',1,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {riak_kv_console,bucket_type_create,2,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {rpc,'-handle_call_call/6-fun-0-',5,
                                    [{file,"rpc.erl"},{line,205}]}]}}
[root@localhost ~]# riak-admin bucket-type create gsets '{"props":{"datatype":"gset"}}'
RPC to 'riak@127.0.0.1' failed: {'EXIT',
                                 {badarg,
                                  [{erlang,binary_to_existing_atom,
                                    [<<"gset">>,utf8],
                                    []},
                                   {riak_kv_wm_utils,erlify_bucket_prop,1,
                                    [{file,"src/riak_kv_wm_utils.erl"},
                                     {line,416}]},
                                   {riak_kv_console,
                                    '-bucket_type_create/2-lc$^0/1-0-',1,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {riak_kv_console,bucket_type_create,2,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {rpc,'-handle_call_call/6-fun-0-',5,
                                    [{file,"rpc.erl"},{line,205}]}]}}
[root@localhost ~]# riak-admin bucket-type create gsets '{"props":{"datatype":"gset"}}'
RPC to 'riak@127.0.0.1' failed: {'EXIT',
                                 {badarg,
                                  [{erlang,binary_to_existing_atom,
                                    [<<"gset">>,utf8],
                                    []},
                                   {riak_kv_wm_utils,erlify_bucket_prop,1,
                                    [{file,"src/riak_kv_wm_utils.erl"},
                                     {line,416}]},
                                   {riak_kv_console,
                                    '-bucket_type_create/2-lc$^0/1-0-',1,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {riak_kv_console,bucket_type_create,2,
                                    [{file,"src/riak_kv_console.erl"},
                                     {line,510}]},
                                   {rpc,'-handle_call_call/6-fun-0-',5,
                                    [{file,"rpc.erl"},{line,205}]}]}}
[root@localhost ~]# riak-admin bucket-type create sets '{"props":{"datatype":"set"}}'
sets created

WARNING: After activating sets, nodes in this cluster
can no longer be downgraded to a version of Riak prior to 2.0
[root@localhost ~]# riak-admin bucket-type create gsets '{"props":{"datatype":"gset"}}'
gsets created

WARNING: After activating gsets, nodes in this cluster
can no longer be downgraded to a version of Riak prior to 2.0

The above is on CentOS 6 with a vanilla install of Riak KV 2.2.6.

[martinsumner]

@Bob-The-Marauder Did anything happen with this issue? Is it still an issue?

I assume the problem is that gset is not an existing atom until the set bucket type is created ... but I don't know why it isn't, and why creating the set type makes it one.

[martinsumner]

OK, so it appears that atoms that exist in code are only loaded when the module that contains them are loaded - and this won't happen until the code is required.

So we can do this to find the atom table:

AtomList = lists:foldl(fun(N, Acc) -> [binary_to_term(<<131,75,N:24>>)|Acc] end, [], lists:seq(0, erlang:system_info(atom_count) - 1)).

If we start Riak:

length(AtomList).
25392
(riak@127.0.0.1)6> erlang:system_info(atom_count).
25392
(riak@127.0.0.1)7> lists:member(gset, AtomList).
false
(riak@127.0.0.1)8> lists:member(set, AtomList). 
true
(riak@127.0.0.1)9> lists:member(hll, AtomList).
true
(riak@127.0.0.1)10> lists:member(counter, AtomList).
true
(riak@127.0.0.1)11> lists:member(counter, AtomList).
true

So when riak starts the other atoms exist. If we now do this:

riak-admin bucket-type create counters '{"props":{"datatype":"counter"}}'

Then get a new AtomListPC, we find

(riak@127.0.0.1)10> DeltaAtoms = AtomListPC -- AtomList.
['AtomListPC',
 '/Users/martinsumner/dbroot/OpenRiak/riak/_build/default/lib/riak_kv/src/riak_kv_crdt.erl',
 invalid_crdt_binary,'-crdt_stats/3-fun-0-',
 '-fun.merge_value/2-','-merge_value/2-fun-0-',
 '-update_object/3-lc$^0/1-0-',new_datatype,operation,
 precondition_context,mod_map,mdc_crdt_epoch,crdt_version,
 to_record,crdt_from_binary,v1_counter_from_binary,
 invalid_binary,later,lastmod,drop_the_dot,update_object,
 fetch_with_default,get_context,update_crdt,decrement,
 invalid_version,deserialize_crdt,merge_value,merge_contents|...]
(riak@127.0.0.1)11> length(DeltaAtoms).
238
(riak@127.0.0.1)7> lists:member(gset, AtomList).
false

In the delta list we see things like this:

... bucket_type_print_list,bucket_type_list,
 bucket_type_print_reset_result,bucket_type_reset,
 bucket_type_print_update_result,bucket_type_update,
 bucket_type_print_create_result,
 bucket_type_print_activate_result,bucket_type_activate, 
 bucket_type_print_props,bucket_type_print_status,
 bucket_type_raw_props,run_reformat,start_reformat,
 reformat_objects,reformat_indexes,index_reformat_options,
 parse_int,human_time_fmt,aae_tree_status,aae_repair_status,
 aae_exchange_status,aae_status,reload_file,reload_path,
 reload_code,nodes_down,different_owners ...

These are all atoms within the riak_kv_console module, which is called when we create the bucket type.

In order to map a data type to the associated module name, this is in include/riak_kv_types:

-define(EMBEDDED_TYPES, [{map, ?MAP_TYPE}, {set, ?SET_TYPE},
                         {counter, ?EMCNTR_TYPE}, {flag, ?FLAG_TYPE},
                         {register, ?REG_TYPE}]).

So using this map, will cause the atoms within it to be loaded, and the only one not yet loaded is gset.

It is likely that the others are loaded for other reasons. For example just in an erlang shell, outside of Riak:

Eshell V14.2 (press Ctrl+G to abort, type help(). for help)
1> AtomList = lists:foldl(fun(N, Acc) -> [binary_to_term(<<131,75,N:24>>)|Acc] end, [], lists:seq(0, erlang:system_info(atom_count) - 1)).
['-predefined_functions/1-lc$^0/1-0-',
 '-predefined_functions/1-lc$^1/1-1-',
 '-predefined_functions/1-lc$^2/1-2-',
 '-predefined_functions/1-lc$^3/1-3-',
 '-get_optional_callbacks/1-lc$^0/1-0-',
 '-module_predef_func_beh_info/2-lc$^0/1-0-',
 module_predef_funcs_mod_info,make_list,
 module_predef_func_beh_info,get_optional_callbacks,
 predefined_functions,add_predefined_functions,pos_integer,
 nonempty_string,nonempty_maybe_improper_list,
 nonempty_improper_list,nonempty_bitstring,nonempty_binary,
 non_neg_integer,neg_integer,maybe_improper_list,iolist,
 identifier,byte,bitsize,send_op,list_op,comp_op,bool_op|...]
2> lists:member(set, AtomList).
true

[martinsumner]

So there is no mystery. But clearly there is a need to take care when using binary_to_existing_atom/1, that the atoms we expect to exist have already been loaded.

There should be a simple fix just to put a guard after we check in riak_kv_console, that it is in the map - and then it will be loaded as part of the module being used, and all the possible atoms will be present.