From 03059e0c7be458284dc684f9d95c74992d236753 Mon Sep 17 00:00:00 2001 From: Eric Redmond Date: Fri, 4 Apr 2014 14:08:44 -0700 Subject: [PATCH] Using Search datatype and Upgrading layout fix --- source/languages/en/riak/dev/using/search.md | 36 ++++++++++--- .../en/riak/ops/advanced/configs/search.md | 1 - .../riak/ops/advanced/upgrading-search-2.md | 52 +++++++++---------- 3 files changed, 54 insertions(+), 35 deletions(-) diff --git a/source/languages/en/riak/dev/using/search.md b/source/languages/en/riak/dev/using/search.md index 4f57da2c30..c16a4e5e1e 100644 --- a/source/languages/en/riak/dev/using/search.md +++ b/source/languages/en/riak/dev/using/search.md @@ -87,19 +87,39 @@ to namespace all buckets you create. Bucket types have a lower overhead within the cluster than the default bucket namespace, but require an additional setup step in on commandline. +When creating a new bucket type, you can create a bucket type without +any properties, and set individual buckets to be indexed. + +```bash +riak-admin bucket-type create animals '{"props":{}}' +riak-admin bucket-type activate animals + +curl -XPUT "$RIAK_HOST/types/animals/buckets/cats/props" \ + -H'content-type: application/json' \ + -d'{"props":{"search_index":"people"}}' +``` + +Or optionally, you can set the `search_index` as a default property +of the bucket type. This means any bucket under that type will +inherit that setting and be its values will be indexed. + ```bash riak-admin bucket-type create animals '{"props":{"search_index":"famous"}}' riak-admin bucket-type activate animals ``` +If you ever need to turn off indexing for a bucket, set `search_index` +property to the `_dont_index_` sentinel. + ### Bucket Properties -If your Solr index is to be used by only one bucket, you can set the -`search_index` property for a bucket in this manner: +Although we recommend all new buckets use under a bucket type, if you have +existing data with a type-free bucket (or under the `default` bucket type) +you can set the `search_index` property for a bucket in this manner. ```curl -curl -XPUT "$RIAK_HOST/buckets/animals/props" \ +curl -XPUT "$RIAK_HOST/buckets/cats/props" \ -H'content-type:application/json' \ -d'{"props":{"search_index":"famous"}}' ``` @@ -149,23 +169,23 @@ curl -XPUT "$RIAK_HOST/types/animals/buckets/cats/keys/panthro" \ -d'{"name_s":"Panthro", "age_i":36}' ``` ```ruby -bucket = client.bucket("animals", "cats") +bucket = client.bucket("animals") cat = bucket.get_or_new("liono") cat.data = {"name_s" => "Lion-o", "age_i" => 30, "leader_b" => true} -cat.store +cat.store(:bucket_type => "cats") cat = bucket.get_or_new("cheetara") cat.data = {"name_s" => "Cheetara", "age_i" => 28, "leader_b" => false} -cat.store +cat.store(:bucket_type => "cats") cat = bucket.get_or_new("snarf") cat.data = {"name_s" => "Snarf", "age_i" => 43} -cat.store +cat.store(:bucket_type => "cats") cat = bucket.get_or_new("panthro") cat.data = {"name_s" => "Panthro", "age_i" => 36} -cat.store +cat.store(:bucket_type => "cats") ``` ```python bucket = client.bucket('animals', 'cats') diff --git a/source/languages/en/riak/ops/advanced/configs/search.md b/source/languages/en/riak/ops/advanced/configs/search.md index 5671ef7cfc..07d614c2b0 100644 --- a/source/languages/en/riak/ops/advanced/configs/search.md +++ b/source/languages/en/riak/ops/advanced/configs/search.md @@ -45,4 +45,3 @@ Riak Search runs a Solr process per node to manage its index and search function Concerning ports, be sure to take the necessary security precautions to prevent exposing the extra solr and JMX ports to the outside world. - diff --git a/source/languages/en/riak/ops/advanced/upgrading-search-2.md b/source/languages/en/riak/ops/advanced/upgrading-search-2.md index 6262ddf70e..c001743ea6 100644 --- a/source/languages/en/riak/ops/advanced/upgrading-search-2.md +++ b/source/languages/en/riak/ops/advanced/upgrading-search-2.md @@ -8,7 +8,7 @@ audience: advanced keywords: [search, upgrading] --- -If you're using Search in a version of Riak prior to 2.0 (1.3.0 to 1.4.x), you should follow these steps to migrate your search indexes from the legacy `merge_index`, to the new Solr backed ([[yokozuna|using riak search]]) indexes. The legacy version of Riak Search is now deprecated, and does not support most new 2.0 features (no [[datatypes]], [[bucket types|using bucket types]], [[strong consistency]], [[security]]), so we highly recommend you migrate. The legacy `merge_index` search will be removed in the future. +If you're using Search in a version of Riak prior to 2.0 (1.3.0 to 1.4.x), you should follow these steps to migrate your search indexes from the legacy `merge_index`, to the new Solr backed ([[yokozuna|using search]]) indexes. The legacy version of Riak Search is now deprecated, and does not support most new 2.0 features (no [[datatypes|using data types]], [[bucket types|using bucket types]], [[strong consistency]], [[security|authentication and authorization]]), so we highly recommend you migrate. The legacy `merge_index` search will be removed in the future. ## Overview of an Upgrade @@ -22,50 +22,50 @@ You will continue running your queries against Riak Search in this phase. Once n ## Steps to Upgrading -1. First, you'll perform a normal [[rolling upgrade]]. As you upgrade, enable Yokozuna on each node using either `app.config` or the new `riak.conf` config option. +1. First, you'll perform a normal [[rolling upgrade|rolling upgrades]]. As you upgrade, enable Yokozuna on each node using either `app.config` or the new `riak.conf` config option. -```appconfig - {yokozuna, [ - {enabled, on} - ]}, -``` -```riakconf -search = on -``` + ```appconfig + {yokozuna, [ + {enabled, on} + ]}, + ``` + ```riakconf + search = on + ``` 2. For every index in Riak Search the user must create a comparable index in Yokozuna. -```curl -# Run for each Riak node + ```curl + # Run for each Riak node -export RIAK_HOST="http://localhost:8098" + export RIAK_HOST="http://localhost:8098" -curl -XPUT "$RIAK_HOST/search/index/famous" \ - -H'content-type:application/json' \ - -d'{"schema":"_yz_default"}' -``` -```erlang -{ok, Ring} = riak_core_ring_manager:get_my_ring(). -{ok, PB} = riakc_pb_socket:start_link(IP, Port), -riakc_pb_socket:create_search_index(Pid, <<"famous">>, <<"_yz_default">>, []), -``` + curl -XPUT "$RIAK_HOST/search/index/famous" \ + -H'content-type:application/json' \ + -d'{"schema":"_yz_default"}' + ``` + ```erlang + {ok, Ring} = riak_core_ring_manager:get_my_ring(). + {ok, PB} = riakc_pb_socket:start_link(IP, Port), + riakc_pb_socket:create_search_index(Pid, <<"famous">>, <<"_yz_default">>, []), + ``` 3. For every bucket which is indexed by Riak Search the user must add the `search_index' bucket property to point to the Yokozuna index which is going to eventually be migrated to. 4. As objects are written or modified they will be indexed by both Riak Search and Yokozuna. But the HTTP and PB query interfaces will continue to use Riak Search. -5a. The YZ AAE trees must be manually cleared so that AAE will notice the missing indexes. +5. A. The YZ AAE trees must be manually cleared so that AAE will notice the missing indexes. -5b. In the background AAE will start building trees for Yokozuna and exchange them with KV. These exchanges will notice objects are missing and index them in Yokozuna. + B. In the background AAE will start building trees for Yokozuna and exchange them with KV. These exchanges will notice objects are missing and index them in Yokozuna. -5c. The user wants Yokozuna to index the missing objects as fast as possible. A command may be used (repair? bucket map-reduce? custom fold function?) to immediately re-index data. + C. The user wants Yokozuna to index the missing objects as fast as possible. A command may be used (repair? bucket map-reduce? custom fold function?) to immediately re-index data. 6. Eventually all partitions will be exchanged (or buckets re-indexed) and the user will be satisfied that queries can now migrate to Yokozuna. This will be accomplished via the AAE status. 7. The user will call some command that hands HTTP and PB query control to Yokozuna. -8. The user must then set the `search' bucket property to `false' for all indexed buckets. +8. The user must then set the `search` bucket property to `false` for all indexed buckets. 9. Then the user can disable Riak Search on all nodes.