Skip to content

Commit a394e3f

Browse files
committed
router: introduce map_callrw()
Closes #147 @TarantoolBot document Title: vshard.router.map_callrw() `vshard.router.map_callrw()` implements consistent map-reduce over the entire cluster. Consistency means all the data was accessible, and didn't move during map requests execution. It is useful when need to access potentially all the data in the cluster or simply huge number of buckets scattered over the instances and whose individual `vshard.router.call()` would take too long. `Map_callrw()` takes name of the function to call on the storages, arguments in the format of array, and not required options map. The only supported option for now is timeout which is applied to the entire call. Not to individual calls for each storage. ``` vshard.router.map_callrw(func_name, args[, {timeout = <seconds>}]) ``` The chosen function is called on the master node of each replicaset with the given arguments. In case of success `vshard.router.map_callrw()` returns a map with replicaset UUIDs as keys and results of the user's function as values, like this: ``` {uuid1 = {res1}, uuid2 = {res2}, ...} ``` If the function returned `nil` or `box.NULL` from one of the storages, it won't be present in the result map. In case of fail it returns nil, error object, and optional replicaset UUID where the error happened. UUID may not be returned if the error wasn't about a concrete replicaset. For instance, the method fails if not all buckets were found even if all replicasets were scanned successfully. Handling the result looks like this: ```Lua res, err, uuid = vshard.router.map_callrw(...) if not res then -- Error. -- 'err' - error object. 'uuid' - optional UUID of replicaset -- where the error happened. ... else -- Success. for uuid, value in pairs(res) do ... end end ``` Map-Reduce in vshard works in 3 stages: Ref, Map, Reduce. Ref is an internal stage which is supposed to ensure data consistency during user's function execution on all nodes. Reduce is not performed by vshard. It is what user's code does with results of `map_callrw()`. Consistency, as it is defined for map-reduce, is not compatible with rebalancing. Because any bucket move would make the sender and receiver nodes 'inconsistent' - it is not possible to call a function on them which could simply access all the data without doing `vshard.storage.bucket_ref()`. This makes Ref stage very intricate as it must work together with rebalancer to ensure neither of them block each other. For this storage has a scheduler specifically for bucket moves and storage refs which shares storage time between them fairly. Definition of fairness depends on how long and frequent the moves and refs are. This can be configured using storage options `sched_move_quota` and `sched_ref_quota`. See more details about them in the corresponding doc section. The scheduler configuration may affect map-reduce requests if they are used a lot during rebalancing. Keep in mind that it is not a good idea to use too big timeouts for `map_callrw()`. Because the router will try to block the bucket moves for the given timeout on all storages. And in case something will go wrong, the block will remain for the entire timeout. This means, in particular, having the timeout longer than, say, minutes is a super bad way to go unless it is for tests only. Also it is important to remember that `map_callrw()` does not work on replicas. It works only on masters. This makes it unusable if at least one replicaset has its master node down.
1 parent 17b6d28 commit a394e3f

File tree

7 files changed

+1164
-5
lines changed

7 files changed

+1164
-5
lines changed

0 commit comments

Comments
 (0)