configure mem pool buckets rotation #261
Conversation
HagarMeir
force-pushed
the
configure-mem-pool-buckets-rotation
branch
4 times, most recently
from
18c714e to
16d302f
Compare
request/pool.go
Outdated
| rp.semaphore.Release(1) | ||
| }, | ||
| Epoch: time.Second, | ||
| Epoch: time.Duration(rand.Intn(50))*time.Millisecond + rp.options.FirstStrikeThreshold/10, |
There was a problem hiding this comment.
Now the mean of the epoch is longer than FirstStrikeThreshold / 10; in fact it is FirstStrikeThreshold / 10 +25ms. I don't think this is the right approach.
Draw a random number d whose range is [-D,D]. Then add the "dither" d when you compute if a first strike had happened, e.g. in here:
func (ps *PendingStore) checkFirstStrike(now time.Time)
if now.Sub(bucket.lastTimestamp) <= ps.FirstStrikeThreshold { <<<
continue
}
by adding d to the ps.FirstStrikeThreshold.
In any event, this is not expected to help much if d is much smaller than the batching interval. If the first TXs forwarded to the primary (by the secondary who strikes first) don't return in a batch by the time the next secondary batcher strikes, they will be forwarded anyway....
It seems to me a good choice for D would be around 2*BatchInterval or 3*BatchInterval.... what do you think?
HagarMeir
force-pushed
the
configure-mem-pool-buckets-rotation
branch
4 times, most recently
from
0928df9 to
4c92ff7
Compare
| Logger: rp.logger, | ||
| SecondStrikeThreshold: rp.options.SecondStrikeThreshold, | ||
| FirstStrikeThreshold: rp.options.FirstStrikeThreshold, | ||
| FirstStrikeThreshold: rp.options.FirstStrikeThreshold + rp.addRandomnessToFirstStrike(), |
There was a problem hiding this comment.
This is adding randomness only once, when the pool is created. Wouldn't it make more sense to do it per bucket, or would that be too much? what do you think?
There was a problem hiding this comment.
I think it will be too much
| rp.pending.Start() | ||
| } | ||
|
|
||
| func (rp *Pool) addRandomnessToFirstStrike() time.Duration { |
There was a problem hiding this comment.
Lets make sure that in the worse case the first strike is still > BatchTimeout. IE compare the BatchTimeout with the FirstStrikeThreshold. It seems to me the FirstStrikeThreshold should be at least 5*BatchTimeout or so...
Even without randomization, if FirstStrikeThreshold < BatchTimeout + network-delay we'll get constant forwarding. So some sanity checks are required.
There was a problem hiding this comment.
Will add checks
Currently the batch timeout is 500ms and we are running tests with first strike of 2s
Signed-off-by: Hagar Meir <[email protected]>
Signed-off-by: Hagar Meir <[email protected]>
Signed-off-by: Hagar Meir <[email protected]>
Signed-off-by: Hagar Meir <[email protected]>
HagarMeir
force-pushed
the
configure-mem-pool-buckets-rotation
branch
from
4c92ff7 to
f00a6a5
Compare
2 participants
#269