fix: Fix the task leak with the lazy in-mem rpc client while still keeping it lazy

fix: Fix the task leak with the lazy in-mem rpc client while still keeping it lazy #91