-
Notifications
You must be signed in to change notification settings - Fork 30
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Use Case
Deterministic tasks that perform the same computation again and again would benefit from caching mechanisms.
Caching behaviour could be configured via an extra task
parameter. For example:
from pyper import task, CachePolicy
def extremely_expensive_calculation(x: int):
return x + 1
pipeline = task(
extremely_expensive_calculation,
multiprocess=True,
cache_policy=CachePolicy(maxsize=100, ...)
)
Alternatives
Currently, functools.cache
and functools.lru_cache
can be used for normal synchronous functions. However, this approach won't straightforwardly extend to asynchronous tasks and multiprocessed tasks. There is a need for a unified caching API across all modes of execution.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request