Skip to content

Update install-cachelib-deps.sh to rebuild Docker #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 29 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
4a95c4b
Run centos and debian workflows on push and PR
igchor Nov 2, 2021
7b01549
Adds createPutToken and switches findEviction
byrnedj Feb 4, 2023
36347e2
Add memory usage statistics for allocation classes
igchor Jul 6, 2022
cdf2522
Initial multi-tier support implementation (rebased with NUMA and cs p…
igchor Sep 28, 2021
19c6828
AC stats multi-tier
byrnedj Jan 17, 2023
67b0d0b
This commit contains the additional memory tiers tests
byrnedj Feb 8, 2023
6d521d6
This is the additional multi-tier support needed
guptask Nov 14, 2022
c33686e
added per pool class rolling average latency (upstream PR version)
guptask Jul 21, 2022
ea33665
added per tier pool class rolling average latency (based on upstream PR)
guptask Jul 21, 2022
1ded485
MM2Q promotion iterators (#1)
byrnedj Aug 9, 2022
2727c05
CS Patch Part 2 for mulit-tier cachelib:
byrnedj Feb 7, 2023
e1d1840
basic multi-tier test based on numa bindings
igchor Dec 30, 2021
5a7db9d
Aadding new configs to hit_ratio/graph_cache_leader_fobj
vinser52 Jan 27, 2022
e7b3709
Background data movement (#20)
byrnedj Oct 21, 2022
3ab0551
fix race in moveRegularItemWith sync where insertOrReplace can cause …
byrnedj Feb 16, 2023
bfe54ad
Per tier pool stats (#70)
byrnedj Mar 23, 2023
3b23c2c
dummy change to trigger container image rebuild
guptask Mar 28, 2023
77fe443
Fix token creation and stats (#79)
igchor Apr 27, 2023
2377c45
Updated the docker gcc version to 12 (#83)
guptask May 9, 2023
c6ae3f0
NUMA bindigs support for private memory (#82)
vinser52 May 17, 2023
344a74c
Do not run cachelib-centos-8-5 on PRs (#85)
igchor Jun 6, 2023
f6528c4
correct handling for expired items in eviction (#86)
byrnedj Jun 6, 2023
7422f02
Add option to insert items to first free tier (#87)
igchor Jun 8, 2023
88dbb56
Chained item movement between tiers - sync on the parent item (#84)
byrnedj Jun 28, 2023
d2c6b3b
edit dockerfile
byrnedj Jul 24, 2023
abe622a
Track latency of per item eviction/promotion between memory tiers
guptask Jul 28, 2023
139e2e9
Update dependencies (#95)
igchor Aug 23, 2023
c398ee8
Update install-cachelib-deps.sh to rebuild Docker
vinser52 Mar 1, 2024
12ff11f
Force docker rebuild
vinser52 Mar 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .github/workflows/build-cachelib-centos-8-5.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,6 @@
# limitations under the License.
name: build-cachelib-centos-8.5
on:
push:
tags:
- 'v*'
pull_request:
workflow_dispatch:
schedule:
- cron: '0 9 * * *'
jobs:
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/build-cachelib-centos-long.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: build-cachelib-centos-latest
on:
schedule:
- cron: '0 7 * * *'

jobs:
build-cachelib-centos8-latest:
name: "CentOS/latest - Build CacheLib with all dependencies"
runs-on: ubuntu-latest
# Docker container image name
container: "centos:latest"
steps:
- name: "update packages"
run: dnf upgrade -y
- name: "install sudo,git"
run: dnf install -y sudo git cmake gcc
- name: "System Information"
run: |
echo === uname ===
uname -a
echo === /etc/os-release ===
cat /etc/os-release
echo === df -hl ===
df -hl
echo === free -h ===
free -h
echo === top ===
top -b -n1 -1 -Eg || timeout 1 top -b -n1
echo === env ===
env
echo === gcc -v ===
gcc -v
- name: "checkout sources"
uses: actions/checkout@v2
- name: "build CacheLib using build script"
run: ./contrib/build.sh -j -v -T
- name: "run tests"
timeout-minutes: 60
run: cd opt/cachelib/tests && ../../../run_tests.sh long
43 changes: 43 additions & 0 deletions .github/workflows/build-cachelib-debian.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: build-cachelib-debian-10
on:
schedule:
- cron: '30 5 * * 0,3'

jobs:
build-cachelib-debian-10:
name: "Debian/Buster - Build CacheLib with all dependencies"
runs-on: ubuntu-latest
# Docker container image name
container: "debian:buster-slim"
steps:
- name: "update packages"
run: apt-get update
- name: "upgrade packages"
run: apt-get -y upgrade
- name: "install sudo,git"
run: apt-get install -y sudo git procps
- name: "System Information"
run: |
echo === uname ===
uname -a
echo === /etc/os-release ===
cat /etc/os-release
echo === df -hl ===
df -hl
echo === free -h ===
free -h
echo === top ===
top -b -n1 -1 -Eg || timeout 1 top -b -n1 ; true
echo === env ===
env
echo === cc -v ===
cc -v || true
echo === g++ -v ===
g++ - || true
- name: "checkout sources"
uses: actions/checkout@v2
- name: "build CacheLib using build script"
run: ./contrib/build.sh -j -v -T
- name: "run tests"
timeout-minutes: 60
run: cd opt/cachelib/tests && ../../../run_tests.sh
50 changes: 50 additions & 0 deletions .github/workflows/build-cachelib-docker.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: build-cachelib-docker
on:
push:
pull_request:

jobs:
build-cachelib-docker:
name: "CentOS/latest - Build CacheLib with all dependencies"
runs-on: ubuntu-latest
env:
REPO: cachelib
GITHUB_REPO: intel/CacheLib
CONTAINER_REG: ghcr.io/pmem/cachelib
CONTAINER_REG_USER: ${{ secrets.GH_CR_USER }}
CONTAINER_REG_PASS: ${{ secrets.GH_CR_PAT }}
FORCE_IMAGE_ACTION: ${{ secrets.FORCE_IMAGE_ACTION }}
HOST_WORKDIR: ${{ github.workspace }}
WORKDIR: docker
IMG_VER: devel
strategy:
matrix:
CONFIG: ["OS=centos OS_VER=8streams PUSH_IMAGE=1"]
steps:
- name: "System Information"
run: |
echo === uname ===
uname -a
echo === /etc/os-release ===
cat /etc/os-release
echo === df -hl ===
df -hl
echo === free -h ===
free -h
echo === top ===
top -b -n1 -1 -Eg || timeout 1 top -b -n1
echo === env ===
env
echo === gcc -v ===
gcc -v
- name: "checkout sources"
uses: actions/checkout@v2
with:
submodules: recursive
fetch-depth: 0

- name: Pull the image or rebuild and push it
run: cd $WORKDIR && ${{ matrix.CONFIG }} ./pull-or-rebuild-image.sh rebuild

- name: Run the build
run: cd $WORKDIR && ${{ matrix.CONFIG }} ./build.sh
90 changes: 90 additions & 0 deletions MultiTierDataMovement.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# Background Data Movement

In order to reduce the number of online evictions and support asynchronous
promotion - we have added two periodic workers to handle eviction and promotion.

The diagram below shows a simplified version of how the background evictor
thread (green) is integrated to the CacheLib architecture.

<p align="center">
<img width="640" height="360" alt="BackgroundEvictor" src="cachelib-background-evictor.png">
</p>

## Background Evictors

The background evictors scan each class to see if there are objects to move the next (lower)
tier using a given strategy. Here we document the parameters for the different
strategies and general parameters.

- `backgroundEvictorIntervalMilSec`: The interval that this thread runs for - by default
the background evictor threads will wake up every 10 ms to scan the AllocationClasses. Also,
the background evictor thread will be woken up everytime there is a failed allocation (from
a request handling thread) and the current percentage of free memory for the
AllocationClass is lower than `lowEvictionAcWatermark`. This may render the interval parameter
not as important when there are many allocations occuring from request handling threads.

- `evictorThreads`: The number of background evictors to run - each thread is a assigned
a set of AllocationClasses to scan and evict objects from. Currently, each thread gets
an equal number of classes to scan - but as object size distribution may be unequal - future
versions will attempt to balance the classes among threads. The range is 1 to number of AllocationClasses.
The default is 1.

- `maxEvictionBatch`: The number of objects to remove in a given eviction call. The
default is 40. Lower range is 10 and the upper range is 1000. Too low and we might not
remove objects at a reasonable rate, too high and it might increase contention with user threads.

- `minEvictionBatch`: Minimum number of items to evict at any time (if there are any
candidates)

- `maxEvictionPromotionHotness`: Maximum candidates to consider for eviction. This is similar to `maxEvictionBatch`
but it specifies how many candidates will be taken into consideration, not the actual number of items to evict.
This option can be used to configure duration of critical section on LRU lock.


### FreeThresholdStrategy (default)

- `lowEvictionAcWatermark`: Triggers background eviction thread to run
when this percentage of the AllocationClass is free.
The default is `2.0`, to avoid wasting capacity we don't set this above `10.0`.

- `highEvictionAcWatermark`: Stop the evictions from an AllocationClass when this
percentage of the AllocationClass is free. The default is `5.0`, to avoid wasting capacity we
don't set this above `10`.


## Background Promoters

The background promoters scan each class to see if there are objects to move to a lower
tier using a given strategy. Here we document the parameters for the different
strategies and general parameters.

- `backgroundPromoterIntervalMilSec`: The interval that this thread runs for - by default
the background promoter threads will wake up every 10 ms to scan the AllocationClasses for
objects to promote.

- `promoterThreads`: The number of background promoters to run - each thread is a assigned
a set of AllocationClasses to scan and promote objects from. Currently, each thread gets
an equal number of classes to scan - but as object size distribution may be unequal - future
versions will attempt to balance the classes among threads. The range is `1` to number of AllocationClasses. The default is `1`.

- `maxProtmotionBatch`: The number of objects to promote in a given promotion call. The
default is 40. Lower range is 10 and the upper range is 1000. Too low and we might not
remove objects at a reasonable rate, too high and it might increase contention with user threads.

- `minPromotionBatch`: Minimum number of items to promote at any time (if there are any
candidates)

- `numDuplicateElements`: This allows us to promote items that have existing handles (read-only) since
we won't need to modify the data when a user is done with the data. Therefore, for a short time
the data could reside in both tiers until it is evicted from its current tier. The default is to
not allow this (0). Setting the value to 100 will enable duplicate elements in tiers.

### Background Promotion Strategy (only one currently)

- `promotionAcWatermark`: Promote items if there is at least this
percent of free AllocationClasses. Promotion thread will attempt to move `maxPromotionBatch` number of objects
to that tier. The objects are chosen from the head of the LRU. The default is `4.0`.
This value should correlate with `lowEvictionAcWatermark`, `highEvictionAcWatermark`, `minAcAllocationWatermark`, `maxAcAllocationWatermark`.
- `maxPromotionBatch`: The number of objects to promote in batch during BG promotion. Analogous to
`maxEvictionBatch`. It's value should be lower to decrease contention on hot items.

5 changes: 5 additions & 0 deletions cachelib/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,11 @@ set(CMAKE_MODULE_PATH
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)

if(COVERAGE_ENABLED)
# Add code coverage
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} --coverage -fprofile-arcs -ftest-coverage")
endif()

# include(fb_cxx_flags)
message(STATUS "Update CXXFLAGS: ${CMAKE_CXX_FLAGS}")

Expand Down
27 changes: 16 additions & 11 deletions cachelib/allocator/BackgroundMover-inl.h
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,8 @@ template <typename CacheT>
void BackgroundMover<CacheT>::setAssignedMemory(
std::vector<MemoryDescriptorType>&& assignedMemory) {
XLOG(INFO, "Class assigned to background worker:");
for (auto [pid, cid] : assignedMemory) {
XLOGF(INFO, "Pid: {}, Cid: {}", pid, cid);
for (auto [tid, pid, cid] : assignedMemory) {
XLOGF(INFO, "Tid: {}, Pid: {}, Cid: {}", tid, pid, cid);
}

mutex_.lock_combine([this, &assignedMemory] {
Expand All @@ -65,25 +65,28 @@ void BackgroundMover<CacheT>::checkAndRun() {
auto assignedMemory = mutex_.lock_combine([this] { return assignedMemory_; });

unsigned int moves = 0;
std::set<ClassId> classes{};
auto batches = strategy_->calculateBatchSizes(cache_, assignedMemory);

for (size_t i = 0; i < batches.size(); i++) {
const auto [pid, cid] = assignedMemory[i];
const auto [tid, pid, cid] = assignedMemory[i];
const auto batch = batches[i];

if (batch == 0) {
continue;
}

classes.insert(cid);
const auto& mpStats = cache_.getPoolByTid(pid, tid).getStats();
// try moving BATCH items from the class in order to reach free target
auto moved = moverFunc(cache_, pid, cid, batch);
auto moved = moverFunc(cache_, tid, pid, cid, batch);
moves += moved;
movesPerClass_[pid][cid] += moved;
totalBytesMoved_.add(moved * cache_.getPool(pid).getAllocSizes()[cid]);
moves_per_class_[tid][pid][cid] += moved;
totalBytesMoved_.add(moved * mpStats.acStats.at(cid).allocSize );
}

numTraversals_.inc();
numMovedItems_.add(moves);
totalClasses_.add(classes.size());
}

template <typename CacheT>
Expand All @@ -92,24 +95,26 @@ BackgroundMoverStats BackgroundMover<CacheT>::getStats() const noexcept {
stats.numMovedItems = numMovedItems_.get();
stats.runCount = numTraversals_.get();
stats.totalBytesMoved = totalBytesMoved_.get();
stats.totalClasses = totalClasses_.get();

return stats;
}

template <typename CacheT>
std::map<PoolId, std::map<ClassId, uint64_t>>
std::map<TierId, std::map<PoolId, std::map<ClassId, uint64_t>>>
BackgroundMover<CacheT>::getClassStats() const noexcept {
return movesPerClass_;
return moves_per_class_;
}

template <typename CacheT>
size_t BackgroundMover<CacheT>::workerId(PoolId pid,
size_t BackgroundMover<CacheT>::workerId(TierId tid,
PoolId pid,
ClassId cid,
size_t numWorkers) {
XDCHECK(numWorkers);

// TODO: came up with some better sharding (use hashing?)
return (pid + cid) % numWorkers;
return (tid + pid + cid) % numWorkers;
}

} // namespace cachelib
Expand Down
19 changes: 13 additions & 6 deletions cachelib/allocator/BackgroundMover.h
Original file line number Diff line number Diff line change
Expand Up @@ -29,17 +29,19 @@ namespace cachelib {
template <typename C>
struct BackgroundMoverAPIWrapper {
static size_t traverseAndEvictItems(C& cache,
unsigned int tid,
unsigned int pid,
unsigned int cid,
size_t batch) {
return cache.traverseAndEvictItems(pid, cid, batch);
return cache.traverseAndEvictItems(tid, pid, cid, batch);
}

static size_t traverseAndPromoteItems(C& cache,
unsigned int tid,
unsigned int pid,
unsigned int cid,
size_t batch) {
return cache.traverseAndPromoteItems(pid, cid, batch);
return cache.traverseAndPromoteItems(tid, pid, cid, batch);
}
};

Expand All @@ -62,31 +64,36 @@ class BackgroundMover : public PeriodicWorker {
~BackgroundMover() override;

BackgroundMoverStats getStats() const noexcept;
std::map<PoolId, std::map<ClassId, uint64_t>> getClassStats() const noexcept;
std::map<TierId, std::map<PoolId, std::map<ClassId, uint64_t>>>
getClassStats() const noexcept;

void setAssignedMemory(std::vector<MemoryDescriptorType>&& assignedMemory);

// return id of the worker responsible for promoting/evicting from particlar
// pool and allocation calss (id is in range [0, numWorkers))
static size_t workerId(PoolId pid, ClassId cid, size_t numWorkers);
static size_t workerId(TierId tid, PoolId pid, ClassId cid, size_t numWorkers);

private:
std::map<PoolId, std::map<ClassId, uint64_t>> movesPerClass_;
std::map<TierId, std::map<PoolId, std::map<ClassId, uint64_t>>>
moves_per_class_;
// cache allocator's interface for evicting
using Item = typename Cache::Item;

Cache& cache_;
std::shared_ptr<BackgroundMoverStrategy> strategy_;
MoverDir direction_;

std::function<size_t(Cache&, unsigned int, unsigned int, size_t)> moverFunc;
std::function<size_t(
Cache&, unsigned int, unsigned int, unsigned int, size_t)>
moverFunc;

// implements the actual logic of running the background evictor
void work() override final;
void checkAndRun();

AtomicCounter numMovedItems_{0};
AtomicCounter numTraversals_{0};
AtomicCounter totalClasses_{0};
AtomicCounter totalBytesMoved_{0};

std::vector<MemoryDescriptorType> assignedMemory_;
Expand Down
4 changes: 3 additions & 1 deletion cachelib/allocator/BackgroundMoverStrategy.h
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@ namespace facebook {
namespace cachelib {

struct MemoryDescriptorType {
MemoryDescriptorType(PoolId pid, ClassId cid) : pid_(pid), cid_(cid) {}
MemoryDescriptorType(TierId tid, PoolId pid, ClassId cid) :
tid_(tid), pid_(pid), cid_(cid) {}
TierId tid_;
PoolId pid_;
ClassId cid_;
};
Expand Down
Loading