Skip to content

Commit 58f86f1

Browse files
pallavigopalpallavigopal
committed
Minor Bug fixes and added sanity tests
Did minor bug fixes to set the initital memory limit Added sanity tests to test the memoryusageanalyzer functionality Added support to analyze mutiple workloads in parallel
1 parent 99d51cb commit 58f86f1

19 files changed

+966
-147
lines changed

‎docs/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
## System Configuration
22

33
Run the command below to configure the system. This config script help to configure
4-
zram, zswap, memory and frequency related kernel parameters
4+
zram, zswap, memory and frequency related kernel parameters. The [config.py](../tests/config_script/config.py) file is located in [tests/config_script](../tests/config_script)
55

66
```shell
77
$ config.py
@@ -389,4 +389,4 @@ DEBUG 2024-03-04 21:26:49,811 : found 1 swap devices
389389
DEBUG 2024-03-04 21:26:49,811 : shell: cat /sys/module/zswap/parameters/compressor
390390
DEBUG 2024-03-04 21:26:49,813 : result: lzo-rle
391391
INFO 2024-03-04 21:26:49,813 : Compressor = lzo-rle
392-
```
392+
```

‎docs/multiple_workloads.md

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
# Analyzing multiple workloads with multiple cgroups in parallel
2+
3+
In some scenarios, workloads are run under different cgroups to allow different resource constraints. This section provides details on how to monitor memory usage with multiple Cgroups running concurrently. multiconfig.json, as provided in the example below, allows monitoring of 2 Cgroups. This can be further extended to more number of Cgrups. In the example below, the first ( memoryusageanalyzer1) run the baseline runs and the second Cgroup (memoryusageanalyzer2)
4+
run the dynamic squeezer
5+
```
6+
multiconfig.json:
7+
[
8+
{
9+
"cmd":"./workload 8 60 1000",
10+
"cgroupname":"memoryusageanalyzer1",
11+
"output":"./profile_workload1",
12+
"reclaimerconfig":""
13+
},
14+
{
15+
"cmd":"./workload 8 60 1000",
16+
"cgroupname":"memoryusageanalyzer2",
17+
"output":"./profile_workload2",
18+
"reclaimerconfig":"../../src/reclaimer/squeezerdynamicconfig.json"
19+
}
20+
]
21+
```
22+
23+
```
24+
memory_usage_analyzer.py -m ./multiconfig.json
25+
#Sample results, actual results may vary
26+
2024-05-22 03:34:12,471 - memoryusageanalyzer - INFO - args = Namespace(verbose=False, output='profile', outputforce=None, sampleperiod=5, docker=None, cgpath='/sys/fs/cgroup', cgname='memoryusageanalyzer', reclaimerconfig=None, multiconfig='multiconfig.json', cmdoptions=[])
27+
cgroup_path_create start
28+
cgroup_path_create CompletedProcess(args='sudo bash /mnt/nvme1/pg3/applications.benchmarking.memory-usage-analyzer/src/core/cgroup_config.sh /sys/fs/cgroup memoryusageanalyzer1 root ', returncode=0)
29+
2024-05-22 03:34:12,492 - memoryusageanalyzer - INFO - **** Storing profiling results in ./profile_workload1.9
30+
2024-05-22 03:34:12,496 - memoryusageanalyzer - INFO - cgroup v2 enabled and memory controller detected
31+
2024-05-22 03:34:12,496 - memoryusageanalyzer - INFO - **** In run Starting
32+
workload start
33+
2024-05-22 03:34:12,500 - memoryusageanalyzer - INFO - **** Starting job in cgroup
34+
2024-05-22 03:34:12,502 - memoryusageanalyzer - INFO - **** Starting stats
35+
cgroup_path_create start
36+
cgroup_path_create CompletedProcess(args='sudo bash /mnt/nvme1/pg3/applications.benchmarking.memory-usage-analyzer/src/core/cgroup_config.sh /sys/fs/cgroup memoryusageanalyzer2 root ', returncode=0)
37+
2024-05-22 03:34:12,529 - memoryusageanalyzer - INFO - **** Storing profiling results in ./profile_workload2.9
38+
2024-05-22 03:34:12,605 - memoryusageanalyzer - INFO - cgroup v2 enabled and memory controller detected
39+
2024-05-22 03:34:12,605 - memoryusageanalyzer - INFO - **** In run Starting
40+
workload start
41+
2024-05-22 03:34:12,608 - memoryusageanalyzer - INFO - **** Starting job in cgroup
42+
2024-05-22 03:34:12,611 - memoryusageanalyzer - INFO - **** Starting stats
43+
2024-05-22 03:34:12,613 - memoryusageanalyzer - INFO - **** Starting reclaimer
44+
memory usage = 864,256 pf_rate = 0.0 => squeeze percent = 1%
45+
[INFO] (main:43) : pid = 7717
46+
[INFO] (main:44) : memory = 8 GiB = 2097152 pages
47+
[INFO] (main:45) : hot timer = 60 sec
48+
[INFO] (main:46) : loops = 1000
49+
squeeze to 855,613 -- ok
50+
[INFO] (main:43) : pid = 7724
51+
[INFO] (main:44) : memory = 8 GiB = 2097152 pages
52+
[INFO] (main:45) : hot timer = 60 sec
53+
[INFO] (main:46) : loops = 1000
54+
[INFO] (main:60) : Done with setup in 3 seconds.
55+
[INFO] (main:60) : Done with setup in 3 seconds.
56+
memory usage = 8,608,489,472 pf_rate = 14.8 => squeeze percent = 2%
57+
squeeze to 8,436,319,682 -- ok
58+
memory usage = 8,607,981,568 pf_rate = 16119.4 => squeeze percent = 0%
59+
memory usage = 8,607,981,568 pf_rate = 0.0 => squeeze percent = 1%
60+
squeeze to 8,521,901,752 -- ok
61+
memory usage = 8,521,850,880 pf_rate = 0.0 => squeeze percent = 2%
62+
squeeze to 8,351,413,862 -- ok
63+
memory usage = 8,351,191,040 pf_rate = 0.0 => squeeze percent = 3%
64+
squeeze to 8,100,655,308 -- ok
65+
memory usage = 8,100,462,592 pf_rate = 0.0 => squeeze percent = 4%
66+
squeeze to 7,776,444,088 -- ok
67+
memory usage = 7,776,321,536 pf_rate = 0.0 => squeeze percent = 5%
68+
squeeze to 7,387,505,459 -- ok
69+
[INFO] (main:77) : Done with test in 37 seconds.
70+
memory usage = 7,387,156,480 pf_rate = 0.0 => squeeze percent = 6%
71+
2024-05-22 03:34:52,774 - memoryusageanalyzer - INFO - **** Job finished
72+
2024-05-22 03:34:52,774 - memoryusageanalyzer - INFO - **** Stopping stats
73+
**** Closing ./profile_workload1.9/baseline/stats.csv.gz
74+
squeeze to 6,943,927,091./profile_workload1.9/baseline
75+
Generating plots ./profile_workload1.9/baseline/memoryusageanalyzer-plots.html
76+
Loading stats from ./profile_workload1.9/baseline/stats.csv.gz
77+
Loading stats from ./profile_workload1.9/baseline/stats.csv.gz
78+
*** Totals without page cache ***
79+
Maximum active+swap memory = 8.029 GiB
80+
Max active memory = 8.0 GiB
81+
Max swap memory = 0.0 GiB
82+
Max page cache = 0.010 GiB
83+
Swap memory = 0.0%
84+
Max inactive anon = 8.002 GiB
85+
Maximum zpool memory = 0.831486 GiB
86+
Maximum zram memory = 0.000019 GiB
87+
incompressible data ratio(zram)= 66.7%
88+
full total memory pressure = 0 us
89+
Average swap memory comp ratio = 2.00
90+
Potential maximum compression savings in percent = -0.0%
91+
Potential median compression savings in percent = -3.0%
92+
Potential compression savings in bytes = -0.001 GiB
93+
Average swap memory comp ratio = 2.00 (including same filled pages)
94+
Major PFs/sec (median, avg, max) = 0.00 0.00 0.00
95+
Total PFs/sec (median, avg, max) = 0.00 51639.35 361475.42
96+
Total PFs (Major+Minor) = 1814311
97+
Major PFs = 0
98+
sampling period = 5.01918227331979
99+
-- ok
100+
101+
[INFO] (main:77) : Done with test in 40 seconds.
102+
2024-05-22 03:34:56,111 - memoryusageanalyzer - INFO - **** Job finished
103+
2024-05-22 03:34:56,111 - memoryusageanalyzer - INFO - **** Stopping stats
104+
memory usage = 393,216 pf_rate = 25.0 => squeeze percent = 7%
105+
squeeze to 365,690 -- ok
106+
**** Closing ./profile_workload2.9/dynamic/stats.csv.gz
107+
2024-05-22 03:34:58,161 - memoryusageanalyzer - INFO - **** Stopping reclaimer
108+
**** Squeezer exiting
109+
./profile_workload2.9/dynamic
110+
Generating plots ./profile_workload2.9/dynamic/memoryusageanalyzer-plots.html
111+
Loading stats from ./profile_workload2.9/dynamic/stats.csv.gz
112+
Loading stats from ./profile_workload2.9/dynamic/stats.csv.gz
113+
*** Totals without page cache ***
114+
Maximum active+swap memory = 8.017 GiB
115+
Max active memory = 8.0 GiB
116+
Max swap memory = 2.5 GiB
117+
Max page cache = 0.000 GiB
118+
Swap memory = 30.7%
119+
Max inactive anon = 7.576 GiB
120+
Maximum zpool memory = 1.210579 GiB
121+
Maximum zram memory = 0.000019 GiB
122+
incompressible data ratio(zram)= 66.7%
123+
full total memory pressure = 671112 us
124+
Average swap memory comp ratio = 2.00
125+
Potential maximum compression savings in percent = 2.8%
126+
Potential median compression savings in percent = 4.5%
127+
Potential compression savings in bytes = 0.226 GiB
128+
Average swap memory comp ratio = 2.00 (including same filled pages)
129+
Major PFs/sec (median, avg, max) = 0.00 2293.66 18349.26
130+
Total PFs/sec (median, avg, max) = 0.00 47318.79 360200.84
131+
Total PFs (Major+Minor) = 1899099
132+
Major PFs = 92054
133+
sampling period = 5.016767889261246
134+
```

‎docs/sanity.md

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Sanity Tests
2+
3+
This section provides details on running Sanity checks before code changes are submitted. This checks the
4+
sure the basic functionality of the memory-usage-analyzer.
5+
6+
```
7+
./sanity_test -o <output_folder>
8+
#Sample results, actual results may vary
9+
Running baseline example workload test
10+
iteration = 0
11+
iteration = 1
12+
iteration = 2
13+
output/baseline/analyze.out
14+
Maximum active+swap memory = 8.018 GiB
15+
16+
Max swap memory = 0.0 GiB
17+
18+
333.751453340 seconds time elapsed
19+
20+
output.1/baseline/analyze.out
21+
Maximum active+swap memory = 8.018 GiB
22+
Max swap memory = 0.0 GiB
23+
24+
336.614511911 seconds time elapsed
25+
26+
output.2/baseline/analyze.out
27+
Maximum active+swap memory = 8.018 GiB
28+
29+
Max swap memory = 0.0 GiB
30+
31+
333.530823142 seconds time elapsed
32+
33+
Baseline max_mem is less than 1% limit of ref and test is passed
34+
Baseline avg_exec_time = 334.07142835766666
35+
36+
Running static squeezer with 10% memory limit
37+
iteration = 0
38+
iteration = 1
39+
iteration = 2
40+
output_static/staticsweep/memory_10/analyze.out
41+
Maximum active+swap memory = 7.217 GiB
42+
43+
Max swap memory = 1.6 GiB
44+
45+
407.835192013 seconds time elapsed
46+
47+
output_static.1/staticsweep/memory_10/analyze.out
48+
Maximum active+swap memory = 7.217 GiB
49+
50+
Max swap memory = 1.6 GiB
51+
52+
408.547875633 seconds time elapsed
53+
54+
output_static.2/staticsweep/memory_10/analyze.out
55+
Maximum active+swap memory = 7.216 GiB
56+
57+
Max swap memory = 1.6 GiB
58+
59+
411.597475507 seconds time elapsed
60+
61+
Static max_mem is less than 1% limit of ref and test is passed
62+
Static max_swap is less than 1% limit of ref and test is passed
63+
Static avg_exec_time = 409.3268477176666
64+
Running dynamic squeezer
65+
iteration = 0
66+
iteration = 1
67+
iteration = 2
68+
Maximum active+swap memory = 8.017 GiB
69+
70+
Max swap memory = 4.3 GiB
71+
72+
Maximum zram memory = 0.000019 GiB
73+
74+
372.675095471 seconds time elapsed
75+
76+
Maximum active+swap memory = 8.017 GiB
77+
78+
Max swap memory = 4.3 GiB
79+
80+
Maximum zram memory = 0.000019 GiB
81+
82+
369.093067576 seconds time elapsed
83+
84+
Maximum active+swap memory = 8.017 GiB
85+
86+
Max swap memory = 4.4 GiB
87+
88+
Maximum zram memory = 0.000023 GiB
89+
90+
365.854511618 seconds time elapsed
91+
92+
Dynamic max_mem is less than 1% limit of ref and test is passed
93+
Dynamic max_swap is less than 5% limit of ref and test is passed
94+
Dynamic avg_max_zram is 2.0333333333333334e-05
95+
Dynamic avg_exec_time is 369.20755822166666
96+
97+
```
98+

‎docs/workload_container.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Run a workload in a docker container:
2+
1. Give the workload's docker container a name on the docker command line (`--name NAME`)
3+
2. For convenience, have docker automatically remove the container when it exits, so the container name can be reused (`--rm`)
4+
3. Add the `--docker NAME` to the `memory-usage-analyzer.py` command.
5+
6+
If cgroup v2 is enabled, do specify the cgroup parent by `--cgroup-parent <memoryusageanalyzer.slice>`, otherwise stats won't be collected correctly.
7+
8+
```shell
9+
docker info
10+
# Sample output
11+
Cgroup Version: 2
12+
13+
```
14+
Example:
15+
```shell
16+
sudo docker run --rm --name memoryusageanalyzer -v $PWD:/memoryusageanalyzer ubuntu /memoryusageanalyzer/example/workload 4 30 1000
17+
memory_usage_analyzer.py --docker memoryusageanalyzer ./docker.sh
18+
19+
# Sample results, actual results may vary
20+
2024-01-17 00:03:09,703 - memoryusageanalyzer - INFO - args = Namespace(verbose=False, output='profile', output_force=None, sample_period=5, compaction_period=0, docker='memoryusageanalyzer', skip_stats=0, cgpath='/sys/fs/cgroup', cgname='memoryusageanalyzer', reclaimer_config=None, cmd='./docker.sh', options=[])
21+
2024-01-17 00:03:09,703 - memoryusageanalyzer - INFO - **** Storing profiling results in profile
22+
2024-01-17 00:03:09,708 - memoryusageanalyzer - INFO - cgroup v2 enabled and memory controller detected
23+
2024-01-17 00:03:09,713 - memoryusageanalyzer - INFO - **** Starting job in docker
24+
2024-01-17 00:03:09,715 - memoryusageanalyzer - INFO - **** Waiting for container "memoryusageanalyzer"
25+
[INFO] (main:56) : pid = 1
26+
[INFO] (main:57) : memory = 4 GiB = 1048576 pages
27+
[INFO] (main:58) : hot timer = 30 sec
28+
[INFO] (main:59) : loops = 1000
29+
[INFO] (main:73) : Done with setup in 0 seconds.
30+
2024-01-17 00:03:10,879 - memoryusageanalyzer - INFO - **** Found container "memoryusageanalyzer" = 659881870e44e57696553cd1c19ac0710c43bfa7ed94bb917c6db328b468b72f
31+
2024-01-17 00:03:10,895 - memoryusageanalyzer - INFO - **** Starting stats
32+
[INFO] (main:90) : Done with test in 10 seconds.
33+
2024-01-17 00:06:41,304 - memoryusageanalyzer - INFO - **** Job finished
34+
2024-01-17 00:06:41,304 - memoryusageanalyzer - INFO - **** Stopping stats
35+
**** Closing profile/baseline/stats.csv.gz
36+
```

‎setup.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,13 +25,14 @@ def run(self):
2525
description='Memory compression profiling tool.',
2626
author='Pallavi G, Ravindran Binuraj',
2727
author_email='[email protected], [email protected]',
28-
28+
2929
python_requires='>=3.7',
3030
# Fixing the bokeh version as newer versions like 3.0 shrink the plots
3131
install_requires=['pandas', 'bokeh==2.4.3', 'shapely', 'matplotlib','pytest', 'xlsxwriter',\
32-
'numpy', 'colorcet', 'scipy', 'jsonschema'],
32+
'numpy==1.26.4', 'colorcet', 'scipy', 'jsonschema'],
3333
packages=find_packages(),
34-
scripts=['src/core/memory-usage-analyzer.py',
34+
scripts=['src/core/memory_usage_analyzer.py',
35+
'tests/example/multi_config.py',
3536
'src/core/stats.py',
3637
'src/analyzer/plot.py',
3738
'src/analyzer/analyze.py',

‎src/analyzer/analyze.py

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@
66
import argparse
77
import subprocess
88

9-
from src.core.util import cold_hot_total, max_memory_usage, read_stats_df, get_file_sum,\
10-
get_cgroup_total, compression_ratio, calculate_savings
9+
from src.core.util import cold_hot_total, max_memory_usage, read_stats_df,\
10+
compression_ratio, calculate_savings
1111

1212
def get_total_pf_count(df, counter, t_sample):
1313
"""get the total paga fault"""
@@ -48,11 +48,6 @@ def memoryanalysis(self , df):
4848
df.cgroup_memory_pressure_full_total.min()
4949

5050
cold, hot, page_cache, total = cold_hot_total(df)
51-
52-
cgroup_total_max = get_cgroup_total(df).max() / (1 << 30)
53-
file_sum = get_file_sum(df).max() / (1 << 30)
54-
total_diff = abs(total - cgroup_total_max)
55-
5651
ratio, ratio_with_same_filled = compression_ratio(df)
5752
potential_memory_savings_bytes, potential_maximum_memory_savings_percent,\
5853
potential_median_memory_savings_percent = calculate_savings(df)
@@ -84,13 +79,12 @@ def memoryanalysis(self , df):
8479
if ratio_with_same_filled > 1.001:
8580
print(f'Average swap memory comp ratio = {ratio_with_same_filled:.2f}\
8681
(including same filled pages)')
87-
return file_sum, total_diff, cgroup_total_max, total
8882

8983
def run(self):
9084
"""Analyzer run method"""
9185
print(f'Loading stats from {self.resultpath}/stats.csv.gz')
9286
df = read_stats_df(f'{self.resultpath}/stats.csv.gz')
93-
file_sum, total_diff, cgroup_total_max, total = self.memoryanalysis(df)
87+
self.memoryanalysis(df)
9488
t_sample = df.time.diff().mean()
9589

9690
# Get Major page fault

‎src/core/cgroup_config.sh

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
#!/usr/bin/env bash
2+
#SPDX-License-Identifier: BSD-3-Clause
3+
#Copyright (c) 2023, Intel Corporation
4+
5+
if [ $EUID -ne 0 ]
6+
then echo "Must run as root or with sudo."
7+
exit
8+
fi
9+
10+
cgpath=$1; shift
11+
cgname=$1; shift
12+
cguser=$1; shift
13+
14+
if [[ -e ${cgpath}/${cgname} ]]; then
15+
# remove existing cgroup v2
16+
until rmdir ${cgpath}/${cgname} >& /dev/null; do
17+
cat ${cgpath}/${cgname}/cgroup.procs | xargs kill -9 >& /dev/null
18+
sleep 1
19+
done
20+
fi
21+
22+
# setup cgroup
23+
mkdir ${cgpath}/${cgname}
24+
echo 'if path exist'
25+
chown ${cguser} ${cgpath}/cgroup.procs
26+
chown -R ${cguser} ${cgpath}/${cgname}
27+
echo 'if path exist'
28+
# set swappiness
29+
echo max > ${cgpath}/${cgname}/memory.swap.max

0 commit comments

Comments
 (0)