Download the stable version as tarball .
Check it on Github.
Read the Changelog for the stable version.

Introduction #

Enables legacy applications to leverage Linux’s memory deduplication. Only works on Linux ≥ 2.6.32.

If you have multiple processes generating similar data (like multiple applications serving different vhosts), this easy to use tool may allow you to spare memory by deduplicating it. If two pages (that is on, most systems, consecutive blocks of 4k) are identical, they will be merged into a single page to reduce memory usage.

Explanation #

Linux ≥ 2.6.32 features a memory-saving mechanism that works by deduplicating areas of memory that are identical in different processes (even if they were generated at runtime and after the fork() of their common ancestors).

This mechanism requires the application to opt-in using the madvise() syscall. KSM Preload enables legacy applications (about any current application) to leverage this system by calling madvise(…, MADV_MERGEABLE) on every heap-allocated pages.

In contrast to similar hacks, special care has been taken to make it safe to use, including with other preloaded libraries.

Numbers #

I did a very few experiments to evaluate the efficiency of the method. They’re not worth much, but still…

Those data where gathered by running 50 instances of Apache on a single machine, each serving a vhost (for a mutualised hosting provider).
Each instance of Apache served different files and forums, but used the same versions of the softwares.

Deduplicated: Pages which were effectively deduplicated, sparing memory.
Unique: Pages that could not be deduplicated because there were no other copies of them on the system. The system lost CPU time trying to deduplicate them.
Volatile: Pages that could not be deduplicated because they’re changing to often. The system lost CPU time trying to deduplicate them.

Situation	Deduplicated	Uniques	Volatiles	Spared memory
No load	1351	2638	10500	~5MB
Static pages under Siege	15289	3239	1116	~60MB
PhpBB under siege	25534	36687	775	~100MB

Instructions #

To run your programs using ksm_preload, first download the stable version. then compile it with “make”, and use the provided ksm-wrapper :

make
sudo ./ksm-wrapper echo "I am being run via ksm-preload"

In order to see if it worked, you can run the following snipet :

cd /sys/kernel/mm/ksm
watch 'for f in *; do echo -n $f: ; cat $f; done'

The interpretation is the same as for the “Numbers” section above.