Memory management in C++ Part 2

29 Jun

While part 1 focussed on avoiding memory leaks, this post will look at memory strategies which optimize speed. While in many cases a normal allocation/reallocation/deallocation scheme works well, there are several instances where this can slow down software considerably. In these cases one of the following strategies can help to improve the performance of your application.

Block cache

A block cache is a collection of preallocated memory blocks from which you can retrieve and return blocks. The obvious performance gain of a using a block cache this is that at runtime you are no longer wasting time allocating memory. To allocate a block of memory, the system needs to find a memory region as large as you request, and do some housekeeping in order to prevent fragmentation. In most cases this is not a big overhead, however on mobile for example allocating and deallocating can be a lot slower.

I used this myself in two instances, one was on mobile where a real-time performance monitor needed to use as little possible CPU in order to monitor other programs consistently, and in a real time transcoding server where network messages would otherwise need a lot of allocations/deallocations. For a full implementation of a working block cache look at BBlockCache.cpp, which is where I encountered it for the first time, while I was a contributor to Haiku OS many years ago. A simple usage would be by overriding the new operator:

class Packet {
  void *operator new(size_t size) {
    return gCache->get(size);
  void operator delete(void *pointer, size_t size) {
    if (pointer == NULL)
      gCache->save(pointer, size);

  static init() {
    sCache = new BlockCache(20, sizeof(Packet), B_OBJECT_CACHE);
  static cleanup() {
    delete sCache;
    gCache = NULL;

  static BlockCache *gCache;

static BlockCache *Packet::gCache = NULL;

COW: Copy On Write

Copy on write means that multiple instances can use the same memory as long as they only read from it. Once they want to write they need to make a copy. A place where this is used often is the string class. When assigning a string, it is common that you actually don’t need a copy, though you copy because you don’t want it to be changed by others outside your context.

class Book {
  void setTitle(const String &title) { fTitle = title; }
  String fTitle;

Book b;
b.setTitle("Great North Road");

Here the string’s buffer would be copied into the fTitle member, then the temporary string would be deleted. This is clearly a waste of CPU time and memory. When using copy on write, we actually use a reference count in order to keep track of how many readers we have. If we want to change the memory, and there is more than one reader, we create a copy, which has a reader count of 1, and decrease the reader count of the original memory buffer. A good example would be Qt, which uses this extensively. For an example of a full implementation you can look at my COW classes here: cow.h. Basically you override the CowData to keep your shared data, then you use a CowPointer in your class to that data. It makes a copy each time you need a non const CowData. I used it in the same real time transcoding server to avoid copies of audio and video data in case one of these didn’t need transcoding.

Gap buffer

This is mostly given as an example of how you need to think about how your memory is going to be used in order to optimize your code. Most buffers, like used in string or array classes, will have their unused memory at the end of the buffer. You usually keep a total and used size, or end and allocated end pointers in case of iterator enabled arrays. A gap buffer, which is mostly used in text editors, keeps its unused memory or gap at the position where the user is typing. This makes adding and removing characters fast as you don’t need to reallocate and move memory unless the user changes the insertion point or your gap is full. For a full implementation of a gap buffer you can look at TextGapBuffer.cpp.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: