Boost C++ Libraries

As we have seen, Boost.Interprocess offers some basic classes to create shared memory objects and file mappings and map those mappable classes to the process' address space.

However, managing those memory segments is not not easy for non-trivial tasks. A mapped region is a fixed-length memory buffer and creating and destroying objects of any type dynamically, requires a lot of work, since it would require programming a memory management algorithm to allocate portions of that segment. Many times, we also want to associate names to objects created in shared memory, so all the processes can find the object using the name.

Boost.Interprocess offers 4 managed memory segment classes:

The first two classes manage memory segments that can be shared between processes. The third is useful to create complex data-bases to be sent though other mechanisms like message queues to other processes. The fourth class can manage any fixed size memory buffer. The first two classes will be explained in the next two sections. basic_managed_heap_memory and basic_managed_external_buffer will be explained later.

The most important services of a managed memory segment are:

All Boost.Interprocess managed memory segment classes are templatized classes that can be customized by the user:

template
      <
         class CharType,
         class MemoryAlgorithm,
         template<class IndexConfig> class IndexType
      >
class basic_managed_shared_memory / basic_managed_mapped_file /
      basic_managed_heap_memory   / basic_external_buffer;

These classes can be customized with the following template parameters:

This way, we can use char or wchar_t strings to identify created C++ objects in the memory segment, we can plug new shared memory allocation algorithms, and use the index type that is best suited to our needs.

As seen, basic_managed_shared_memory offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, Boost.Interprocess defines the most common managed shared memory specializations:

//!Defines a managed shared memory with c-strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef
   basic_managed_shared_memory<char
                              ,/*Default memory algorithm defining offset_ptr<void> as void_pointer*/
                              ,/*Default index type*/>
   managed_shared_memory;

//!Defines a managed shared memory with wide strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef
   basic_managed_shared_memory<wchar_t
                              ,/*Default memory algorithm defining offset_ptr<void> as void_pointer*/
                              ,/*Default index type*/>
   wmanaged_shared_memory;

managed_shared_memory allocates objects in shared memory associated with a c-string and wmanaged_shared_memory allocates objects in shared memory associated with a wchar_t null terminated string. Both define the pointer type as offset_ptr<void> so they can be used to map the shared memory at different base addresses in different processes.

If the user wants to map the shared memory in the same address in all processes and want to use raw pointers internally instead of offset pointers, Boost.Interprocess defines the following types:

//!Defines a managed shared memory with c-strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes*/
typedef basic_managed_shared_memory
   <char
   ,/*Default memory algorithm defining void * as void_pointer*/
   ,/*Default index type*/>
fixed_managed_shared_memory;

//!Defines a managed shared memory with wide strings as keys for named objects,
//!the default memory algorithm (with process-shared mutexes,
//!and offset_ptr as internal pointers) as memory allocation algorithm
//!and the default index type as the index.
//!This class allows the shared memory to be mapped in different base
//!in different processes
typedef basic_managed_shared_memory
   <wchar_t
   ,/*Default memory algorithm defining void * as void_pointer*/
   ,/*Default index type*/>
wfixed_managed_shared_memory;

Managed shared memory is an advanced class that combines a shared memory object and a mapped region that covers all the shared memory object. That means that when we create a new managed shared memory:

When we open a managed shared memory

To use a managed shared memory, you must include the following header:

#include <boost/interprocess/managed_shared_memory.hpp>

//1.  Creates a new shared memory object
//    called "MySharedMemory".
//2.  Maps the whole object to this
//    process' address space.
//3.  Constructs some objects in shared memory
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment      ( create_only
                                   , "MySharedMemory" //Shared memory object name
                                   , 65536);          //Shared memory object size in bytes

//1.  Opens a shared memory object
//    called "MySharedMemory".
//2.  Maps the whole object to this
//    process' address space.
//3.  Obtains pointers to constructed internal objects
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment      (open_only,       "MySharedMemory");//Shared memory object name

//1.  If the segment was previously created
//    equivalent to "open_only" (size is ignored).
//2.  Otherwise, equivalent to "create_only"
//!!  If anything fails, throws interprocess_exception
//
managed_shared_memory segment      ( open_or_create
                                   , "MySharedMemory" //Shared memory object name
                                   , 65536);          //Shared memory object size in bytes

When the managed_shared_memory object is destroyed, the shared memory object is automatically unmapped, and all the resources are freed. To remove the shared memory object from the system you must use the shared_memory_object::remove function. Shared memory object removing might fail if any process still has the shared memory object mapped.

The user can also map the managed shared memory in a fixed address. This option is essential when using using fixed_managed_shared_memory. To do this, just add the mapping address as an extra parameter:

fixed_managed_shared_memory segment      (open_only      ,"MyFixedAddressSharedMemory" //Shared memory object name
   ,(void*)0x30000000            //Mapping address

Windows users might also want to use native windows shared memory instead of the portable shared_memory_object managed memory. This is achieved through the basic_managed_windows_shared_memory class. To use it just include:

#include <boost/interprocess/managed_windows_shared_memory.hpp>

This class has the same interface as basic_managed_shared_memory but uses native windows shared memory. Note that this managed class has the same lifetime issues as the windows shared memory: when the last process attached to the windows shared memory is detached from the memory (or ends/crashes) the memory is destroyed. So there is no persistence support for windows shared memory.

To communicate between system services and user applications using managed_windows_shared_memory, please read the explanations given in chapter Native windows shared memory.

Unix users might also want to use XSI (system V) instead of the portable shared_memory_object managed memory. This is achieved through the basic_managed_xsi_shared_memory class. To use it just include:

#include <boost/interprocess/managed_xsi_shared_memory.hpp>

This class has nearly the same interface as basic_managed_shared_memory but uses XSI shared memory as backend.

As seen, basic_managed_mapped_file offers a great variety of customization. But for the average user, a common, default shared memory named object creation is needed. Because of this, Boost.Interprocess defines the most common managed mapped file specializations:

//Named object creation managed memory segment
//All objects are constructed in the memory-mapped file
//   Names are c-strings,
//   Default memory management algorithm(rbtree_best_fit with no mutexes)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_mapped_file <
   char,
   rbtree_best_fit<mutex_family, offset_ptr<void> >,
   flat_map_index
   >  managed_mapped_file;

//Named object creation managed memory segment
//All objects are constructed in the memory-mapped file
//   Names are wide-strings,
//   Default memory management algorithm(rbtree_best_fit with no mutexes)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_mapped_file<
   wchar_t,
   rbtree_best_fit<mutex_family, offset_ptr<void> >,
   flat_map_index
   >  wmanaged_mapped_file;

managed_mapped_file allocates objects in a memory mapped files associated with a c-string and wmanaged_mapped_file allocates objects in a memory mapped file associated with a wchar_t null terminated string. Both define the pointer type as offset_ptr<void> so they can be used to map the file at different base addresses in different processes.

Managed mapped file is an advanced class that combines a file and a mapped region that covers all the file. That means that when we create a new managed mapped file:

When we open a managed mapped file

To use a managed mapped file, you must include the following header:

#include <boost/interprocess/managed_mapped_file.hpp>

//1.  Creates a new file
//    called "MyMappedFile".
//2.  Maps the whole file to this
//    process' address space.
//3.  Constructs some objects in the memory mapped
//    file to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      ( create_only
                               , "MyMappedFile" //Mapped file name
                               , 65536);        //Mapped file size

//1.  Opens a file
//    called "MyMappedFile".
//2.  Maps the whole file to this
//    process' address space.
//3.  Obtains pointers to constructed internal objects
//    to implement managed features.
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      ( open_only
                               , "MyMappedFile"); //Mapped file name

//1.  If the file was previously created
//    equivalent to "open_only".
//2.  Otherwise, equivalent to "open_only" (size is ignored)
//
//!!  If anything fails, throws interprocess_exception
//
managed_mapped_file mfile      (open_or_create
                               , "MyMappedFile" //Mapped file name
                               , 65536);        //Mapped file size

When the managed_mapped_file object is destroyed, the file is automatically unmapped, and all the resources are freed. To remove the file from the filesystem you could use standard C std::remove or Boost.Filesystem's remove() functions, but file removing might fail if any process still has the file mapped in memory or the file is open by any process.

To obtain a more portable behaviour, use file_mapping::remove(const char *) operation, which will remove the file even if it's being mapped. However, removal will fail in some OS systems if the file (eg. by C++ file streams) and no delete share permission was granted to the file. But in most common cases file_mapping::remove is portable enough.

If a basic raw-byte allocation is needed from a managed memory segment, (for example, a managed shared memory), to implement top-level interprocess communications, this class offers allocate and deallocate functions. The allocation function comes with throwing and no throwing versions. Throwing version throws boost::interprocess::bad_alloc (which derives from std::bad_alloc) if there is no more memory and the non-throwing version returns 0 pointer.

#include <boost/interprocess/managed_shared_memory.hpp>

int main()
{
   using namespace boost::interprocess;

   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   //Managed memory segment that allocates portions of a shared memory
   //segment with the default management algorithm
   managed_shared_memory managed_shm(create_only,"MySharedMemory", 65536);

   //Allocate 100 bytes of memory from segment, throwing version
   void *ptr = managed_shm.allocate(100);

   //Deallocate it
   managed_shm.deallocate(ptr);

   //Non throwing version
   ptr = managed_shm.allocate(100, std::nothrow);

   //Deallocate it
   managed_shm.deallocate(ptr);
   return 0;
}

The class also offers conversions between absolute addresses that belong to a managed memory segment and a handle that can be passed using any interprocess mechanism. That handle can be transformed again to an absolute address using a managed memory segment that also contains that object. Handles can be used as keys between processes to identify allocated portions of a managed memory segment or objects constructed in the managed segment.

//Process A obtains the offset of the address
managed_shared_memory::handle handle =
   segment.get_handle_from_address(processA_address);

//Process A sends this address using any mechanism to process B

//Process B obtains the handle and transforms it again to an address
managed_shared_memory::handle handle = ...
void * processB_address = segment.get_address_from_handle(handle);

When constructing objects in a managed memory segment (managed shared memory, managed mapped files...) associated with a name, the user has a varied object construction family to "construct" or to "construct if not found". Boost.Interprocess can construct a single object or an array of objects. The array can be constructed with the same parameters for all objects or we can define each parameter from a list of iterators:

//!Allocates and constructs an object of type MyType (throwing version)
MyType *ptr = managed_memory_segment.construct<MyType>("Name") (par1, par2...);

//!Allocates and constructs an array of objects of type MyType (throwing version)
//!Each object receives the same parameters (par1, par2, ...)
MyType *ptr = managed_memory_segment.construct<MyType>("Name")[count](par1, par2...);

//!Tries to find a previously created object. If not present, allocates
//!and constructs an object of type MyType (throwing version)
MyType *ptr = managed_memory_segment.find_or_construct<MyType>("Name") (par1, par2...);

//!Tries to find a previously created object. If not present, allocates and
//!constructs an array of objects of type MyType (throwing version). Each object
//!receives the same parameters (par1, par2, ...)
MyType *ptr = managed_memory_segment.find_or_construct<MyType>("Name")[count](par1, par2...);

//!Allocates and constructs an array of objects of type MyType (throwing version)
//!Each object receives parameters returned with the expression (*it1++, *it2++,... )
MyType *ptr = managed_memory_segment.construct_it<MyType>("Name")[count](it1, it2...);

//!Tries to find a previously created object. If not present, allocates and constructs
//!an array of objects of type MyType (throwing version).  Each object receives
//!parameters returned with the expression (*it1++, *it2++,... )
MyType *ptr = managed_memory_segment.find_or_construct_it<MyType>("Name")[count](it1, it2...);

//!Tries to find a previously created object. Returns a pointer to the object and the
//!count (if it is not an array, returns 1). If not present, the returned pointer is 0
std::pair<MyType *,std::size_t> ret = managed_memory_segment.find<MyType>("Name");

//!Destroys the created object, returns false if not present
bool destroyed = managed_memory_segment.destroy<MyType>("Name");

//!Destroys the created object via pointer
managed_memory_segment.destroy_ptr(ptr);

All these functions have a non-throwing version, that is invoked with an additional parameter std::nothrow. For example, for simple object construction:

//!Allocates and constructs an object of type MyType (no throwing version)
MyType *ptr = managed_memory_segment.construct<MyType>("Name", std::nothrow) (par1, par2...);

Sometimes, the user doesn't want to create class objects associated with a name. For this purpose, Boost.Interprocess can create anonymous objects in a managed memory segment. All named object construction functions are available to construct anonymous objects. To allocate an anonymous objects, the user must use "boost::interprocess::anonymous_instance" name instead of a normal name:

MyType *ptr = managed_memory_segment.construct<MyType>(anonymous_instance) (par1, par2...);

//Other construct variants can also be used (including non-throwing ones)
...

//We can only destroy the anonymous object via pointer
managed_memory_segment.destroy_ptr(ptr);

Find functions have no sense here, since anonymous objects have no name. We can only destroy the anonymous object via pointer.

Sometimes, the user wants to emulate a singleton in a managed memory segment. Obviously, as the managed memory segment is constructed at run-time, the user must construct and destroy this object explicitly. But how can the user be sure that the object is the only object of its type in the managed memory segment? This can be emulated using a named object and checking if it is present before trying to create one, but all processes must agree in the object's name, that can also conflict with other existing names.

To solve this, Boost.Interprocess offers a "unique object" creation in a managed memory segment. Only one instance of a class can be created in a managed memory segment using this "unique object" service (you can create more named objects of this class, though) so it makes easier the emulation of singleton-like objects across processes, for example, to design pooled, shared memory allocators. The object can be searched using the type of the class as a key.

// Construct
MyType *ptr = managed_memory_segment.construct<MyType>(unique_instance) (par1, par2...);

// Find it
std::pair<MyType *,std::size_t> ret = managed_memory_segment.find<MyType>(unique_instance);

// Destroy it
managed_memory_segment.destroy<MyType>(unique_instance);

// Other construct and find variants can also be used (including non-throwing ones)
//...

// We can also destroy the unique object via pointer
MyType *ptr = managed_memory_segment.construct<MyType>(unique_instance) (par1, par2...);
managed_shared_memory.destroy_ptr(ptr);

The find function obtains a pointer to the only object of type T that can be created using this "unique instance" mechanism.

One of the features of named/unique allocations/searches/destructions is that they are atomic. Named allocations use the recursive synchronization scheme defined by the internal mutex_family typedef defined of the memory allocation algorithm template parameter (MemoryAlgorithm). That is, the mutex type used to synchronize named/unique allocations is defined by the MemoryAlgorithm::mutex_family::recursive_mutex_type type. For shared memory, and memory mapped file based managed segments this recursive mutex is defined as interprocess_recursive_mutex.

If two processes can call:

MyType *ptr = managed_shared_memory.find_or_construct<MyType>("Name")[count](par1, par2...);

at the same time, but only one process will create the object and the other will obtain a pointer to the created object.

Raw allocation using allocate() can be called also safely while executing named/anonymous/unique allocations, just like when programming a multithreaded application inserting an object in a mutex-protected map does not block other threads from calling new[] while the map thread is searching the place where it has to insert the new object. The synchronization does happen once the map finds the correct place and it has to allocate raw memory to construct the new value.

This means that if we are creating or searching for a lot of named objects, we only block creation/searches from other processes but we don't block another process if that process is inserting elements in a shared memory vector.

As seen, managed memory segments, when creating named objects, store the name/object association in an index. The index is a map with the name of the object as a key and a pointer to the object as the mapped type. The default specializations, managed_shared_memory and wmanaged_shared_memory, use flat_map_index as the index type.

Each index has its own characteristics, like search-time, insertion time, deletion time, memory use, and memory allocation patterns. Boost.Interprocess offers 3 index types right now:

As an example, if we want to define new managed shared memory class using boost::interprocess::map as the index type we just must specify [boost::interprocess::map_index map_index] as a template parameter:

//This managed memory segment can allocate objects with:
// -> a wchar_t string as key
// -> boost::interprocess::rbtree_best_fit with process-shared mutexes
//       as memory allocation algorithm.
// -> boost::interprocess::map<...> as the index to store name/object mappings
//
typedef boost::interprocess::basic_managed_shared_memory
         <  wchar_t
         ,  boost::interprocess::rbtree_best_fit<boost::interprocess::mutex_family, offset_ptr<void> >
         ,  boost::interprocess::map_index
         >  my_managed_shared_memory;

Boost.Interprocess plans to offer an unordered_map based index as soon as this container is included in Boost. If these indexes are not enough for you, you can define your own index type. To know how to do this, go to Building custom indexes section.

All Boost.Interprocess managed memory segment classes construct in their respective memory segments (shared memory, memory mapped files, heap memory...) some structures to implement the memory management algorithm, named allocations, synchronization objects... All these objects are encapsulated in a single object called segment manager. A managed memory mapped file and a managed shared memory use the same segment manager to implement all managed memory segment features, due to the fact that a segment manager is a class that manages a fixed size memory buffer. Since both shared memory or memory mapped files are accessed though a mapped region, and a mapped region is a fixed size memory buffer, a single segment manager class can manage several managed memory segment types.

Some Boost.Interprocess classes require a pointer to the segment manager in their constructors, and the segment manager can be obtained from any managed memory segment using get_segment_manager member:

managed_shared_memory::segment_manager *seg_manager =
   managed_shm.get_segment_manager();

Once an object is constructed using construct<> function family, the programmer can obtain information about the object using a pointer to the object. The programmer can obtain the following information:

Here is an example showing this functionality:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <cassert>
#include <cstring>

class my_class
{
   //...
};

int main()
{
   using namespace boost::interprocess;

   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   managed_shared_memory managed_shm(create_only, "MySharedMemory", 10000*sizeof(std::size_t));

   //Construct objects
   my_class *named_object  = managed_shm.construct<my_class>("Object name")[1]();
   my_class *unique_object = managed_shm.construct<my_class>(unique_instance)[2]();
   my_class *anon_object   = managed_shm.construct<my_class>(anonymous_instance)[3]();

   //Now test "get_instance_name" function.
   assert(0 == std::strcmp(managed_shared_memory::get_instance_name(named_object), "Object name"));
   assert(0 == std::strcmp(managed_shared_memory::get_instance_name(unique_object), typeid(my_class).name()));
   assert(0 == managed_shared_memory::get_instance_name(anon_object));

   //Now test "get_instance_type" function.
   assert(named_type     == managed_shared_memory::get_instance_type(named_object));
   assert(unique_type    == managed_shared_memory::get_instance_type(unique_object));
   assert(anonymous_type == managed_shared_memory::get_instance_type(anon_object));

   //Now test "get_instance_length" function.
   assert(1 == managed_shared_memory::get_instance_length(named_object));
   assert(2 == managed_shared_memory::get_instance_length(unique_object));
   assert(3 == managed_shared_memory::get_instance_length(anon_object));

   managed_shm.destroy_ptr(named_object);
   managed_shm.destroy_ptr(unique_object);
   managed_shm.destroy_ptr(anon_object);
   return 0;
}

Sometimes the programmer must execute some code, and needs to execute it with the guarantee that no other process or thread will create or destroy any named, unique or anonymous object while executing the functor. A user might want to create several named objects and initialize them, but those objects should be available for the rest of processes at once.

To achieve this, the programmer can use the atomic_func() function offered by managed classes:

//This object function will create several named objects
create_several_objects_func func(/**/);

//While executing the function, no other process will be
//able to create or destroy objects
managed_memory.atomic_func(func);

Note that atomic_func does not prevent other processes from allocating raw memory or executing member functions for already constructed objects (e.g.: another process might be pushing elements into a vector placed in the segment). The atomic function only blocks named, unique and anonymous creation, search and destruction (concurrent calls to construct<>, find<>, find_or_construct<>, destroy<>...) from other processes.

These functions are available to obtain information about the managed memory segments:

Obtain the size of the memory segment:

managed_shm.get_size();

Obtain the number of free bytes of the segment:

managed_shm.get_free_memory();

Clear to zero the free memory:

managed_shm.zero_free_memory();

Know if all memory has been deallocated, false otherwise:

managed_shm.all_memory_deallocated();

Test internal structures of the managed segment. Returns true if no errors are detected:

managed_shm.check_sanity();

Obtain the number of named and unique objects allocated in the segment:

managed_shm.get_num_named_objects();
managed_shm.get_num_unique_objects();

Once a managed segment is created the managed segment can't be grown. The limitation is not easily solvable: every process attached to the managed segment would need to be stopped, notified of the new size, they would need to remap the managed segment and continue working. Nearly impossible to achieve with a user-level library without the help of the operating system kernel.

On the other hand, Boost.Interprocess offers off-line segment growing. What does this mean? That the segment can be grown if no process has mapped the managed segment. If the application can find a moment where no process is attached it can grow or shrink to fit the managed segment.

Here we have an example showing how to grow and shrink to fit managed_shared_memory:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/interprocess/managed_mapped_file.hpp>
#include <cassert>

class MyClass
{
   //...
};

int main()
{
   using namespace boost::interprocess;
   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   {
      //Create a managed shared memory
      managed_shared_memory shm(create_only, "MySharedMemory", 1000);

      //Check size
      assert(shm.get_size() == 1000);
      //Construct a named object
      MyClass *myclass = shm.construct<MyClass>("MyClass")();
      //The managed segment is unmapped here
   }
   {
      //Now that the segment is not mapped grow it adding extra 500 bytes
      managed_shared_memory::grow("MySharedMemory", 500);

      //Map it again
      managed_shared_memory shm(open_only, "MySharedMemory");
      //Check size
      assert(shm.get_size() == 1500);
      //Check "MyClass" is still there
      MyClass *myclass = shm.find<MyClass>("MyClass").first;
      assert(myclass != 0);
      //The managed segment is unmapped here
   }
   {
      //Now minimize the size of the segment
      managed_shared_memory::shrink_to_fit("MySharedMemory");

      //Map it again
      managed_shared_memory shm(open_only, "MySharedMemory");
      //Check size
      assert(shm.get_size() < 1000);
      //Check "MyClass" is still there
      MyClass *myclass = shm.find<MyClass>("MyClass").first;
      assert(myclass != 0);
      //The managed segment is unmapped here
   }
   return 0;
}

managed_mapped_file also offers a similar function to grow or shrink_to_fit the managed file. Please, remember that no process should be modifying the file/shared memory while the growing/shrinking process is performed. Otherwise, the managed segment will be corrupted.

As mentioned, the managed segment stores the information about named and unique objects in two indexes. Depending on the type of those indexes, the index must reallocate some auxiliary structures when new named or unique allocations are made. For some indexes, if the user knows how many named or unique objects are going to be created it's possible to preallocate some structures to obtain much better performance. (If the index is an ordered vector it can preallocate memory to avoid reallocations. If the index is a hash structure it can preallocate the bucket array).

The following functions reserve memory to make the subsequent allocation of named or unique objects more efficient. These functions are only useful for pseudo-intrusive or non-node indexes (like flat_map_index, iunordered_set_index). These functions have no effect with the default index (iset_index) or other indexes (map_index):

managed_shm.reserve_named_objects(1000);
managed_shm.reserve_unique_objects(1000);

managed_shm.reserve_named_objects(1000);
managed_shm.reserve_unique_objects(1000);

Managed memory segments also offer the possibility to iterate through constructed named and unique objects for debugging purposes. Caution: this iteration is not thread-safe so the user should make sure that no other thread is manipulating named or unique indexes (creating, erasing, reserving...) in the segment. Other operations not involving indexes can be concurrently executed (raw memory allocation/deallocations, for example).

The following functions return constant iterators to the range of named and unique objects stored in the managed segment. Depending on the index type, iterators might be invalidated after a named or unique creation/erasure/reserve operation:

typedef managed_shared_memory::const_named_iterator const_named_it;
const_named_it named_beg = managed_shm.named_begin();
const_named_it named_end = managed_shm.named_end();

typedef managed_shared_memory::const_unique_iterator const_unique_it;
const_unique_it unique_beg = managed_shm.unique_begin();
const_unique_it unique_end = managed_shm.unique_end();

for(; named_beg != named_end; ++named_beg){
   //A pointer to the name of the named object
   const managed_shared_memory::char_type *name = named_beg->name();
   //The length of the name
   std::size_t name_len = named_beg->name_length();
   //A constant void pointer to the named object
   const void *value = named_beg->value();
}

for(; unique_beg != unique_end; ++unique_beg){
   //The typeid(T).name() of the unique object
   const char *typeid_name = unique_beg->name();
   //The length of the name
   std::size_t name_len = unique_beg->name_length();
   //A constant void pointer to the unique object
   const void *value = unique_beg->value();
}

Sometimes it's interesting to be able to allocate aligned fragments of memory because of some hardware or software restrictions. Sometimes, having aligned memory is a feature that can be used to improve several memory algorithms.

This allocation is similar to the previously shown raw memory allocation but it takes an additional parameter specifying the alignment. There is a restriction for the alignment: the alignment must be power of two.

If a user wants to allocate many aligned blocks (for example aligned to 128 bytes), the size that minimizes the memory waste is a value that's is nearly a multiple of that alignment (for example 2*128 - some bytes). The reason for this is that every memory allocation usually needs some additional metadata in the first bytes of the allocated buffer. If the user can know the value of "some bytes" and if the first bytes of a free block of memory are used to fulfill the aligned allocation, the rest of the block can be left also aligned and ready for the next aligned allocation. Note that requesting a size multiple of the alignment is not optimal because lefts the next block of memory unaligned due to the needed metadata.

Once the programmer knows the size of the payload of every memory allocation, he can request a size that will be optimal to allocate aligned chunks of memory maximizing both the size of the request and the possibilities of future aligned allocations. This information is stored in the PayloadPerAllocation constant of managed memory segments.

Here is a small example showing how aligned allocation is used:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <cassert>

int main()
{
   using namespace boost::interprocess;

   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   //Managed memory segment that allocates portions of a shared memory
   //segment with the default management algorithm
   managed_shared_memory managed_shm(create_only, "MySharedMemory", 65536);

   const std::size_t Alignment = 128;

   //Allocate 100 bytes aligned to Alignment from segment, throwing version
   void *ptr = managed_shm.allocate_aligned(100, Alignment);

   //Check alignment
   assert(std::size_t(static_cast<char*>(ptr)-static_cast<char*>(0)) % Alignment == 0);

   //Deallocate it
   managed_shm.deallocate(ptr);

   //Non throwing version
   ptr = managed_shm.allocate_aligned(100, Alignment, std::nothrow);

   //Check alignment
   assert(std::size_t(static_cast<char*>(ptr)-static_cast<char*>(0)) % Alignment == 0);

   //Deallocate it
   managed_shm.deallocate(ptr);

   //If we want to efficiently allocate aligned blocks of memory
   //use managed_shared_memory::PayloadPerAllocation value
   assert(Alignment > managed_shared_memory::PayloadPerAllocation);

   //This allocation will maximize the size of the aligned memory
   //and will increase the possibility of finding more aligned memory
   ptr = managed_shm.allocate_aligned
      (3u*Alignment - managed_shared_memory::PayloadPerAllocation, Alignment);

   //Check alignment
   assert(std::size_t(static_cast<char*>(ptr)-static_cast<char*>(0)) % Alignment == 0);

   //Deallocate it
   managed_shm.deallocate(ptr);

   return 0;
}

	Caution
	This feature is experimental, interface and ABI are unstable

If an application needs to allocate a lot of memory buffers but it needs to deallocate them independently, the application is normally forced to loop calling allocate(). Managed memory segments offer an alternative function to pack several allocations in a single call obtaining memory buffers that:

This allocation method is much faster than calling allocate() in a loop. The downside is that the segment must provide a contiguous memory segment big enough to hold all the allocations. Managed memory segments offer this functionality through allocate_many() functions. There are 2 types of allocate_many functions:

//!Allocates n_elements of elem_bytes bytes.
//!Throws bad_alloc on failure. chain.size() is not increased on failure.
void allocate_many(size_type elem_bytes, size_type n_elements, multiallocation_chain &chain);

//!Allocates n_elements, each one of element_lengths[i]*sizeof_element bytes.
//!Throws bad_alloc on failure. chain.size() is not increased on failure.
void allocate_many(const size_type *element_lengths, size_type n_elements, size_type sizeof_element, multiallocation_chain &chain);

//!Allocates n_elements of elem_bytes bytes.
//!Non-throwing version. chain.size() is not increased on failure.
void allocate_many(std::nothrow_t, size_type elem_bytes, size_type n_elements, multiallocation_chain &chain);

//!Allocates n_elements, each one of
//!element_lengths[i]*sizeof_element bytes.
//!Non-throwing version. chain.size() is not increased on failure.
void allocate_many(std::nothrow_t, const size_type *elem_sizes, size_type n_elements, size_type sizeof_element, multiallocation_chain &chain);

//!Deallocates all elements contained in chain.
//!Never throws.
void deallocate_many(multiallocation_chain &chain);

Here is a small example showing all this functionality:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <boost/move/utility_core.hpp> //boost::move
#include <cassert>//assert
#include <cstring>//std::memset
#include <new>    //std::nothrow
#include <vector> //std::vector

int main()
{
   using namespace boost::interprocess;
   typedef managed_shared_memory::multiallocation_chain multiallocation_chain;

   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   managed_shared_memory managed_shm(create_only,"MySharedMemory", 65536);

   //Allocate 16 elements of 100 bytes in a single call. Non-throwing version.
   multiallocation_chain chain;
   managed_shm.allocate_many(std::nothrow, 100, 16, chain);

   //Check if the memory allocation was successful
   if(chain.empty()) return 1;

   //Allocated buffers
   std::vector<void*> allocated_buffers;

   //Initialize our data
   while(!chain.empty()){
      void *buf = chain.pop_front();
      allocated_buffers.push_back(buf);
      //The iterator must be incremented before overwriting memory
      //because otherwise, the iterator is invalidated.
      std::memset(buf, 0, 100);
   }

   //Now deallocate
   while(!allocated_buffers.empty()){
      managed_shm.deallocate(allocated_buffers.back());
      allocated_buffers.pop_back();
   }

   //Allocate 10 buffers of different sizes in a single call. Throwing version
   managed_shared_memory::size_type sizes[10];
   for(std::size_t i = 0; i < 10; ++i)
      sizes[i] = i*3;

   managed_shm.allocate_many(sizes, 10, 1, chain);
   managed_shm.deallocate_many(chain);
   return 0;
}

Allocating N buffers of the same size improves the performance of pools and node containers (for example STL-like lists): when inserting a range of forward iterators in a STL-like list, the insertion function can detect the number of needed elements and allocate in a single call. The nodes still can be deallocated.

Allocating N buffers of different sizes can be used to speed up allocation in cases where several objects must always be allocated at the same time but deallocated at different times. For example, a class might perform several initial allocations (some header data for a network packet, for example) in its constructor but also allocations of buffers that might be reallocated in the future (the data to be sent through the network). Instead of allocating all the data independently, the constructor might use allocate_many() to speed up the initialization, but it still can deallocate and expand the memory of the variable size element.

In general, allocate_many is useful with large values of N. Overuse of allocate_many can increase the effective memory usage, because it can't reuse existing non-contiguous memory fragments that might be available for some of the elements.

When programming some data structures such as vectors, memory reallocation becomes an important tool to improve performance. Managed memory segments offer an advanced reallocation function that offers:

The expansion can be combined with the allocation of a new buffer if the expansion fails obtaining a function with "expand, if fails allocate a new buffer" semantics.

Apart from this features, the function always returns the real size of the allocated buffer, because many times, due to alignment issues the allocated buffer a bit bigger than the requested size. Thus, the programmer can maximize the memory use using allocation_command.

Here is the declaration of the function:

enum boost::interprocess::allocation_type
{
   //Bitwise OR (|) combinable values
   boost::interprocess::allocate_new        = ...,
   boost::interprocess::expand_fwd          = ...,
   boost::interprocess::expand_bwd          = ...,
   boost::interprocess::shrink_in_place     = ...,
   boost::interprocess::nothrow_allocation  = ...
};


template<class T>
std::pair<T *, bool>
   allocation_command( boost::interprocess::allocation_type command
                     , std::size_t limit_size
                     , size_type &prefer_in_recvd_out_size
                     , T *&reuse_ptr);

Preconditions for the function:

Which are the effects of this function:

Throws an exception if two conditions are met:

This function returns:

Notes:

Here is a small example that shows the use of allocation_command:

#include <boost/interprocess/managed_shared_memory.hpp>
#include <cassert>

int main()
{
   using namespace boost::interprocess;

   //Remove shared memory on construction and destruction
   struct shm_remove
   {
      shm_remove() { shared_memory_object::remove("MySharedMemory"); }
      ~shm_remove(){ shared_memory_object::remove("MySharedMemory"); }
   } remover;

   //Managed memory segment that allocates portions of a shared memory
   //segment with the default management algorithm
   managed_shared_memory managed_shm(create_only, "MySharedMemory", 10000*sizeof(std::size_t));

   //Allocate at least 100 bytes, 1000 bytes if possible
   managed_shared_memory::size_type min_size = 100;
   managed_shared_memory::size_type first_received_size = 1000;
   std::size_t *hint = 0;
   std::size_t *ptr = managed_shm.allocation_command<std::size_t>
      (boost::interprocess::allocate_new, min_size, first_received_size, hint);

   //Received size must be bigger than min_size
   assert(first_received_size >= min_size);

   //Get free memory
   managed_shared_memory::size_type free_memory_after_allocation = managed_shm.get_free_memory();

   //Now write the data
   for(std::size_t i = 0; i < first_received_size; ++i) ptr[i] = i;

   //Now try to triplicate the buffer. We won't admit an expansion
   //lower to the double of the original buffer.
   //This "should" be successful since no other class is allocating
   //memory from the segment
   min_size = first_received_size*2;
   managed_shared_memory::size_type expanded_size = first_received_size*3;
   std::size_t * ret = managed_shm.allocation_command
      (boost::interprocess::expand_fwd, min_size, expanded_size, ptr);
   //Check invariants
   assert(ptr != 0);
   assert(ret == ptr);
   assert(expanded_size >= first_received_size*2);

   //Get free memory and compare
   managed_shared_memory::size_type free_memory_after_expansion = managed_shm.get_free_memory();
   assert(free_memory_after_expansion < free_memory_after_allocation);

   //Write new values
   for(std::size_t i = first_received_size; i < expanded_size; ++i)  ptr[i] = i;

   //Try to shrink approximately to min_size, but the new size
   //should be smaller than min_size*2.
   //This "should" be successful since no other class is allocating
   //memory from the segment
   managed_shared_memory::size_type shrunk_size = min_size;
   ret = managed_shm.allocation_command
      (boost::interprocess::shrink_in_place, min_size*2, shrunk_size, ptr);

   //Check invariants
   assert(ptr != 0);
   assert(ret == ptr);
   assert(shrunk_size <= min_size*2);
   assert(shrunk_size >= min_size);

   //Get free memory and compare
   managed_shared_memory::size_type free_memory_after_shrinking = managed_shm.get_free_memory();
   assert(free_memory_after_shrinking > free_memory_after_expansion);

   //Deallocate the buffer
   managed_shm.deallocate(ptr);
   return 0;
}

allocation_command is a very powerful function that can lead to important performance gains. It's specially useful when programming vector-like data structures where the programmer can minimize both the number of allocation requests and the memory waste.

When mapping a memory segment based on shared memory or files, there is an option to open them using open_copy_on_write option. This option is similar to open_only but every change the programmer does with this managed segment is kept private to this process and is not translated to the underlying device (shared memory or file).

The underlying shared memory or file is opened as read-only so several processes can share an initial managed segment and make private changes to it. If many processes open a managed segment in copy on write mode and not modified pages from the managed segment will be shared between all those processes, with considerable memory savings.

Opening managed shared memory and mapped files with open_read_only maps the underlying device in memory with read-only attributes. This means that any attempt to write that memory, either creating objects or locking any mutex might result in an page-fault error (and thus, program termination) from the OS. Read-only mode opens the underlying device (shared memory, file...) in read-only mode and can result in considerable memory savings if several processes just want to process a managed memory segment without modifying it. Read-only mode operations are limited:

Here is an example that shows the use of these two open modes:

#include <boost/interprocess/managed_mapped_file.hpp>
#include <fstream> //std::fstream
#include <iterator>//std::distance


int main()
{
   using namespace boost::interprocess;

   //Define file names
   const char *ManagedFile  = "MyManagedFile";
   const char *ManagedFile2 = "MyManagedFile2";

   //Try to erase any previous managed segment with the same name
   file_mapping::remove(ManagedFile);
   file_mapping::remove(ManagedFile2);
   remove_file_on_destroy destroyer1(ManagedFile);
   remove_file_on_destroy destroyer2(ManagedFile2);

   {
      //Create an named integer in a managed mapped file
      managed_mapped_file managed_file(create_only, ManagedFile, 65536);
      managed_file.construct<int>("MyInt")(0);

      //Now create a copy on write version
      managed_mapped_file managed_file_cow(open_copy_on_write, ManagedFile);

      //Erase the int and create a new one
      if(!managed_file_cow.destroy<int>("MyInt"))
         throw int(0);
      managed_file_cow.construct<int>("MyInt2");

      //Check changes
      if(managed_file_cow.find<int>("MyInt").first && !managed_file_cow.find<int>("MyInt2").first)
         throw int(0);

      //Check the original is intact
      if(!managed_file.find<int>("MyInt").first && managed_file.find<int>("MyInt2").first)
         throw int(0);

      {  //Dump the modified copy on write segment to a file
         std::fstream file(ManagedFile2, std::ios_base::out | std::ios_base::binary);
         if(!file)
            throw int(0);
       file.write(static_cast<const char *>(managed_file_cow.get_address()), (std::streamsize)managed_file_cow.get_size());
      }

      //Now open the modified file and test changes
      managed_mapped_file managed_file_cow2(open_only, ManagedFile2);
      if(managed_file_cow2.find<int>("MyInt").first && !managed_file_cow2.find<int>("MyInt2").first)
         throw int(0);
   }
   {
      //Now create a read-only version
      managed_mapped_file managed_file_ro(open_read_only, ManagedFile);

      //Check the original is intact
      if(!managed_file_ro.find<int>("MyInt").first && managed_file_ro.find<int>("MyInt2").first)
         throw int(0);

      //Check the number of named objects using the iterators
      if(std::distance(managed_file_ro.named_begin(),  managed_file_ro.named_end())  != 1 &&
         std::distance(managed_file_ro.unique_begin(), managed_file_ro.unique_end()) != 0 )
         throw int(0);
   }
   return 0;
}

Sometimes, the user wants to create simple objects, STL compatible containers, STL compatible strings and more, all in a single buffer. This buffer could be a big static buffer, a memory-mapped auxiliary device or any other user buffer.

This would allow an easy serialization and we-ll just need to copy the buffer to duplicate all the objects created in the original buffer, including complex objects like maps, lists.... Boost.Interprocess offers managed memory segment classes to handle user provided buffers that allow the same functionality as shared memory classes:

//Named object creation managed memory segment
//All objects are constructed in a user provided buffer
template <
            class CharType,
            class MemoryAlgorithm,
            template<class IndexConfig> class IndexType
         >
class basic_managed_external_buffer;

//Named object creation managed memory segment
//All objects are constructed in a user provided buffer
//   Names are c-strings,
//   Default memory management algorithm
//    (rbtree_best_fit with no mutexes and relative pointers)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_external_buffer <
   char,
   rbtree_best_fit<null_mutex_family, offset_ptr<void> >,
   flat_map_index
   >  managed_external_buffer;

//Named object creation managed memory segment
//All objects are constructed in a user provided buffer
//   Names are wide-strings,
//   Default memory management algorithm
//    (rbtree_best_fit with no mutexes and relative pointers)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_external_buffer<
   wchar_t,
   rbtree_best_fit<null_mutex_family, offset_ptr<void> >,
   flat_map_index
   >  wmanaged_external_buffer;

To use a managed external buffer, you must include the following header:

#include <boost/interprocess/managed_external_buffer.hpp>

Let's see an example of the use of managed_external_buffer:

#include <boost/interprocess/managed_external_buffer.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <boost/interprocess/containers/list.hpp>
#include <cstring>
#include <boost/aligned_storage.hpp>

int main()
{
   using namespace boost::interprocess;

   //Create the static memory who will store all objects
   const int memsize = 65536;

   static boost::aligned_storage<memsize>::type static_buffer;

   //This managed memory will construct objects associated with
   //a wide string in the static buffer
   wmanaged_external_buffer objects_in_static_memory
      (create_only, &static_buffer, memsize);

   //We optimize resources to create 100 named objects in the static buffer
   objects_in_static_memory.reserve_named_objects(100);

   //Alias an integer node allocator type
   //This allocator will allocate memory inside the static buffer
   typedef allocator<int, wmanaged_external_buffer::segment_manager>
      allocator_t;

   //Alias a STL compatible list to be constructed in the static buffer
   typedef list<int, allocator_t>    MyBufferList;

   //The list must be initialized with the allocator
   //All objects created with objects_in_static_memory will
   //be stored in the static_buffer!
   MyBufferList *list = objects_in_static_memory.construct<MyBufferList>(L"MyList")
                           (objects_in_static_memory.get_segment_manager());
   //Since the allocation algorithm from wmanaged_external_buffer uses relative
   //pointers and all the pointers constructed int the static memory point
   //to objects in the same segment,  we can create another static buffer
   //from the first one and duplicate all the data.
   static boost::aligned_storage<memsize>::type static_buffer2;
   std::memcpy(&static_buffer2, &static_buffer, memsize);

   //Now open the duplicated managed memory passing the memory as argument
   wmanaged_external_buffer objects_in_static_memory2
      (open_only, &static_buffer2, memsize);

   //Check that "MyList" has been duplicated in the second buffer
   if(!objects_in_static_memory2.find<MyBufferList>(L"MyList").first)
      return 1;

   //Destroy the lists from the static buffers
   objects_in_static_memory.destroy<MyBufferList>(L"MyList");
   objects_in_static_memory2.destroy<MyBufferList>(L"MyList");
   return 0;
}

Boost.Interprocess STL compatible allocators can also be used to place STL compatible containers in the user segment.

basic_managed_external_buffer can be also useful to build small databases for embedded systems limiting the size of the used memory to a predefined memory chunk, instead of letting the database fragment the heap memory.

Note: The external memory supplied by the user shall be aligned to the maximum value between alignof(max_align_t) and the alignment of the strictest over-aligned type to be built inside that memory.

The use of heap memory (new/delete) to obtain a buffer where the user wants to store all his data is very common, so Boost.Interprocess provides some specialized classes that work exclusively with heap memory.

These are the classes:

//Named object creation managed memory segment
//All objects are constructed in a single buffer allocated via new[]
template <
            class CharType,
            class MemoryAlgorithm,
            template<class IndexConfig> class IndexType
         >
class basic_managed_heap_memory;

//Named object creation managed memory segment
//All objects are constructed in a single buffer allocated via new[]
//   Names are c-strings,
//   Default memory management algorithm
//    (rbtree_best_fit with no mutexes and relative pointers)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_heap_memory <
   char,
   rbtree_best_fit<null_mutex_family>,
   flat_map_index
   >  managed_heap_memory;

//Named object creation managed memory segment
//All objects are constructed in a single buffer allocated via new[]
//   Names are wide-strings,
//   Default memory management algorithm
//    (rbtree_best_fit with no mutexes and relative pointers)
//   Name-object mappings are stored in the default index type (flat_map)
typedef basic_managed_heap_memory<
   wchar_t,
   rbtree_best_fit<null_mutex_family>,
   flat_map_index
   >  wmanaged_heap_memory;

To use a managed heap memory, you must include the following header:

#include <boost/interprocess/managed_heap_memory.hpp>

The use is exactly the same as basic_managed_external_buffer, except that memory is created by the managed memory segment itself using dynamic (new/delete) memory.

basic_managed_heap_memory also offers a grow(std::size_t extra_bytes) function that tries to resize internal heap memory so that we have room for more objects. But be careful, if memory is reallocated, the old buffer will be copied into the new one so all the objects will be binary-copied to the new buffer. To be able to use this function, all pointers constructed in the heap buffer that point to objects in the heap buffer must be relative pointers (for example offset_ptr). Otherwise, the result is undefined. Here is an example:

#include <boost/interprocess/containers/list.hpp>
#include <boost/interprocess/managed_heap_memory.hpp>
#include <boost/interprocess/allocators/allocator.hpp>
#include <cstddef>

using namespace boost::interprocess;
typedef list<int, allocator<int, managed_heap_memory::segment_manager> >
   MyList;

int main ()
{
   //We will create a buffer of 1000 bytes to store a list
   managed_heap_memory heap_memory(1000);

   MyList * mylist = heap_memory.construct<MyList>("MyList")
                        (heap_memory.get_segment_manager());

   //Obtain handle, that identifies the list in the buffer
   managed_heap_memory::handle_t list_handle = heap_memory.get_handle_from_address(mylist);

   //Fill list until there is no more memory in the buffer
   BOOST_TRY{
      while(1) {
         mylist->insert(mylist->begin(), 0);
      }
   }
   BOOST_CATCH(const bad_alloc &){
      //memory is full
   } BOOST_CATCH_END
   //Let's obtain the size of the list
   MyList::size_type old_size = mylist->size();

   //To make the list bigger, let's increase the heap buffer
   //in 1000 bytes more.
   heap_memory.grow(1000);

   //If memory has been reallocated, the old pointer is invalid, so
   //use previously obtained handle to find the new pointer.
   mylist = static_cast<MyList *>
               (heap_memory.get_address_from_handle(list_handle));

   //Fill list until there is no more memory in the buffer
   BOOST_TRY{
      while(1) {
         mylist->insert(mylist->begin(), 0);
      }
   }
   BOOST_CATCH(const bad_alloc &){
      //memory is full
   } BOOST_CATCH_END

   //Let's obtain the new size of the list
   MyList::size_type new_size = mylist->size();

   assert(new_size > old_size);

   //Destroy list
   heap_memory.destroy_ptr(mylist);

   return 0;
}

Note: Heap memory is not guaranteed to be compatible with over-aligned types so it is not supported to use memory allocation algorithms or construct objects that need over-aligned memory:

// 64 byte aligned memory might not be supported
typedef basic_managed_heap_memory<
    char,
    rbtree_best_fit<mutex_family, offset_ptr<void, long, unsigned long, 64>, 64>,
    iset_index
> managed_heap_with_overalignment_t;

All managed memory segments have similar capabilities (memory allocation inside the memory segment, named object construction...), but there are some remarkable differences between managed_shared_memory, managed_mapped_file and managed_heap_memory, managed_external_file.

To see the utility of managed heap memory and managed external buffer classes, the following example shows how a message queue can be used to serialize a whole database constructed in a memory buffer using Boost.Interprocess, send the database through a message queue and duplicated in another buffer:

//This test creates a in memory data-base using Interprocess machinery and
//serializes it through a message queue. Then rebuilds the data-base in
//another buffer and checks it against the original data-base
bool test_serialize_db()
{
   //Typedef data to create a Interprocess map
   typedef std::pair<const std::size_t, std::size_t> MyPair;
   typedef std::less<std::size_t>   MyLess;
   typedef node_allocator<MyPair, managed_external_buffer::segment_manager>
      node_allocator_t;
   typedef map<std::size_t,
               std::size_t,
               std::less<std::size_t>,
               node_allocator_t>
               MyMap;

   //Some constants
   const std::size_t BufferSize  = 65536;
   const std::size_t MaxMsgSize  = 100;

   //Allocate a memory buffer to hold the destiny database using vector<char>
   std::vector<char> buffer_destiny(BufferSize, 0);

   message_queue::remove(test::get_process_id_name());
   {
      //Create the message-queues
      message_queue mq1(create_only, test::get_process_id_name(), 1, MaxMsgSize);

      //Open previously created message-queue simulating other process
      message_queue mq2(open_only, test::get_process_id_name());

      //A managed heap memory to create the origin database
      managed_heap_memory db_origin(buffer_destiny.size());

      //Construct the map in the first buffer
      MyMap *map1 = db_origin.construct<MyMap>("MyMap")
                                       (MyLess(),
                                       db_origin.get_segment_manager());
      if(!map1)
         return false;

      //Fill map1 until is full
      BOOST_TRY{
         std::size_t i = 0;
         while(1){
            (*map1)[i] = i;
            ++i;
         }
      }
      BOOST_CATCH(boost::interprocess::bad_alloc &){} BOOST_CATCH_END

      //Data control data sending through the message queue
      std::size_t sent = 0;
      message_queue::size_type recvd = 0;
      message_queue::size_type total_recvd = 0;
      unsigned int priority;

      //Send whole first buffer through the mq1, read it
      //through mq2 to the second buffer
      while(1){
         //Send a fragment of buffer1 through mq1
       std::size_t bytes_to_send = MaxMsgSize < (db_origin.get_size() - sent) ?
                                       MaxMsgSize : (db_origin.get_size() - sent);
         mq1.send( &static_cast<char*>(db_origin.get_address())[sent]
               , bytes_to_send
               , 0);
         sent += bytes_to_send;
         //Receive the fragment through mq2 to buffer_destiny
       mq2.receive( &buffer_destiny[total_recvd]
                , BufferSize - recvd
                  , recvd
                  , priority);
         total_recvd += recvd;

         //Check if we have received all the buffer
         if(total_recvd == BufferSize){
            break;
         }
      }

      //The buffer will contain a copy of the original database
      //so let's interpret the buffer with managed_external_buffer
      managed_external_buffer db_destiny(open_only, &buffer_destiny[0], BufferSize);

      //Let's find the map
      std::pair<MyMap *, managed_external_buffer::size_type> ret = db_destiny.find<MyMap>("MyMap");
      MyMap *map2 = ret.first;

      //Check if we have found it
      if(!map2){
         return false;
      }

      //Check if it is a single variable (not an array)
      if(ret.second != 1){
         return false;
      }

      //Now let's compare size
      if(map1->size() != map2->size()){
         return false;
      }

      //Now let's compare all db values
     MyMap::size_type num_elements = map1->size();
     for(std::size_t i = 0; i < num_elements; ++i){
         if((*map1)[i] != (*map2)[i]){
            return false;
         }
      }

      //Destroy maps from db-s
      db_origin.destroy_ptr(map1);
      db_destiny.destroy_ptr(map2);
   }
   message_queue::remove(test::get_process_id_name());
   return true;
}