Multi-cage mode and multiple sandboxes

Multi-cage mode and multiple sandboxes

Hello colleagues, recently I’ve been working on some interesting parts of V8 and I want to announce this work to the public because it could be useful for deno, node, and other folks who are working with V8 embedding.

Pointer compression problems

So, let’s imagine that you are using V8 to execute JS on the server side, and let’s say you handle HTTP requests. Each request is some JS code and so, to handle it you start a new V8 isolate (or use an existing one) and execute the request’s code in that isolate. When there are only a few clients there is no problem but when the number of clients is becoming bigger you naturally want to reduce memory consumption of all requests.

You start searching and there is already an answer for that in V8 - pointer compression. In general, pointer compression allows you to save up to 50% of memory and speed up GC but there is a limitation - pointer compression limits the V8 heap to a maximum of 4 gigabytes. By the way, V8 uses the special term for this shared between all isolate heaps space, it is called a pointer cage (pic. 1), and this mode of V8 is correspondingly called the shared-cage mode.

If each request uses 2 megabytes of heap, V8’s pointer compression would limit you to being able to handle only 2048 isolates at the same time in one process. Obviously, this number is not promising so folks just don’t use pointer compression and can’t obtain any value from it. In Chromium this isn’t a problem because one isolate corresponds to one tab and one process so 4Gb for one tab is completely ok. But what if we could relax this limitation and get all the benefits of pointer compression?

Isolate groups

The answer is - isolate groups or the so-called multi-cage mode. So, we can’t extend the limit of 4GB, however, there are some ways to make it 8GB but 8GB is not acceptable too because we want to allocate as much memory as we have without limits. Since each request consumes far less memory than 4Gb and they are all independent we could add an ability to allocate more pointer cages. If we had this ability to create new pointer cages we could still use pointer compression and save memory but at the same time, we could allocate as many cages as we want. So, we added this ability to V8 and it is called multi-cage mode.

On the API level, we introduced a new entity inside the V8 ecosystem - an isolate group. An isolate group represents one independent pointer cage where you can allocate all your isolates. The API looks like this:

v8::IsolateGroup group = v8::IsolateGroup::Create();
v8::Isolate* isolate = v8::Isolate::New(group, create_params);

The introduction of isolate groups relaxes the limitation for 4Gb because now one can allocate as many isolate groups per single process as one wants and each group will use pointer compression.

Isolate groups extend V8’s API in a compatible way by introducing an implicit default isolate group. These two lines are therefore equivalent:

v8::Isolate* isolate = v8::Isolate::New(create_params);
v8::Isolate* isolate = v8::Isolate::New(IsolateGroup::GetDefault(), create_params);

In this way, embedders can write code that is agnostic with regards to the various configurations of V8: shared-cage, multi-cage and configurations without pointer compression.

Security

Some time after introducing pointer compression the V8 team introduced the sandbox, a security feature based on pointer compression. The sandbox is a terabyte of virtual memory that stores the V8 heap pointer cage as well as backing stores for WebAssembly memories and ArrayBuffers. References to sandboxed objects are replaced with offsets from the base; the idea is pretty similar to the pointer compression.

One of the main ideas of the sandbox is to ban all native pointers from the cage so hackers like you can’t corrupt memory outside of the sandbox. In practice, it means that if you want to obtain a native pointer to some C++ object from your JS object you need to go through a special security border - external pointer tables to translate offset from sandbox to the real address.

Multiple sandboxes

So, the sandbox is based on pointer compression and shared-cage mode and therefore can’t be used in multi-cage mode. At the same time, it is a cool security feature and it will be nice to have it for multi-cage mode too. Our isolate groups work enables this use case by arranging that each isolate group get its own sandbox.

However, there is one limitation, since the sandbox is huge by default (1 TB!), you can’t allocate many of them per process because of limited virtual address space. Therefore heavy users of multiple sandboxes may need to use a compile-time parameter that controls the default size of the sandbox. By reducing sandbox size alongside limiting the maximum index for ArrayBuffers you can archive full security properties of the sandbox with the needed amount of sandboxes per process.

The API for creating isolates is the staying the same only behaviour of V8 is changed - each time you allocate a new isolate group with enabled sandbox a new sandbox will be allocated. The only new thing is that you can now obtain sandbox from isolate group:

v8::IsolateGroup group = v8::IsolateGroup::Create();
group->GetSandbox();

Summary

To sum up, now with the introduction of isolate groups and sandbox support V8 embedders could enjoy pointer compression alongside all the benefits of the newly added sandbox feature.

You can use it with the following gn args:

v8_enable_sandbox = true
v8_enable_pointer_compression = true
v8_enable_pointer_compression_shared_cage = false

Special thanks section

This work was sponsored and motivated by Cloudflare, in particular by James M Snell, Kenton Varda, Eric Corry and co.

Implementation was done by the Igalia compiler team, in particular by Andy Wingo and me.