Runtime code reloading in D, part 1

One of the biggest problems in game developement are turnaround times. Tournaround time is the time you have to wait until you can see what your change actually did. Shorter turnaround times improve productivity as you don’t spend that much time waiting for changes to become visible. Also shorter turnaround times support the creative process as humans have huge dificulties connecting an action and a result that happens with a huge delay. For mappers, 3d artists, effect artists and sound designers the turnaround times in modern 3d engines are almost zero. A lot of work is put into their tools to keep turnaround times to a minimum. Programmers however usually have very long turnaround times. Mostly this is due to the fact that compile times of the C++ language, which is mostly used in game developement, can go into multipe minutes. This problem is almost solved in the D programming language, as compile times are really quick and it usually only takes a few seconds to compile a project. But after making changes to the code you still have to restart the game, wait for it to load all the resources and then fly/walk to the spot in the level you are currently interrested in. I got really tired of this process so I decided to build a runtime reloading for certian parts of my own little game engine.

Game simulations in general have nice properties for runtime code reloading. The simulation is performed in steps, and after each step there usually is a synchronisation point. At that point you exactly know what part of your code is currently executing, so this is ideal to patch other parts of the code as it is currently not executing. For a start I wanted to build a simple system that allows reloading modified algorithms but does not support changes in the structure of the data. This has the advantage that objects remain at the same position on the heap and only the vptrs have to be patched. This makes patching easier for a start.

Runtime type information

To be able to patch existing object instances you will usually need some kind of runtime type infomration (RTTI) system. Usually you need to know

  • Where are the vptrs inside the class
  • Which members point to other objects

The first point is already covered by the D’s TypeInfo instances that exist for every type at runtime. To have information about the members I wrote my own little RTTI system that uses the RTInfo template inside object_.d. This template is instanciated for every type that is used during compilation and thus is ideal to generate RTTI information. At runtime the generated information can be accessed via the “rtInfo” member of the respective “TypeInfo” object. At some point this template will be used for a percise garbage collector but at the time of this writing it is unused (dmd 2.061). Additionally I’m not using D’s GC so even if it is used at some point, I can replace it with my own version because I won’t need percise GC data.

The RTTI info stored for each member looks as follows:

I use D’s compile time function execution (CTFE) to generate the code which creates the RTTI info. Note the “static if” statements that avoid cyclic references.

For the class “RigidBody”:

This will produce the following RTTI information:

The first line of the RTTI info contains the mangeled name of the type this information was generated for, the TypeInfo object for the current type and the size for the current type. The manageld name can be used as a unique identifier when building lookup tables for type information. The remaining entries of the array contain information about all members (including the members of the base classes) in the following order:

  • Member name
  • member type info
  • member size
  • pointer to the RTTI information for this member

The pointer to the RTTI information for each member is a workaround for a strange bug I encountered. For POD structs the rtInfo template will be instancated, but for some reason you can not query the results at runtime. It is possible that the linker stripps the information away but I could not yet investigate this issue further. If I can not query the RTTI information at runtime I fallback to the RTTI info pointer stored in the “next” member of thMemberInfo.

To trigger the generation of the RTTI info the following code is used:

And the RTInfo template inside object_.d and object.di looks like this:

As soon as you pass the “RTTI” version identifier to the compiler all types will have RTTI generated for them. As you might notice I only generate RTTI for structs and classes with the exception of classes that are dervived from TypeInfo. TypeInfo classes are ignored because you don’t really need reflection data for the refelection data ;-).

Plugin system

To be able to actually reload code at runtime, I did split of the part of the code that should reloadable into a own dll. This was quite compilcated as D does currently not support shared dlls and I had to make small modifications to the runtime to allow for this feature. I might write another article about this in the future.
All calls into this dlls are done over D interfaces, which means that they are virtual and always go through a vtable. By patching the vptrs all these calls can be redirected to the new code. Each plugin also has a few “roots”. For the system to work, all objects that have been allocated by a plugin should be findable by wakling down the object hirarchy from the roots of a plugin.

Patching

To be able to recompile and reload the dll at run time, the dll and its debugging info (.pdb file) is copied to a temporary subfolder before it is loaded with a LoadLibrary call. To detect modifications to the plugin dll ReadDirectoryChangesW is used. (see my standard library on github). After a modification has been detected the reloading works as follows:

  • Wait for the end of the current simulation cycle
  • Copy the dll and pdb file to a new subdirectory
  • Load the new dll
  • Walk the type hirarchy of the new dll and build a dictionary of all type information objects
  • Iterate over all objects allocated by the old dll and patch them
  • Continue the simulation

The old plugin remains loaded to keep references to static or TLS memory valid. Each plugin has a small proxy allocator that keeps a list of all allocations done by a plugin so that iterating over them becomes possible. Refernces to objects that are not allocated by the plugin that is currently reloaded remain untouched.

Building a type info dictionary

The following function runs down the type hirarchy to build a hashmap with all type info objects for the newly loaded plugin. Please note that it does not operate on actual object instances but merely on the pure type information. This gets a lot easier with a small extension to D’s TypeInfo classes that has been submitted as a pull request to the regular druntime at the point of this writing: druntime pull request 370. This exstension adds a “type” property to all TypeInfo classes which returns the type of the TypeInfo object as a enum value. This has the advantage that it works flawlessly across dll bounadries which would not be possible with the “is” operator. This is due to the fact that currently the TypeInfo instances are duplicated for each dll, which results in different addresses for instances of the same TypeInfo object across dll boundaries.

Patching the objects

After a dictionary of all type info objects has been build the runtime objects can be patched. All runtime objects are iterated by running down the object hirarchy starting at the plugin roots. For each plugin it is checked if it was allocated by the object to be patched, and if yes its new type information is recieved from the hashmap and all vptrs are patched. This is quite easy because D’s TypeInfo system already provides enough information to locate all vptrs. The memory layout of a D object is as follows (tested with DMD 2.060, no garantuees for correctness):

memory layout of a D object

We are only interrested in the location of the vptrs. The object vptr is always at the very start of the instance so we can easily patch that.
The location of all interface vptrs can obtained from the “interfaces” member of the TypeInfo_Class type information. Additionally the inheritance chain has to be walked until the very top is reached to find all interface vptrs. The new values for all vptrs can be obtained from the TypeInfo “init” property. Which is a void[] containing the values a class instance should be initialized with. This will result in the following code for patching interface vptrs.

The code for patching a entire object hirarchy looks currently like this:

For my current physics simulation plugin this is already enough to reload it at runtime. Usually I have two visual studio instances open at the same time. One running the main application in the debugger. I use this instance to debug issues and make changes. When the changes are done I switch to the second visual studio instance which only has a solution with the physics simulation plugin in it. This visual studio instance is only used to recompile the plugin. When I then give the focus back to the game application, it checks for modified files, reloads the plugin because it was modified and then I can see the results of my changes right away.

There are obviously many issues left to be addressed. The next one I will focus on is reloading a plugin, which changed the structure of some of its classes. I plan to serialize all class instances and deserialize them again with the new plugin code. As the address of all these instances will change all references to these instances have to be found and patched, witch will be the hard part of it.

I recognize that the solution presented here only works with all the preconditions I listed and might not be aplicable for other fields of software developement, which is a huge downside. Despite this, I hope that it holds some interresting information for other fields of software developement too.