Lightning Engine

Inspired by a series of articles by Jelle van der Beek ([1-3]), as well as the implementation by Maciej Sinilo ([4]), we’ve decided to write an application for tracking memory usage and statistics of unmanaged applications. Its primary use would be to monitor the engine’s memory requirements and tracking down memory leaks.

The prototype is based on the idea of having a DLL (the client) staticaly linked to each and every module you want to monitor (ala Visual Leak Detector). The client intercepts all memory related functions (malloc/realloc/free, as well as scalar and vector versions of new and delete, both from msvcrXX.dll and mfcXX.dll), and sends messages to a server application through RPCs-over-tcp. The RPCs are synchronous at the moment, but since they don’t do a lot of work it shouldn’t be a big problem. Processing of all messages is done by the server, outside of the RPCs, in order to keep the overhead to the minimum.

Following the useful tips given in [5], the real-time view of the server displays only statistics about the current state of the monitored application (see figures below). Actual processing of all messages is performed on demand, from the other views of the application. Currently, the server takes up to about 2 – 4% of CPU time, on a Q6600, when displaying the real-time view (which is updated constantly). The main problem right now is the required network bandwidth to transfer callstack information from the client to the server, because we aren’t compressing the callstack in any way (except from limiting the number of frames). This might actually appear as a big problem when the server is running on a different PC, which would require actual data transfer over a wire.

The project is still in alpha version, and there are a lot of things to be tested and added. I.e. currently, the user has the ability to save all the information to a file, at any given time, and load it later for inspection. It would be nice if there was a mechanism of easiliy comparing two dumps, and finding differences between them (e.g. track the memory allocated by a specific action in the monitored application). Other things we are currently planning (in no specific order):

• Addition of another tab, for finding the “critical path” of your application, like “CallGraph analysis” view from [1].
• An API for communicating between the monitored application and the server. This way, the user would be able to add markers inside the code (e.g. FrameStart and FrameEnd) in order to track down the memory used in a specific frame, or the subsystem which made the allocations (like in [6]).
• Detection of buffer overruns, by adding guard bytes at the end of every allocation

If you find such application useful, we’ll consider releasing a demo/beta version in the future. But first we must fix some minor but annoying bugs.

Thanks for reading. As always, comments are welcome.

References:
[1] van der Beek, J., Monitoring Your PC’s Memory Usage For Game Development”
[2] van der Beek, J., Monitoring Your Console’s Memory Usage, Part One
[3] van der Beek, J., Monitoring Your Console’s Memory Usage, Part Two
[4] Sinilo, M., Posts about MemTracer at .mischief.mayhem.soap.
[5] Sinilo, M., MemTracer strikes back
[6] Lewis, M., Managing Your Memory – Part III : Tracking and Reporting Allocation Information

Tags: memory_tracking

This entry was posted on Sunday, January 18th, 2009 at 2:14 pm and is filed under memory_tracking. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.