Resource management can be a challenge for Direct3D applications. Before DirectX, display adapters had limited amounts of fast device memory and programmers had to manage that memory manually in order to achieve the best performance. The Direct3D API provides different memory pools to allow the application to specify the lifetime management policy of a resource.
The available resource pools are: default, managed, system memory and scratch. Resources in the default pool live only on the device and must be restored to the device when it is lost. Resources in the managed pool contain a system memory backing store for the resource and are spooled into device memory as they are needed by the runtime. Allowing the runtime to spool the resource onto the device when it is needed simplifies resource management for the application.
Suppose your application has more textures than will fit in device memory. They cannot all be allocated into the default resource pool. Your application will need to create textures at startup that are sufficiently large to handle the largest texture used by your application. They will also need to be created with the appropriate usage flags (render target usage, sRGB usage, etc.) and format.
Worse, you may need to destroy and create textures on the fly due to resource exhaustion. Destroying and creating resources is expensive and can cause a stall in your real-time rendering frame rate. You’ll need to write a bunch of infrastructure code in order to manage the limited amount of device memory for your resources. You’ll probably want a priority system to specify which resources are more important than others and you’ll want a prefetch system to load resources into device memory in advance of when they will be needed to avoid stalling the pipeline.
Congratulations, you’ve just re-implemented the managed resource pool!
The managed resource pool is even better than what I’ve described above. Resources in the managed pool persist across a device reset or a device lost condition. You don’t need to worry about creating a single resource that is big enough to hold your largest asset of a particular format and usage combination; you can simply create managed resources with the desired usage and format. The system memory backing store for a managed resource is used as the source of data when the resource needs to be moved into device memory. The system memory backing store also allows the application to update the resource by modifying the backing store and allowing the runtime to update the changed data on the device as needed.
The SetPriority and PreLoad methods on the IDirect3DResource9 interface let you fine tune the resource management behavior for an individual resource. All managed resources can be evicted from device memory using the EvictManagedResources method on the IDirect3DDevice9 interface.
Windows Vista and Windows 7
The situation is slightly different for devices using the Windows Vista display driver model (WDDM) under Windows Vista or Windows 7. Under the WDDM, device memory is virtualized and the GPU can page in resources on demand similar to the way memory is paged in from the swap file on demand for the CPU. With a WDDM device, you can use IDirect3DDevice9Ex to create resources in the default pool that won’t be lost on a device reset or device lost condition. In effect, the default resource pool becomes equivalent to the managed resource pool in this situation.
Legalize Adulthood! 
12-October-2009 at 6:47 am
“Before DirectX, display adapters had limited amounts of fast device memory and programmers had to manage that memory manually in order to achieve the best performance.”
Uhm… OpenGL?
LikeLike
12-October-2009 at 10:27 am
What about OpenGL? The hardware hasn’t supported virtualization of graphics memory until Direct3D 10. OpenGL doesn’t change that. If you were using OpenGL and wanted to keep all your textures resident on the card for maximum performance, you had to manage that yourself.
LikeLike
13-October-2009 at 1:37 am
Well yes and no. Yes virtualization of graphics memory is something new in Direct3D 10. No, you don’t need to manage that yourself in OpenGL. At most you have to be careful how much texture memory you’re using otherwise OpenGL would be transferring textures back and forth all the time, causing performance degradation. Unlike D3D, OpenGL never had any problems with ‘lost devices’ which required you to upload all your textures again.
So unless i’m completely misunderstanding you here or something, you’re mistaken about this.
LikeLike
12-October-2009 at 8:55 am
That eats your process address space, thought. Right?
LikeLike
12-October-2009 at 10:25 am
If these are resources that you need to restore when the device is reset or lost, you’ll need to have a copy in system memory in order to restore them. Either that or you’ll have to go out to disk and fetch in the resources from disk in order to get them in system memory so that you can upload them to the device. Fetching from disk when your device is restored after being lost isn’t exactly going to be the fastest.
Using the managed pool doesn’t mean you don’t have to think about resource management, but it does simplify things. Use the managed pool until you’ve actually measured that its a problem in your code.
LikeLike
12-October-2009 at 4:08 pm
Keeping resources in memory and fragmenting address space just in case of device loss/reset is just not practical for large projects (as long as we are talking 32-bit, of course).
Loading textures form disk is a far better option. Even though it might be slower than using managed pool.
Mainly because waiting 20 or so seconds for your textures to stream in after a device reset (which is a relatively rare event) is better than failing to HeapAlloc and crashing.
To illustrate the issue: http://kayru.org/wp-content/uploads/2009/10/default_vs_managed_vm.png
Bottom line is: I would not recommend managed pool textures for any kind of large 32-bit application.
LikeLike
15-October-2009 at 6:17 am
Yes, this is called managing it yourself in order to obtain the best performance.
At any rate, talking about OpenGL is irrelevant since this is a Direct3D programming tip, not an OpenGL programming tip.
Everyone starts with a small project that becomes larger over time. You may never get to the point you are talking about, in which case had you implemented your own resource management system you would have wasted precious engineering resources, violating YAGNI.
Do the simplest thing that could possibly work. You are not required to use the managed resource pool, but until you need to do things differently, it is the simplest thing that could possibly work.
LikeLike
16-October-2009 at 9:08 am
> Everyone starts with a small project that becomes larger over time.
I think that your tech choices must be guided by your game design document.
If your project is a 2d puzzle game, you are unlikely to get any kind of resource problems.
However if your game is something like ours (www.totalwar.com), you can pretty much guarantee that you will run out of something at some point. In this case it is better to prepare for it from the start. Otherwise you risk running out of memory right before release.
> Do the simplest thing that could possibly work.
I agree with this statement for most parts of the game. Except memory or other kind of resource management.
LikeLike
16-October-2009 at 10:40 am
Starting small and becoming larger over time doesn’t mean you can’t plan for what you know you will need before you’re done.
Doing the simplest thing that could possibly work also doesn’t mean you don’t plan for what you know you will need.
So in both cases, neither of these guiding principles is at odds with what you are doing.
However, what I see over and over again is that everyone thinks they are in your situation when most of the time they aren’t. For some reason game programmers think that they must create everything from scratch. Most games that are sold don’t even need anywhere near the complexity of the infrastructure that goes into something like Gears of War, but they all act like they will need that.
If you have game industry experience and you know you will need it, then YAGNI doesn’t apply — because you do know that you need it.
However, if you’ve never written a game before, or you are not sure if your game is going to have resource exhaustion problems, then YAGNI says its wasted effort to build infrastructure you may not ever need.
When I look at the Xbox 360 games I have on my shelf or that I have downloaded, only a handfull of them are in the resource exhaustion demands custom infrastructure category. Those are games created by large teams with multi-million dollar development budgets. I don’t think those teams are the ones reading my blog to figure out how to develop their games, although they may find some of my observations interesting.
For the indy game developer or small team, my advice to use the managed pool still stands. Those teams are development time limited and spending a critical resource developing something they may not need is a mistake. Particularly if you are a new game developer, you should not be spending your time on this type of infrastructure activity because not only are you unlikely to need it, you are quite likely to get it wrong if you try to build it.
Its been said that Walt Disney shunned any new animation technology that didn’t fit the story of the movie/short they were making. I think that for game developers, particularly small and indy game developers, they should be focused on gameplay and not infrastructure. Using the managed pool is a choice that supports the idea of spending your time creating great gameplay first.
Who cares how great your infrastructure management is if the gameplay sucks?
LikeLike
23-May-2010 at 12:32 am
I agree with legalize, use the simplest thing that works and design something new when it stops working.
LikeLike