XML Database for Unity (Part 2)

Asset Bundle Setup
Asset Bundle Setup

Asynchronous Asset Loading using XML Database

In Part 1 of this article, we saw how game data can be defined in C#, authored in XML, and stored in a database at runtime.  This provides an easy and extensible way to manage content in your game which lives outside of the Unity scene graph.  If you have not already done so, download the example projects here.  This article will use the AssetBundleExample project.

In Part 2, we will apply this technique to build an asynchronous resource loading pipeline using Asset Bundles.

Unity provides two APIs for loading assets in this manner: Resources.Load<T>() and Asset Bundles.  Let’s review those first, then build on top of them.

Resources.Load<T>()

The easiest way to load an asset is with the Resources.Load API.  Resources.Load<T>() will load an asset given a path relative to the Resources directory in your project, and Resources.UnloadAsset() or Resources.UnloadUnusedAssets() will free the memory.  This API works well, but has some limitations:

  • Any object you want to load must be in a Resources directory in your project and will be included in the build.  Unlike direct asset references, Unity will not perform any dependancy tracking and will include all assets in your Resources directory, even if you aren’t using them in your game.
  • You also cannot append to this collection of resources after the game is built – you have to make another build to add new ones.
  • Loading is a blocking operation.  Resources.Load will block the game’s main thread while the asset finishes loading.  This is bad if the asset being loaded is large, as your game will appear to hitch while it’s loading.

Asset Bundles

Unity provides a more sophisticated means of loading assets in the form of Asset Bundles.  Asset Bundles are packages of assets which you can build via the BuildPipeline API.  These bundles can be downloaded by (or shipped with) your Unity game and assets can be loaded from them.  They can also be downloaded, allowing you to add assets to your game after it has shipped.  Unlike Resources.Load, loading from an Asset Bundle can be done asynchronously, preventing your game from stalling while the load completes.

Asset Bundles are really great, but they’re a very boilerplate feature.  We have to write code that decides which bundles to build, and what goes in them.  We also need to write code which understands what is in our bundles when we receive them, and is able to load the correct assets from the correct bundle.  The lower-level nature of asset bundles may be off-putting for some, but as we will see, it gives us a lot of flexibility to control exactly how our assets are organized.

A quick note before we go any further: Asset Bundles are a Pro only feature.  If you’re a free Unity user, the rest of this article may still be useful to you, so please read on!

Asset Bundles + XML Database

The primary challenge when creating an Asset Bundle build pipeline is specifying which assets go in which bundles.  This information needs to be present in both the Editor when the bundles are created, and because Asset Bundles don’t contain any metadata about what is contained in them, we need that information in the game as well.  It is up to us to track all of the information about our bundles.  This is a perfect use for our XML database system.

First, we need to define our data.  There are two types of information we’ll need: what bundles there are, and what assets are in each bundle.  In Part 1 of this article, we saw how entries in our database can reference each other using the DatabaseEntryRef data type.  Let’s build two classes to represent information about each Asset Bundle and each asset – the asset referencing the bundle.  These classes are found in the Common/Script/ResourceManager/ directory.

Each of these info classes is very simple.  AssetBundleInfo contains information about each bundle we will make – the name of the bundle and the path to the bundle.  The AssetInfo class describes an asset, and has a reference to the bundle it belongs to as well as a path to the actual asset.

Now that we have defined our data, we can make XML entries to represent the Asset Bundle we want to build, called MainContent, and the assets that we want to put in it.  Here’s what the XML looks like (StreamingAssets/XML/AssetData.xml):

We define the bundle at the top then define each asset.  Each AssetInfo entry references the MainContent AssetBundleInfo entry.  For simplicity, we will only define one bundle, but you could easily define as many as you want.

The data we have defined in our XML database will serve two purposes.  It will inform the editor-side build process which bundles to build with which assets, and at runtime will act as a manifest for the bundles we will load and the assets in each bundle.  Let’s look at the build process first.

ResourceManager

Both the build process API and the runtime API for asset loading are defined in a class called ResourceManager in Common/Script/ResourceManager/ResourceManager.cs.  This script has a series of static methods used to build the bundles in the editor, and sits on the GameController object when the game runs.

The ResourceManager.cs script is a bit long, so I won’t put the entire listing here (though I encourage you to look over it).  We will look at the important parts.  First, let’s see how the bundles are built:

In order to build an Asset Bundle, we need two arrays: one array with the objects which will be included in the bundle, and one array with the names of the objects (in our system, the objects’ paths are their names).  This is easily done with our XML database.  The BuildAssetBundles() static method simply loops through each AssetInfo in the database and stores of the asset and the asset name into lists.  Each list is kept in a dictionary keyed to the appropriate AssetBundleInfo ID.  We then loop through each AssetBundleInfo in the database, fetch the list of objects and the list of names for that info’s ID, and build the bundle.

Next, let’s look at how assets are loaded at runtime.  When the game initializes, we call the LoadBundlesAsync() method, and pass in a delegate to be called when loading is finished (this happens in GameController.cs in our example).  This method starts a coroutine called LoadBundlesCo() which looks like this:

This coroutine loops through all of the AssetBundleInfos in the database and loads each one using the WWW class.  Once loaded, the resulting Asset Bundle is cached in the loadedBundles dictionary, keyed to the AssetBundleInfo’s database ID.  When all bundles are finished loading, the onComplete delegate is called.

Now that each Asset Bundle is downloaded and cached, we can load assets from them.  To do this, we call LoadAssetAsync<T>() and pass in an AssetInfo and a callback delegate.  This function kicks of the LoadAssetAsyncCo coroutine, which looks like this:

The AssetInfo has a reference to the AssetBundleInfo which it belongs to.  We use the database ID of the AssetBundleInfo to get the actual Asset Bundle from the loadedBundles dictionary, then load the asset.  Once loaded, we call the provided delegate, passing in the asset which is casted appropriately.

The last bit of code we will look at in the ResourceManager class is the ValidateAssetBundleAssets() method.  One of the benefits of keeping data in a database is that it is more easily validated at design time.  Here is what the business logic of ResourceManager’s validation looks like:

This code simply loops through all AssetInfos in the database and tries to load each one.  If it is able to load the asset, we know the asset’s path as defined in the XML is correct.  If it can’t be loaded, we throw a warning.  This is a very simple example of validation.  You could get much more sophisticated with this.

There is a lot more code in the ResourceManager class which I will not show in this article.  Mainly, the ability to force the editor to use Asset Bundles or to bypass them and load directly from disk.  This is helpful when iterating quickly in the editor.  Check it out if you are so inclined.

Example Usage

In the two examples in Part 1, assets were loaded via path strings defined in the TileInfo and FeatureInfo classes.  Now that we have defined these assets in XML, we can replace those strings with DatabaseEntryRefs referencing AssetInfos.  Here’s what the new TileInfo and FeatureInfo classes look like:

To load an asset using the ResourceManager, simply pass in the AssetInfo for that asset and a callback to fire when loading is complete.  Here’s what the AsyncMapBuilder looks like now:

It looks much the same as the last example in Part 1, but we have replaced calls to Resources.Load with calls to ResourceManager’s LoadAssetAsync<T>().  We use a lambda function which calls InstantiateMapPrefab() as the complete delegate.

When you run this example, you will see the tiles and features “pop” into place as they are loaded in the background while the game remains interactive.

Conclusion

The examples in Part 1 and Part 2 are a small sample of what is possible with an XML database system and hopefully have showcased the advantages of externalizing your data in this manner.  To reiterate, here are some of the benefits:

  • XML is easy to source control.
  • XML is platform agnostic.  It can be used by a server just as easily as it can be used by Unity.
  • Linq is awesome.  Even if you don’t use a database for game data, you can still use Linq on any C# collection – just include the System.Linq namespace.
  • You get modding for free!
  • Artists and designers can work independently of each other, especially if you’ve set up an Asset Bundle pipeline as illustrated in Part 2.
  • Designers don’t need to use Unity to change any data that lives in XML.

Thanks for reading!  As always, leave comments and I’m quick to respond.  Happy coding!

2 Replies to “XML Database for Unity (Part 2)”

Comments are closed.