Astra Unity 2.6.6
Astra Unity Plugin
|
This guide will help you understand the Astra SDK for Unity, walking you through some of the architecture and a few short examples. This is meant for programmers familiar with Unity to learn how to integrate Astra into their own projects. If you're interested in a custom engine or Unreal support please contact jjanz. er@m aketa fi.c om
For creating your own content please refer to the Creating Content reference.
For assistance in configuration and installation please refer to the Installation Guide.
For an overview of the folder structure see Astra Project Organization.
Note: All C/C++ calls in the Astra Unity SDK are hidden for you so you only need to deal with the high level calls, greatly simplifying what you need to interface with.
The Astra SDK uses a custom job system to manage tasks. Most tasks in Astra will be asynchronous and return an Astra.Job or a derived class from this. Handling tasks asynchronously is done to avoid blocking on the main thread and to create consistency during operations (processing, networking, etc).
Jobs can run on both main thread and side threads and their events can fire on either as well. There are some explicit events such as Astra.Job.OnJobDoneMain which will ensure callbacks bound for event OnJobDone
are run on main thread.
Generally speaking you will receive a job for a task then wait for the result of the job using events or get a coroutine back and yield.
Jobs fire events on either main or side threads depending on their primary mainThread
flag. There are exceptions such as OnJobDoneSide
or OnJobDoneMain
which fire based on their suffix. Job events are fired in order after triggering all events at one stage before moving to the next stage. As such Astra.Job.OnJobDone will fire before Astra.Job.OnJobFinally.
Order | Event | Thread | Description |
---|---|---|---|
1 | Astra.Job.OnJobDoneSide | Side | Fired after a job is completed regardless of success or failure. |
2 | Astra.Job.OnJobDoneMain | Main | Fired after a job is completed regardless of success or failure. |
3 | Astra.Job.OnJobDone | Either | Fired after a job is completed regardless of success or failure. |
4 | Astra.Job.OnJobSuccess | Either | Fired after a job is completed successfully. |
5 | Astra.Job.OnJobFailed | Either | Fired after a job is completed unsuccessfully. |
6 | Astra.Job.OnJobAborted | Either | Fired only if a job is aborted. |
7 | Astra.Job.OnJobFinally | Either | Fired after done, success, and failures are triggered. Useful to ensure OnJobDoneXYZ callbacks are finished before this one. |
8 | Astra.Job.OnJobPostFinally | Either | Used by the job queue. |
Astra.Job.OnJobProgress | Either | Fired intermittently on some jobs when they have progress to report such as long running tasks. |
Here are a few examples of how to handle various returned jobs in Astra from the API
Jobs are generally created, then pushed into the main queue for running at a later time. They can also be run inline by firing the Astra.Job.Run method which will run the job immediately and not push it into the queue. To push a job into the queue, which is the recommended way of running a job, simply use Astra.MainQueue.Push.
You can create your own jobs quite simply.
Jobs can depend on the successful completion of other jobs, this is controlled through Astra.Job.DependsOn. In addition there are other helper methods such as Astra.Job.CompleteAfter which will trigger the caller job to auto complete when the callee job is completed. For additional information about the Job system in general check out the Astra.Job class.
An asset is a low level building block that has a public GUID
associated with, which we use to build up things that our Avatars can consume. Assets can be things such as a Mesh
, Texture
, Rig
, Eye
, Mouth
, Decal
, Morphable
, or any other basic building block. Assets can also depend on other assets. We use this as a way to build up complex structures from simple pieces. An example of this is our Schematic
which in turn can load or depend on things such as Material
, Texture
, Mesh
, or other simple structures. A Schematic
in turn is used and extended by high level concepts such as Cloth
, Hair
, Figure
, and others.
Every main asset has a GUID and most sub type also have a GUIDs. Assets, internally, are referenced via GUIDs and you should refer to them via GUIDs as well and not by names.
Assets are almost always loaded onto a new or existing Astra.Asset.AvatarItem.
There are several atomic elements such as textures and meshes along with more complicated elements such as avatars in Astra. Below is a table that shows a loose relationship between Astra's systems and Unity's.
Element | Unity Primitive | Astra Primitive | Astra Component |
---|---|---|---|
Texture | Texture2D | PrimitiveTexture | |
Mesh | Mesh | PrimitiveMesh | Astra.Asset.MeshItem |
Material | Material | PrimitiveMaterial | Astra.Asset.MaterialItem |
Rig | PrimitiveRig | Astra.Asset.RigItem | |
Avatar | PrimitiveAvatar | Astra.Asset.AvatarItem | |
Cloth | SkinnedMeshRenderer | PrimitiveSchematic | Astra.Asset.ClothItem |
Hair | SkinnedMeshRenderer | PrimitiveSchematic | Astra.Asset.HairItem |
Part | SkinnedMeshRenderer | PrimitiveSchematic | Astra.Asset.PartItem |
Figure | SkinnedMeshRenderer | PrimitiveSchematic | Astra.Asset.FigureItem |
Decal | PrimitiveDecal | Astra.Asset.DecalItem | |
Material Variation | Material | PrimitiveMaterialVariation | Astra.Asset.MaterialVariationItem |
Collections | PrimitiveCollection | Astra.Asset.CollectionItem |
As seen from the above list Astra exposes most components with the suffix of Item
when these are attached to GameObjects. Here is a list of components you will see when using Astra along with a brief description of their purpose.
Item | Description |
---|---|
Astra.Asset.AvatarItem | The container object that houses the recipe for the character and it's main driver of methods |
Astra.Asset.BoneItem | Helper component that aids in mapping our bones to the RigItem |
Astra.Asset.ClothItem | Clothing objects typically attached to an avatar |
Astra.Asset.CollectionItem | Groups other assets like cloth, hair, decals into a suite loadable all at once as a Collection |
Astra.Asset.DecalItem | Modifiers to the mesh (note: can be attached to non figure items) |
Astra.Asset.EyeItem | Used specifically for the container of the eyes |
Astra.Asset.FigureItem | The overall body of the character |
Astra.Asset.HairItem | Hair objects typically attached to an avatar |
Astra.Asset.MaterialItem | Provides additional meta information about materials and links to a Unity Material as well |
Astra.Asset.MorphableItem | Modifies the mesh and or bones of the avatar |
Astra.Asset.PartItem | Generic type that can contain other generic types (this is used commonly to deal with the mouth) |
Astra.Asset.PoseItem | Used to set the character in specific fixed poses, all base figures will have at least one of these as a T-Pose if using animations |
Astra.Asset.RigItem | The skeletal system of the figure and is used for other sub assets such as clothing |
Finding assets is done through the Astra.Catalog. The first use of the Catalog will trigger a Astra.Catalog.Refresh call which sends a request to the server and determines which assets are available for the current application configuration being used. If you add new assets on the admin portal and the application needs to see those assets without restarting you should trigger Astra.Catalog.Refresh periodically (such as in a long running application or game).
Assets will appear in the catalog once they are bound to a application via the Asset Category
relationship. Looking at the sample whitelabel application is a good way to determine how this process works.
To search the catalog you can use the various SearchXYZ method calls from Astra.Catalog such as Astra.Catalog.SearchForAvatars or Astra.Catalog.SearchForClothing.
Results of these calls will yield a job that upon calling GetResult will typically be a list of Astra.CatalogEntry records.
If you already know the asset GUID or once you have found the asset in question you can load it using the Astra.Loader class.
Typically you will filter assets so only compatible assets and potentially specific types of assets (such as shirts but not pants) are returned from the query.
Here is a list of how assets types are filtered for compatibility matching
Type | Mesh | Rig | Notes |
---|---|---|---|
Cloth | X | ||
Hair | X | ||
Decal | X | ||
Part | X | ||
Pose | X | ||
Material Variation | X | ||
Morphables | X | X | MeshTargets use mesh guids, RigTargets use rig guids |
Collection | X | X | Typically collections are filtered by rigs though |
When filtering with a Rig this will come from the Astra.Asset.RigItem.AstraGuid, you can locate this for a loaded avatar via:
In most cases when using a mesh filter, like in decals, this will be the Astra.Asset.MeshItem.AstraGuid of the body mesh.
Thankfully the Astra.Catalog takes care of this for you if you pass in your Astra.Asset.AvatarItem into the various search calls. You are encouraged to use these overloaded calls over freeform as it simplifies filtering for you.
To filter by regions such as finding clothing that fits on the torso you will need to specify a list of strings containing these regions as tags.
The following regions are used when creating attachable assets to avatars for cloth and hair.
In addition when creating your own content you can provide custom tags as well and these can be used to filter the same as regions.
Tags are paired using AND boolean logic, so if you provide a list of three tags the content that is in the result will match ALL three tags.
In most cases assets will be loaded onto an existing avatar. This lets you dress and modify an avatar for your users, or let your users modify them directly as well. You can create a new GameObject and attach the Astra.Asset.AvatarItem component if you wish to make one from scratch. You will then need to load an Astra.Asset.FigureItem to get a basic nude or underwear wearing avatar.
When loading assets onto an avatar most assets, such as clothing, will automatically handle removing assets that are in conflict (such as replacing a pair of pants with a pair of shorts). This greatly simplifies needing to know when and how to remove or hide assets when adding new ones.
Assets are loaded using the Astra.Loader class with various LoadXYZ methods such as Astra.Loader.LoadDecal.
Most load calls accept a parameter that wraps all options for loading, of which you will typically embed an avatar item.
Avatars are the primary building block component that house other sub types such as figures, clothing, hair, decals, morphables, and others. In the SDK these are represented in your scene as Astra.Asset.AvatarItem. When avatars are saved they create a recipe that tells you how to load a complete representation of them at a later point. This allows you to create and save avatars and load them elsewhere (on different devices, different accounts, different applications, etc) and recreate the same avatar.
There are both App Avatars
and User Avatars
. Where App Avatars
are controlled by you, are read-only, and useful for "default characters". User Avatars
are what your customers will create and use to modify their own avatars.
Below is an example of an avatar using our Arcade
figure with Arcade
clothing:
After loading you will have an avatar that may look something like this:
Astra.Asset.AvatarItem is the main component for avatars; there are many methods inside for handling actions like syncing, serialization, etc. Below is a list of some common ones you might need while developing your game or application.
One of the biggest features of the Astra system is dynamically, at runtime, support for updating the mecanim system. The easiest way to use this is to trip the Astra.Asset.AnimationManager.GenerateHumanAvatar from the AvatarItem you want to use. This takes care of creating a new Unity Avatar
representation along with updating proportions and muscle restrictions.
This is automatically done for you if using morphables which move bones, you do not need to do this yourself.
You can find an example at Animating an Avatar. Or in your project folder at Assets/Astra/Examples/Animation/AnimatingAnAvatar.cs
.
You can save an avatar through the Unity Editor via clicking on an AvatarItem
component and clicking on the Save Avatar
button. This will in turn call the following code:
Figures are the basic body of an avatar, they can be loaded standalone via FigureLoader.Load
or via the main avatar loading. They typically contain at least a Astra.Asset.FigureItem, one or more Astra.Asset.MeshItem, Astra.Asset.EyeItem, and Astra.Asset.RigItem. The Astra.Asset.FigureItem is a component that is responsible for the overall skin of the figure and ultimately is a primary pivot area that is frequently queried by the developer and Astra.Asset.AvatarItem as a way to get access to the body mesh or the rig for further calculations.
Figures contain all the essential parts of the nude character (bones, eyes, mouth, skin) and as such are also visual skin of the figure (through various materials and mesh items, eyes, and mouth), if you are swapping a character from say an android skin to a human skin you would do so via the FigureItem.
Unity C# example
If you want to change out the existing skin of an avatar with a new FigureItem you can do so via
Unity C# example
Rigs are a crucial component of any avatar represented in Unity as Astra.Asset.RigItem. They are essentially the hierarchy of the character's bone system. In addition to providing the bone data for an avatar they also handle much of the Morphable
syncing requirements such as updating animation retargeting data, bind pose manipulation, and donating their bone data to child nodes such as Hair
or Cloth
items.
Typically rigs are not handled directly by the developer and instead the Figure
will handle loading instead.
NOTE: when loading assets for an avatar, only one rig is used even though dependent assets such as Cloth will have their own rig.
An example of the Arcade
Rig:
Unity C# example
If you are doing advanced manipulation of your avatar you may find it helpful to trigger the Astra.Asset.RigItem.Sync method which will take care of moving the bones to their desired targets, adjusting the mesh to fit through bind pose manipulation, and regenerating a new Unity Avatar definition (used for animations). Typically this is done for you automatically and you won't need to worry about it.
Cloth is how you dress your avatar and is represented in Unity as Astra.Asset.ClothItem. The system for attachment to an avatar typically takes care of automatic swapping (optional). These are handled by the Layering and Eviction
system along with which regions the Cloth exists in.
An example of Cloth:
Cloth will include a copy of the rig so as to be spawnable outside of an avatar and to link the weights between the desired avatar (if any) and the cloth itself.
Cloth physics is not yet implemented but is a planned feature for 2019.
Decals, Astra.Asset.DecalItem, allow easy layering of textures on top of the normal mesh. There are two types of decals, static and dynamic, and they can have their compatibilities marked by an optional Mesh GUID.
Decals support preserving order layers and strength of application. They can blend Albedo, Metal, and/or Normal maps.
Static decals are UV based textures that are marked specifically for specific meshes in preconfigured locations. These are typically used for applications as fixed makeup, scars, blemishes, age, wear, dirt or other such features.
An avatar without (left) and with (right) a static decal:
Eyes, Astra.Asset.EyeItem, are simple elements that create containers for loading the eyes in the Avatar. Eyes can be either simple pre-designed fixed eyes or contain procedural elements for controlling the pupil, iris, or other elements. Currently eye colors and sizes are modifiable and supported across engines.
Eyes can be modeled as simple orbs, half orbs, or anatomically shaped. They can either use basic shaders, such as on mobile targets, or advanced shaders with parallax effects or photorealistic rendering such as in the Unreal engine.
Example of the default eyes for Arcade:
Eyes are not typically loaded directly by the developer and instead come pre-packaged with a figure, but if you wanted to swap one out you can, such as the below example:
Unity C# example
The EyeItem
itself has helpers to expose the left and right eye via: EyeItem.LeftMesh
and EyeItem.RightMesh
which point to their respective MeshItem
elements. It's also important to note that the EyeItem
component itself is the parent component that references both sub-elements for the eyes which are simpler MeshItem
elements.
Hair, Astra.Asset.HairItem, is used for adding hair to an avatar's body. It can be used for both hair on the top of the head and for facial hair. It is important to note that when processing hair assets you mark them as Hair
instead of Cloth
so that they can take advantage of upcoming features for hair.
An example of a Hair item compatible with the Arcade figure:
Currently work is underway for hair based physics both soft body and bone based and will be available sometime in 2019.
Unity C# example
Meshes in Astra contain all the geometry and data necessary to recreate a skinned or unskinned mesh in Unity, Unreal, or any other game engine. They include the vertices, normals, tangents, triangles, uvs, skin weights, and any other supporting data set you would expect a mesh to have. They do not include the Astra.Asset.RigItem however as these are simpler lower level building blocks. Meshes also contain information about their submeshes and how to map between them.
Meshes are rarely loaded by themselves outside of development purposes, nevertheless you still can manually load them.
Unity C# example
The Astra.Asset.MeshItem component in Unity exposes a few useful helper parameters and methods that you will have access to.
Poses, Astra.Asset.PoseItem, are used to force a rig into a specific orientation. Poses can set positions and rotations but not scale. They also have the following types:
PoseUnknown
- An error statePoseT
- A rigid T-Pose used specifically for processing, rebinding, and retargeting of animationsPoseRelaxed
- Typically this is the rest pose or post rotation posePoseFreeform
- A generic pose type for custom orientationsPoseUnreal
- A slightly relaxed A-Pose used by the UE4 mannequinYou may find the Astra.Asset.PoseItem.HoldPose and Astra.Asset.PoseItem.ReleasePose useful.
An example of an avatar with no specified pose (left) and the posed character (right) in a T-Pose.