What are the UX Tools?

UX Tools for Unreal Engine is a UE game plugin with code, blueprints and example assets created to help you add in features commonly needed when you're developing UX for Mixed Reality applications.

NOTE: All OpenXR devices should be supported, although only HoloLens 2 and Windows Mixed Reality VR devices have been tested.

Getting started with UX Tools

If you're new to MRTK or Mixed Reality development in Unreal, we recommend starting at the beginning of our Unreal development journey in the Microsoft Docs. The Unreal development journey is specifically tailored to walk new developers through the installation, core concepts, and usage of the UX Tools.

CAUTION: The Unreal development journey currently uses UX Tools 0.12.x and Unreal 4.27.0+.

If you're already familiar with Mixed Reality development in Unreal, you can find the installation instructions for the UX Tools plugin here.

Documentation

Release Notes	Welcome to UX Tools	Feature Guides	API Reference

Required software

Windows SDK 18362+	Unreal 5.0.0+	Visual Studio 2019
To build apps with MRTK-Unreal and UX Tools, you need the Windows 10 May 2019 Update SDK	The Unreal Engine provides support for building Mixed Reality projects in Windows 10	Visual Studio is used for code editing

UX building blocks

Input Simulation	Hand Interaction	Button
In-editor simulation of articulated hands and headset pose	Support for far and near hand interactions with Unreal's primitives	A 3d button specifically designed for hand interactions
Slider	Bounds Control	Manipulators
A 3d slider specifically designed for hand interactions	A component that allows the user to transform an actor via affordances	A component that allows an actor to be picked up by a user and then moved, rotated or scaled
Hand Menu	Near Menu	UMG support
Hand-attached UI great for frequently used functions	Floating menu UI for near interactions	A component that allows hand-driven interactions with Unreal's UMG widgets
Tap to Place	Follow Behavior	Surface Magnetism
A component that can be used to easily place primitive components against surfaces	A component used to keep objects "following" the user by applying a set of constraints on the component's owner	A component that allows an actor to stick to a surface
UI Element	Tooltips	Touchable Volume
A component that allows an actor's visibility to be controlled by its parent actor, allowing for easy construction of hierarchical UIs	Annotation UI with a flexible anchor/pivot system, which can be used for labeling objects	A component that allows the user to interact with an object by touching it

Tools

Utilities	Graphics
Plugin containing a handful of tools that augment the Unreal Engine editor	Breakdown of shaders, materials, and graphics techniques used to render UX components

Example maps

If you want to explore the different UXT features you can try out the UX Tools Game project. You can download the pre-built UX Tools Game application and deploy it to a HoloLens device right away. Alternatively, you can build the latest version of UX Tools Game from source.

It is also possible to try out the UX Tools examples inside your own project. Please note that the examples should be only used as a reference. Your final project should not have a dependency on the UX Tools Examples plugin.

Packaged UX Tools Game example application

We also provide the UX Tools Game pre-packaged for HoloLens 2, so you can try out the main UX Tools features directly on device easily. To use it:

1. Obtain the packaged game from the latest release page (i.e. UXTGame-HoloLens.<version>.zip) and unzip it to a local directory.
2. Install it in the device via the Device Portal.

Building the latest UX Tools Game example application

For that you should:

1. Clone this repository.
2. Checkout the latest 'public/version.x' branch.
   • Bear in mind that this branch is alive. It's not a release, and will be updated regularly with potentially breaking changes. There will be a release tag (e.g. release/0.11.0) marked as such in GitHub.

You can now open the UX Tools Game (/UXToolsGame/UXToolsGame.uproject) and explore individual example maps or open the Loader level to access some of the examples from a centralized hub.

Importing examples to your project

You can also copy the UX Tools Examples to a new project. This is a convenient way to use the provided sample scenes as a reference when building your own Mixed Reality experiences. You are free to reuse the sample UI elements but please ensure that your final project does not declare a dependency on the UX Tools Examples plugin.

There are two ways in which you can add the example scenes to your project:

• You can obtain a pre-built version of the UX Tools Examples plugin from the latest GitHub release. You can download UXToolsExamples.<version>.zip from release attachments and extract its contents into the Plugins folder inside your Unreal Engine project.
• Alternatively, you can clone this repository and copy UXToolsGame/Plugins/UXToolsExamples to the Plugins subdirectory in your own project.

If you want to use content from the examples plugin as part of your project, make sure to copy that content to your project's Content folder. Newer versions of the examples plugin may change or remove existing content without notice.

Please note that you also need to add the UX Tools plugin to your project. Always use a UX Tools Plugin version that matches the version of the UX Tools Examples. Example content is only guaranteed to work with a single version of the UX Tools Plugin as releases can introduce breaking changes.

Sample apps made with UX Tools

This simple Chess App is the final result of following the getting started tutorial located on Microsoft's Mixed Reality docs. If you're new to Mixed Reality development and UX Tools, get started here: Build a chess app with MRTK UX Tools	Kippy's Escape is an open-source HoloLens 2 sample app demonstrating how to use UX Tools components to create a fun and interactive Mixed Reality experience. Read the story behind the app: The Making of Kippy's Escape

Videos

A step-by-step tutorial on how to set up a dev environment for HoloLens 2 and a simple level from scratch	Walk through how to get started developing for HoloLens 2 using Unreal Engine	Webinar showing how to build a product configurator using UX Tools

Feedback and contributions

Contributions and feedback in the shape of bug reports, suggestions and feature requests are always welcome! However, we are not in a position to accept external contributions through pull requests at this time.