A HUD (head-up display) offers a simple way to communicate and interact in a video game. UE4 (Unreal Engine 4) offers a way to create HUDs through their UMG (Unreal Motion Graphics) system. However, when it comes to Virtual Reality (VR), HUDs need to work differently for it to be effective. My goal in this post is to go over how to create a three-dimensional interactive HUD for VR in UE4.

Note: Before starting, I have another post about creating static HUDs for VR in UE4. I recommend you go over the post first if you are new to UE4.

Terminologies

• HUD ↔ User Interface, User Widget
• 3D HUD ↔ WidgetComponent in a Blueprint
• 3D space ↔ World

Note: the left-hand side is more generic, while the right-hand side is UE4

Creating an Interactive HUD

Creating an interactive HUD is similar to creating a static HUD. The difference is that there is some code running behind the scene to make the HUD respond to the user’s inputs.

When creating a HUD for VR, there are two things you should keep in mind. One is the visibility of your HUD in VR. The second is that the HUD needs to be set at a custom size. The height and width values are up to you; just make sure it has good visibility.

The best way I believe that will show how interactive HUD works is an example. For this example, I will be using Blueprints and the VR template that comes with UE4.15 and up.

Interactive HUD Example

The HUD will be a simple advance to next level prompt. There are two buttons, yes and no, and the user can select either button. Selecting the yes button will trigger a function to advance to a new level and the no button, in this case, will do nothing.

Steps

1. Create UMG for prompt HUD

• In your project, add a new blueprint “widget”.
• Set it to a custom size (500 x 400 in this example) of your choosing.
• Fill the entire space with a background image color (black).
• Add a text block with prompt text. Make sure to set it to a large font size because it is hard to read in VR if it is too small. For this example, the font size is 36. In addition, make sure it is a variable that way you can modify it later in code or Blueprints.
• Add two buttons, one for yes and another for no.

Begin level UMG with two buttons.

2. Interactive buttons

• Add interaction to the buttons by selecting each button individually and enable them to handle the events “OnClicked”, “OnHovered”, and “OnUnhovered”. An easy way to do this is to select the button then on the Details tab click on the green button corresponding to the event you want to handle.

Location of the possible events for a button to handle.

• Add some logic to handle each event for the Yes button. For this example, when the button is clicked, a function is called to advance to the next level. When the user points to the button, the text will turn red. When the user stops pointing at the button, the text will turn back to white.

Yes button logic for events OnClicked, OnHovered, and OnUnhovered

Advance to Next Level function logic

• Add some logic to handle each event for the No button. For this example, when the button is clicked nothing will happen. When the user points to the button, the text will turn red. When the user stops pointing at the button, the text will turn white.

Logic for the No button for events OnClicked, OnHovered, and OnUnhovered

3. Set up Widget Interaction for Controller(s)

• Add ‘Widget Interaction Component’ to the motion controller. Align it to match the orientation of the motion controller and make sure it is attached to the controller that way everything adjusts dynamically. Note that the big red arrow is the direction the pointer of the Widget Interaction Component will face and that it only acts as a mouse hovering over the HUD.

Adding Widget Interaction Component to motion controller

Creating a 3D HUD

A regular HUD does not work for VR. It would even cause rendering issues. For you to use a HUD in VR, you must convert it into a 3D object.

Luckily, UE4 provides an easy way to bring your HUDs into 3D space. You can bring your HUD into 3D space by the following steps:

1. Create an empty Blueprint Actor
2. Add a new component called ‘widget’
3. Select the widget component and in the Details tab navigate to User Interface
4. Set the Widget class to the HUD class that you want to bring into 3D space
5. Set the drawing size to match your HUD’s height and width. The X value corresponds to the width and Y value corresponds to the height.

Adding ‘widget’ component into empty Blueprint Actor

Adding 3D HUD into World

There are two simple ways to add your 3D HUD into the VR world. One is to drag the Blueprint HUD actor into the world in the editor. The second is to spawn the HUD dynamically, maybe from the result of some user input. To spawn an actor dynamically, you can use the “SpawnActor” function available in Blueprint and choose the actor class to spawn; in this case, it will be your 3D HUD.

Tying It All Together

You have the 3D interactive HUD and the widget interaction on your controller(s). This will allow the buttons to respond to hover and unhovered events. What you have left is to send a click to the buttons.

For the purpose of this example, I will have the right trigger button activate a click. In the same class where you added the ‘Widget Interaction Component’ (BP_MotionController in this example) add an event for the right trigger of the motion controllers. Whenever the right trigger is pressed, the widget interaction component will also send a click.

Send a click via widget interaction component when the right trigger of the motion controller is pressed.

I hope you found this post helpful. If you found this post helpful, share it with others so they can benefit too.

Was there anything I left out, that you think I should add? Do you have another way of creating interactive HUDs that you would like to share? Feel free to leave a comment, send me a tweet, or send me an email at steven@brightdevelopers.com.

To stay in touch, you can follow me on twitter.