Looking for Interactive 1 docs? Find them here.

Introduction

Mixer Interactive enables viewers, also known as participants, to directly control the environment in and around streamer’s broadcasts by interacting with user interface controls. When a broadcast has interactivity enabled, controls appear beneath the video on the viewer’s screen. These controls dynamically change according to live events and update in response to different situations that occur within the broadcast.

Developers and producers can create interactive experiences which can run as a part of a game, entirely as a standalone application, or as a tool. These experiences are then used by broadcasters to make their broadcasts interactive.

When a viewer interacts with the controls, their input is sent directly to the experience, allowing the developers to see exactly who is interacting and what they are doing. This level of engagement allows for the creation of truly unique and interactive experiences that let viewers and broadcasters experience Mixer broadcasts on a whole new level.

Overview

These four major components make up an interactive experience:

  1. Game Client
  2. Interactive Service
  3. Interactive Project
  4. Participants
Diagram showing the structure of an interactive project

Diagram showing the structure of an interactive project

The Game Client

A Game Client is software code which processes interactive events. It is written by developers who want to create an interactive experience. Game Clients connect to the Interactive Service and listen to events and updates sent from the service to take action within their environment, thus affecting the broadcast.

A Game Client can be created for a variety of environments, such as:

  • Within a game's current codebase
  • In a third party mod of a game
  • As a standalone application
  • As a game's server-side service

The Interactive Service

The Interactive Service is a service operated by Mixer. A Game Client connects to it to create an interactive session. Once a session is established, the service acts as a mediator for the Interactive session. The service manages data flow and state within the session by processing and distributing data sent to and from both the Game Client and Participants. The Game Client sends messages to the service when it needs to update the session state or to interact with the session.

An Interactive Project

An Interactive Project stores settings and metadata about an interactive experience. They are created by developers through the Developer Lab. Projects are owned by a single Mixer user but can be shared with other users.

Within the Studio you can:

  • Edit your project's name and description
  • Create and store controls and scenes
  • Control who can access your project
  • Publish your project for everyone on Mixer to see and use

When a Game Client connects to the Interactive Service, it provides your project's ID to the service. The service reads this ID and sets up an interactive session with the saved settings and controls created in the Studio.

Participants

Participants are viewers of a live stream. They are Mixer users watching a broadcast in a broadcaster's channel. When they join the channel, they are connected to the interactive session. They are provided with the controls within the Project, that they can use to affect the broadcast. A Game Client can react to events and interactive input to change the controls that are displayed to participants.

Interactive Experience Structure

An Interactive experience contains hierarchical structure of various elements.

These elements are:

  • Scenes
  • Controls
  • Groups
  • Participants
Diagram showing the Relationship between elements in the interactive hierarchy.

Diagram showing the relationship between elements in the interactive hierarchy.

Scenes

A scene is a named collection of controls. Within a scene, controls are arranged on a grid. The grid's layout determines how the controls are displayed to participants.

The Game Client can add or remove controls from a scene. It controls which scenes (and controls) are shown to which participants throughout the session. Scenes are used to group controls together in a coherent fashion that is meaningful to the experience.

For example, in an adventure game, you might have a "Battle" scene displayed when a broadcaster's game character is in battle; a "Field" scene displayed when the character is walking around in the game world.

All Interactive Projects have a default scene. Without any intervention from the Game Client, all participants and groups are shown the default scene.

The Game Client can change the scene that a participant sees by updating the group they belong to.

Groups

Individual participants can be segmented into groups. Participants within a group all see the same Scene and contribute input to the controls that are part of that scene. Game Clients can create and update groups any time, including changing the scene that the group is set to see. A participant can only be a member of one group at any time.

Groups can be used to create team-based experiences where groups compete to achieve a goal within the experience. Using an adventure game again as an example, you could create an "Allies" group and an "Adversaries" group.

The "Allies" group is provided with controls that gives them the ability to heal the character that the broadcaster is controlling or grant buffs to increase their chance of winning. Alternatively, the "Adversaries" group could spawn traps or monsters to try to get in the way of winning - using their controls.

Participants always start out in the default group but Game Clients can re-assign participants into any group.

Controls

A control is an interactive element in a user interface within a scene. Participants can interact with the control using touch, keyboard, mouse, or controller. There are currently two types of controls - Buttons and Joysticks. Additional control types may be added in the future.

Buttons

A button is a rectangular, interactive control within a scene. Game Clients receive events when participants interact with a button.

  • mousedown event is sent when a button is pressed by a participant
  • mouseup event is sent when a button is released by a participant
Sample button control which costs the participant one spark if they click on it.

Sample button control which costs the participant one spark if they click on it.

Developers can use buttons to enable participants to vote, cause in-game actions to happen, or control in-game entities.

Buttons are customizable. These button properties can be edited from both the Interactive Studio and a Game Client:

  1. Text displayed on the button
  2. Spark cost (For more info about sparks, see What are Sparks?)
  3. Width of the progress bar, which is displayed at the bottom of a button
  4. Disabled state - Buttons which are disabled cannot be interacted with
  5. Cooldown duration - Prevents interaction until it expires
What are Sparks?

Sparks are Mixer's virtual currency. Participants earn them while watching or participating in broadcasts everywhere on Mixer.

Spark Transactions
Diagram of a transaction's life cycle

Diagram of a spark transaction's life cycle, showing its transition between states.

When a button with a spark cost is pressed, it creates a transaction. To deduct sparks from a participant, a Game Client must capture the transaction. If a transaction remains uncaptured for 5 minutes, it automatically expires, and the cost associated with the transaction is not deducted from the participant's spark balance.

This mechanism allows the Game Client to decide whether sparks should be deducted from a participant. This feature is a great way to ensure that input from a participant has been converted into the expected action associated with the button press before deducting sparks from their balance.

Note that deducted sparks are not transferred to the broadcaster.

Joysticks

Joysticks are circular controls positioned within a scene that participants can click and drag. Moving a joystick sends an input event down to the Game Client with the coordinates of the joystick relative to its center. Joystick coordinates range between -1 and 1.

A joystick display to a participant.

An idle joystick displayed to a participant. Its coordinates are 0, 0.

An illustration of the coordinate system for joysticks.

An illustration of the coordinate system for joysticks. The top left is -1, -1 and the bottom right is 1, 1.

Getting Started

Interested in making an Interactive Project? Let's get started!

Prerequisites

Before you begin, we recommend that you have:

  • A Mixer user account
  • Some knowledge of programming, unless you're using an existing Game Client or Interactive Application
  • An awesome idea

Choose a SDK

If you're making a Game Client from scratch, you'll need to use an Interactive SDK.

Environment Repository / Download Documentation Description
Typescript, Node.js & Browsers GitHub Reference Docs Great for creating Interactive tools and utilities. Suitable for beginners.
C++ GitHub Reference Docs Add Mixer Interactive features directly into a C++ Game.
Unity Asset Store Reference Docs
Getting Started
Add Mixer Interactive features into a game created using Unity and also good for quick prototyping.
Unreal Engine 4 GitHub Getting Started Add Mixer Interactive features into a game created using Unreal Engine 4.
Java GitHub Reference Docs Add Mixer interactive to your Java project.

Download the appropriate SDK that your project needs from the repository and create a new project in your favorite IDE.

Create a new Interactive Project

Interactive projects are created and configured in the Developer Lab.

Go to the Developer Lab and click on the plus button to create a new project.

Screenshot showing the location of the create new project button

Screenshot shows the location of the Create new project button.

You'll then be prompted to enter a name for your project. Enter a suitable name for your project.

After creating the project, you can still modify your project using the editor.

Configure, build, and publish the project

The project editor contains tabs where each tab is a step in the creation of your project. Use the project editor to configure, build, and publish your project.

studio editor tabs

The editor consists of these four tabs:

  • Info - Edit the project name and metadata
  • Build - Configure Scenes and Controls
  • Code - Hook your Game Client up to the Project and start coding
  • Publish - Submit your project for review by the Mixer team.

Info

The info step is your opportunity to describe your project and provide potential users with all the information they need to get up and running.

It is important to describe what your project does and how to install it in a clear and concise way so that users understand what you have developed and how to use it. Be sure to include some information on how users can contact you if they need help or want to report an issue.

During the pre-publish review process, the Mixer team examines your project more closely. We use this information to gain an understanding of what your project does, so be as thorough as possible.

Build

The build step is where you design your scenes and controls for your project. The interface is divided up into three sections:

  • Scenes
  • Controls
  • Grid

Scenes section

This is where you'll manage the scenes for your project. Scenes can be created, renamed, or deleted here. You can also select a specific scene that you would like to manage controls for.

studio scenes

Controls section

This is where you'll manage the controls in your project. You can create, rename, change the type (button or joystick), adjust spark cost, or delete controls here.

studio controls

Grid section

The grid is how you specify the layout of your controls for the scene. There are three different layouts for which you can set the look of the scene: small (mobile phones), medium (tablets), and large (desktops). These different layouts are indicated by icons in the upper left area of the grid section.

To add a control to the grid, click and drag the control from the Controls section on to the Grid section. Once the control has been added to the grid you have the ability to resize, reposition, or remove the control.

For more detail on the configuration of the controls on the scene, the View JSON option in the top right of the grid section will display the relevant JSON for the scene.

Once you are finished building your scenes/controls, press the Save button in the top right of the grid section.

studio grid

Code

The code step is to help you complete the code portion of your project. This is usually done after you have set up the scenes and controls. This is also where you can grab the Project Version ID which is displayed in the center of the screen.

You will need to write your own Game Client to connect to the Interactive service. For SDK downloads, documentation, and samples, go to Choose a SDK.

Publish

The final step in the studio is Publish. Publish allows you to share your project with everyone on Mixer.

To help you decide whether your Interactive project is ready for publishing, these are some questions to ask:

  • Is your project something that everyone would use?
  • Is your project ready for every Mixer broadcaster to have access to?
  • Have you filled in the Info tab with detailed installation and setup instructions?

If you’ve answered no to any of the questions above, you should not publish your project yet. If you would like to have a small group of users to be able to access it, consider sharing your project instead.

Understand the Publishing Flow

studio publish process

A Project starts off as a draft. In this state, you are free to edit, test, and share it with other users. Many projects don't leave the draft state. It's perfectly ok not to publish your project.

When you've decided to publish your project, go to the Publish tab in the interactive studio. It will perform some checks on the project to ensure that it meets some basic requirements before allowing you to click on the Submit button.

Once you've submitted your project, the Interactive review team at Mixer is notified, and we will begin the review process. While a project is in review, you won't be able to edit the project. A Mixer representative may also reach out to you with questions.

After a review is conducted, you'll be notified whether your project is accepted for publishing or not. If your project is not accepted for publishing, it goes back to the draft state with feedback from the team, detailing what needs to change so that your project can be successfully published.

Once your project is accepted for publishing, we can start discussing the actual launch date; when you would like to make your project available for all Mixer users in the Interactive Store.

Sharing an Interactive Project

By default, only the owner can use the project in a broadcast. This means that when you create a project, you are the only person who can broadcast the project. However, if you'd like other Mixer broadcasters to stream it, you have several options.

To manage a project's share settings, click the Share icon on the top right of the editor, as shown in the screenshot below.

studio share button

You'll be presented with a dialog with three options. Changing the sharing settings of a project will delete any previous share settings

The first option, "Nobody can play this game until published", is the default. Selecting this option allows only the project owner to use the project in a broadcast.

Share Codes

The second, "Anyone with the versionId and code can play", option generates a share code. This code can be handed out to broadcasters who want to use the project in their broadcasts. These broadcasters, provide it to the Game Client which can then gain access to use the project.

studio share code

Explicit Sharing

The third option, "Only allow specific users to play until published", is explicit sharing. It lets you share the project with named Mixer users. Only users in the list can use the project. To share a project with a specific user, search for their username in the search box and then click "Add".

studio explicit sharing

Protocol Overview

Mixer's Interactive protocol is defined in a separate downloadable document that has precise implementation details. This section provides an introduction to the protocol.

Wire Format

The Interactive Service communicates using a protocol similar to JSON-RPC except that it is bi-directional. Clients and Servers can both call and respond to methods.

The protocol operates over a standard WebSocket connection. Both Participants and Game Clients use the same protocol definition, but different subsets of methods are available to each.

Packets

The protocol contains two types of packets: methods and replies.

Method

A method is a request for a connected entity to perform an operation. Methods are sent by both the client and the server. When a method is received, it is processed and acknowledged by the recipient, who can then reply to the method with a result or an error.

A method can contain parameters which get provided to the recipient.

Methods contain an additional property called discard, which when true, indicates that the recipient can choose not to respond. Methods that can be treated as events have the discard property set to true.

Reply

A reply is sent from a recipient back to the caller informing them about the result after executing the method that was sent. It can contain a result or an error which indicates what went wrong.

For full packet implementation details, please refer to the protocol specification which you can download here.

Compression

By default, packets on the wire are transmitted as plain text, but the Game Client can opt to use GZIP or LZ4 compression. To do this, the Game Client must call a method providing its supported compression formats. The server will then respond with its chosen compression format.

Authentication

A Game Client needs to authenticate as a Mixer user when establishing an interactive session. There are two authentication methods available.

OAuth 2.0

Mixer supports OAuth 2.0 flows which enable you to get a valid OAuth 2.0 Bearer token. Tokens can be passed in the Authorization header when you initiate a connection to the interactive service.

The only required scope for an interactive connection is interactive:robot:self. For more information about Mixer's OAuth, go to OAuth reference page.

XToken

You can provide a Xbox Live XToken in the Authorization header when you initiate a connection to the interactive service. This authentication method is useful for Universal Windows Platform (UWP) applications that are Xbox Live enabled, as well as games on Xbox One.

Where to get help

Have questions? Stuck? We're here to help! We have places you can get help so drop by!

  • Gitter - Chat with our team and other developers.

  • Forums - Post a topic, and we'll get back to you.

  • Community Discord - Hang out with other developers in our community Discord.