Reality Platform SDK - Overview

The PureWeb Reality Platform SDK is the foundational building block for building applications and services that interact with the Reality Platform. The SDK is used for much more than building streaming web clients for Unreal or Unity applications. The SDK is also incorporated into the console & preview client, the streaming agent, and the core platform API.

The Reality platform architecture uses an Agent Based Model for how services within the system interact [1]. All systems built on top of the SDK are capable of participating in a secure peer interaction with other agent based systems built with the SDK. For example, while the process of launching an interactive 3D stream appears to be a client-server interaction, it is actually a peered interaction between a web client (built on top of our SDK), an a streaming agent (@pureweb/platform-streaming-agent - npm) running in the cloud. This architecture makes it easy for multiple web clients to join into a shared collaborative session with a streaming Unreal or Unity application.

Fig 1. Example of an Agent interaction involving two streaming users (client & collaborator), an Unreal or Unity game (proxied by the streaming agent), and a custom IoT data service. All agents are connected and exchange data through the shared and secure agent environment.

Similarly, the streaming agent can also be used in a local desktop development environment. This use case is a key differentiator for PureWeb Reality, as it allows for simultaneous development of a 3D model and the corresponding custom client in a fast, local development loop, while ensuring that when the 3D model is uploaded to the Reality platform, the client/streaming-agent interactions will behave identically, because they’re both built on top of the same SDK.

Finally, this SDK and architecture also unlocks a powerful scenario where developers can create fully custom agents capable of participating in a collaboration with both users, and a streaming 3D model. Such a custom agent could theoretically contribute a wide range of data into the environment for the clients and 3D model to act upon. An example of this might be creating an agent that brokers IoT data into an agent environment to be consumed and acted on by the 3D model (Fig 1).

SDK Functions

The SDK can be broadly divided into 3 categories of APIs:

Stream Launching

Authenticated generation of Launch Requests from the Reality platform. Additionally, we provide event hooks to track the status of Launch Requests as they progress through the system (ex. accepted -> in queue -> ready).

Streaming

Setup and signaling of a WebRTC interactive stream from the platform, configuration of an established stream (ex. audio, video resolution, bandwidth and mouse input configuration).

Agents

The Agent capabilities of the SDK allow for the coordination and sharing of information between agents built on top of the SDK. These APIs can be grouped into 4 sub-categories:

  1. Authorization: Token based access to scoped agent resources. This ensures that agents can only connect to environments to which they have been granted access.
  2. Presence: Subscribing to notifications of agents arriving or departing from a shared environment.
  3. Messaging: Agents can communicate with all or specific agents within the agent environment. Agents receiving messages are able to respond or not in whatever way the agent has been developed, to such messages.
  4. Key / Value data sync: Agents can share data into an agent environment and optionally have it persist beyond any agent's time in that environment. Other agents can subscribe to changes in this shared data, and act on it in whatever way is appropriate based on the agent's encoded behaviors.

React Web Template

In addition to the SDK, PureWeb also publishes a React template (@pureweb/cra-template-pureweb-client - npm). This template serves as a wrapper for the SDK and provides an easy way to bootstrap a new web client in a matter of seconds. The React template provides convenient abstractions for the most frequently used elements of the SDK.

References:

[1] Agent-based model - Wikipedia