Step 1: SDK Version Requirement

Ensure you are using SDK version 4.10.2 or higher. You can test this functionality in the latest web template client or in your own custom client by following Steps 2-4 below. Please refer to the instructions here to update your existing web client to the latest SDK.

Step 2: Implement the onClose Function

In your App.tsx file, add an onClose function to handle the session reset. This function closes the current session, resets hooks, and manages the loading state.

onClose={async () => {
  if (launchRequest) {
    await launchRequest.close(platform);
    // Reset the hooks
    resetLaunchRequest();
    resetStreamer();
    // Reset loading state
    setLoading(false);
  }
}}

Pass this onClose function to your streaming view, where it will be invoked when the session needs to end.

Step 3: Add a Close Button to EmbeddedView

In the EmbeddedView component, add a close button to trigger the onClose  function. Place it in the top right corner and hide it conditionally based on session state or device compatibility.

<Button
  onClick={props.onClose}
  style={{ position: 'absolute', top: 10, right: 10 }}
  className={isIPhone || handle.active || props.StreamerStatus !== StreamerStatus.Connected ? 'hidden' : ''}>
  <Icon name="window close outline" />
</Button>

This button will be available only when appropriate, based on "StreamerStatus."

Step 4: Reset to Launch View on Session Completion

In the App.tsx, monitor the stream status, and reset to initial launch view when the session ends StreamerStatus.Completed.

if (streamerStatus === StreamerStatus.Completed) {
  return <LaunchView Launch={launch} />;
}

Important Notes

• Note that launching a new session will not work on the sessions that require fixed environmentId or on collaboration sessions.
• Every time a session is reset the launch request will work on the new unique environmentId.

Many experiences on our platform have use cases where an interactive stream doesn’t start immediately when the user navigates to the page, but instead starts in response to a user interaction. For example, in an architectural configurator a configuration page may load first, but the user can later switch to a pixel streamed view vs 2D images by toggling a control in the web client.

In order to use the pre-caching feature, there are some necessary platform connection activities that can be done in advance of the user launching a stream. By performing these activities when the page loads, you can reduce the perceived time-to-first-frame for your users.

Our SDK now implements an await platform.getLatencyMeasurements(); call. This call can be made immediately after your call to platform.connect();, which can happen any time before the user action triggers a queueLaunchRequest();.

In order to use this feature you will need to use version 4.10.2 or higher of our SDK, which can be downloaded here: npm: @pureweb/platform-sdk.

Black Screen

We have noticed that displaying certain text on the screen can lead to black screen problems. For example, running the stats FPS command to show the FPS or having messages like "Lighting needs to be rebuilt" can cause the stream to display a black screen.

To address the issue with the stats FPS command, we recommend removing the FPS counter. If you need to see your FPS, it can be manually calculated and added to a widget. If the FPS counter is necessary, adding a delay before running the command can help. A 5-second delay has been tested and found effective, whereas shorter delays may not work.

For on-screen error messages like unbuilt lighting notifications, please have the developer resolve these errors to prevent black screen issues.

For more details on the impact of the FPS counter, please refer to the "Jittery Experience" section below.

Jittery Experience

To improve your experience, ensure that nothing is printed on the VR screen. A common cause of jitterness is the default UE FPS display being turned on. Disabling this display can help resolve the issue.

Jagged Edges

If you notice that the edges of meshes appear jagged, especially when objects are far away or viewed at an angle, it might be due to Anti-Aliasing being turned off. To improve the appearance, you can enable Anti-Aliasing by going to Project Settings, Engine -> Rendering -> Default Settings -> Anti-Aliasing Method.

Important:  If you choose the Multisample Anti-Aliasing (MSAA) method, you will also need to enable  Forward Shading. This setting can be found under  Engine -> Rendering -> Forward Renderer -> Forward Shading.

For more information, please refer to the Anti-Aliasing and Upscaling in Unreal Engine section of the Unreal Engine documentation.

Low Resolution Issues

If your models appear to be in low resolution, it might be due to the Default Screen Percentage being set too low. You can find and adjust this setting by going to Project Settings, Engine -> Rendering -> Default Screen Percentage.

We recommend setting the Manual Screen Percentage to 100 and the Screen Percentage Mode for Desktop renderer to Manual. You can increase the Manual Screen Percentage  to values like 110 or 120 for better resolution, but please note that this may impact performance and streaming quality.

Note: Additionally, be mindful that adding too many post-process effects can also negatively affect the visual quality.

FPS Issues

If you are experiencing low FPS, here are some steps you can take to improve performance:

• Reduce Post-Processing Effects: Post-processing effects can be GPU intensive, especially in VR. Removing or minimizing these effects can help improve FPS.
• Switch to Forward Rendering: You can change the rendering system to forward rendering by going to Project Settings, Engine -> Rendering -> Forward Renderer -> Forward Shading.
• Change Lighting Settings: Switching your lighting from from stationary to static can also help enhance performance.

These adjustments can help optimize your experience and improve FPS.

Controller Beam/Lines Not Showing

When setting up VR projects, you may encounter two types of pointer lines that are displayed though the VR controllers:

1. Debugging Lines: These lines show where the hands/controllers are pointing and are typically in red color.
2. Purpose-Built Pointers: These lines are designed for user control, often used for player movement or interactions.

If you see the red bug controller beam lines when launching your VR project and don't want to see them, it is likely because the project is built in debug mode. Rebuilding the project in shipping mode will remove the red bug lines and fix the issue.

If the purpose-built pointer lines are not appearing as they should, but they do show up in the editor, it could be due to improper implementation or missing files in the build process. A good way to confirm this is checking through a local workflow (e.g., Running Steam VR > from Headset, connect via a Steam Link > Starting the model on the connected machine).

Package Optimization

To reduce your package size, a simple method is to only include items necessary for the maps you are using.

You can do this by going into the Project Settings and searching for package. Under the  List of maps to include in a packaged build, select only the maps your model runs.

Model Is Not Launching As A VR Model

If your model is not configured to launch as a VR model, you may notice the following symptoms:

When the model is launched from the headset, the stream goes to the Grid (Steam waiting room) and never gets past this point. Usually you should see the stream at this point within 30 seconds, depending on model load time, if it is launched properly.

Secondly, attempting to launch a model through a local workflow (e.g., Running Steam VR > from Headset, connect via a Steam Link > Starting the model on the connected machine) results in a desktop view instead of a VR stream.

To ensure your model launches correctly in VR, check the following Unreal Editor settings:

• Set Game Mode to VR:

Navigate to Edit > Project Settings > Maps & Modes > Default Game Mode > select VRGameMode

• Add and Enable the OpenXR Plugin:

Add the OpenXR plugin: Edit > Plugins > Search for OpenXR and enable the Plugin. (Restart the Editor if prompted.)

• Set the Game to Start in VR:

Set the game to start in VR: Edit > Project Settings > Search for start in vr and enable the setting.

• Remove On-Screen Console Messages:

Ensure there are no on-screen console messages. Refer to the "Black Screen" section above for guidance.

• Optional: If the model was initially started as a non-VR model, add VR content to the project:

In the Content Browser, go to Add > Add Feature Or Content Pack To The Project > select Virtual Reality > Add to project

Unity Cloud XR Troubleshooting

Jittery Objects

In some models, objects in the scene can be jittery when moving your head. We have found that objects controlled by the physics engine exhibit this behaviour when the physics engine is set to update at a lower rate than the FPS the headset receives.

In Unity the default is 50 FPS. Most sources recommend 90 FPS, but the current limitation in SteamVR is 72 FPS. Our recommendation is to update the physics engine to a range within 72-75 FPS until we can figure out how to increase this limit. (For more information on how the calculation is done click here.)

It is recommended that the physics engine match or exceed the FPS rate that the headset is receiving.

NOTE: Exceeding the rate by a significant amount will typically reduce the overall model performance due to unnecessary calculations, so it is recommended to only match or slightly exceed the FPS if you know the FPS will be variable.

To change this setting, in the Unity Editor, go to: Edit > Project settings > Time > Fixed Timestep

General Cloud XR Troubleshooting

Index

1. HMD connection sometimes doesn't connect to the model/game.
2. Models launched can leave a server in a bad state.

(1) HMD Connection sometimes doesn’t connect to the model/game

In some launches, typically less than 5%, the HMD fails to connect to the model and the client ends up in the Steam Mountain Cabin waiting room, [need screenshot]. Unfortunately the cause of this isn’t known yet. The workaround to this is to relaunch the session.

(2) Models launched can leave a server in a bad state

We came across this with an early version of the ford model from Imagination. When their model was launched, this error steam was thrown: SteamVR failed initialization with error code VRInitError_Init_HmdNotFound: "Hmd Not Found (108)" Please verify SteamVR is properly installed and try again.

Additional CloudXR Troubleshooting Guide and CloudXR Game Package Preparation (FAQ)

1. Do I need to follow the PixelStreaming preparation guides for CloudXR or VR game packages?

No, for CloudXR or VR game setups, the standard PixelStreaming guides for Unity and Unreal (listed here for Unreal and here for Unity) should not be followed. These setups have different requirements, particularly around handling audio and input.

2. Why is audio not working in my Unity VR game with the PureWebStreaming script?

The issue is likely caused by a conflict between the script’s audio listener and the main camera’s audio listener. The PureWebStreaming script from the PureWeb Plugin is designed for PixelStreaming, which takes over the audio stream. To resolve this:

• Solution: Remove the PureWeb Plugin and recompile the VR game. The audio should then work in the VR headset.

3. Can I use a single model for both PixelStreaming and VR?

Yes, it is technically possible to build a model that works for both PixelStreaming and VR, but it is not officially supported. Audio and other elements would require additional configuration, particularly in the plugin and game project, to function properly in both modes.

4. How should I prepare separate game packages for VR?

For VR-specific packages, follow the setup guidelines from the Unity and Unreal documentation focused on VR:

• Unity VR setup guide
• Unreal VR setup documentation

5. Why does PixelStreaming conflict with VR setups?

PixelStreaming takes control over the audio and video streams, which interferes with the functionality needed for VR. In VR, the audio and video streams are processed differently and must be directly handled by the headset. As a result, PixelStreaming should not be included in CloudXR or VR game packages.

Additional Tips

If you are preparing both VR and non-VR versions of your game, we recommend keeping separate builds and project folders in the console to avoid conflicts with audio and other systems.

To achieve this:

1. Add an AudioSource for the Video - Ensure each video has an AudioSource component attached. This component should play the video's audio.
2. Route the AudioSource to the AudioListener - Ensure the AudioSource components are routed through the AudioListener used in the WebRTC setup.

Please see sample code below to help get you started:

using UnityEngine;
using UnityEngine.Video;

public class VideoAudioSetup : MonoBehaviour
{
	public VideoPlayer videoPlayer;
    private AudioSource audioSource;
    
    void Start()
    {
    	audioSource = gameObject.AddComponent<AudioSource>();
        videoPlayer.audioOutputMode = VideoAudioOutputMode.AudioSource;
        videoPlayer.SetTargetAudioSource(0, audioSource);
    }
}

Feb 27, 2024

Instance-On Feature and Launch Time Improvements

In our latest release, we’ve made significant improvements to launch times with our new Instance-On feature. This release introduces two key changes aimed at reducing launch times: Pre-Launch and Reserved Streams.

Pre-Launch

With Pre-Launch, we've moved the bulk of the discovery and connection process to occur during the initial launch, shifted away from when the user clicks the play button. This change aims to have most (if not all) of the connection process done before the user is ready to start the connection. As a result, the first-frame time can potentially be reduced to just a few seconds, depending on the current client workflow and existing model load times.

Reserved Streams

The second feature, Reserved Streams, allows customers to configure a model to automatically start on a server, wait in a ready state, and connect once the client is ready. This eliminates the need to wait for the model to start before the connection can occur. Documentation regarding this feature can be found here

Expected Launch Time Improvements

By utilizing the Pre-Launch feature, launch times will be significantly reduced. When both Pre-Launch and Reserved Streams are used together, we are targeting a typical launch time of about 2 seconds.

Seamless Developer Integration

The secondary goal of this release was to ensure that these changes are as seamless as possible for developers. A new SDK version must be implemented in your client, upgrade notes can be found here. To take advantage of the Pre-Launch feature, simply uncomment the new call Described here.
Model changes are only required if you have content that auto-plays on start. In that case, you will need to set up a pause until an on-client-connect event occurs before that content begins.

Oct 30, 2024

Summary

In this release we’ve significantly upgraded the computing power of all of our AWS-based on-demand servers. Previously we relied on an Nvidia T4 GPU, but as graphical workloads have gotten more demanding, we felt it was time to upgrade the default GPU in our pools, migrating to an A10G GPU. While there are more powerful GPUs available, in our testing and profiling we’ve found the A10G to be an appropriate entry point, which offers a bit of headroom for models that may not fully utilize everything this GPU has to offer.

We’ve done extensive performance profiling on this GPUs, and the results point to significant improvements in frame rates. On average we saw an 80% improvement in streaming framerate compared to a T4 equipped compute instance. However, for the most demanding workloads in our test bench, we saw improvements in excess of 300% in FPS.

In addition to improvements in framerate, these instances also deliver a slightly better time to first frame (i.e. launch time), with the average coming in around 15% faster, than the same models running on a T4 based instance. Additionally, due to shifts in platform load over the last year, we’ve decided to migrate the Mumbai pool to Dubai, as this should provide better connectivity for a greater portion of our user base. This change will have rolled out automatically and transparently, and every project configured to use the PureWeb on-demand provider will be able to take advantage of the change immediately, with no changes needed in your project or model configuration.

Oct 11, 2024

Summary

This week we released a number of small improvements and updates across the entire platform.

Unreal 5.4
While we had unofficial Unreal 5.4 support for users of our dedicated providers for a couple of months now, we now have official 5.4 support for all provider types in PureWeb (dedicated and on-demand).
There are no breaking changes to either the setup workflow or the plugin if you’re using Unreal 5.4. The latest version of the PureWeb plugin built for Unreal 5.4 can be found in our getting started guide.
Keeping with our engine version support matrix, Unreal 5.2 is now deprecated. While 5.2 models may continue to run successfully for some time, it is no longer part of our testing matrix. Additionally, Unreal 5.3 will now only receive platform support, not plugin and SDK support. For more information please see our deprecation policy.

SDK - Precaching Latency Check
When you trigger a launch request, the first thing our SDK does is test the user's network latency to all of our possible server endpoints. This process takes between 0.7 and 2 seconds. We have a number of use cases where an interactive stream doesn’t start immediately when a user navigates to a page, but instead is started in response to some user interaction on that page. An example of this might be a architectural configurator where a user loads a configuration page, but then can toggle to pixel streamed view vs 2D images. In these situations, there are some necessary platform connection activities that can be done in advance of the user launching a stream. By performing these activities when the page loads, you can reduce the perceived time-to-first-frame for your users. To this end, we have now implemented a new await platform.getLatencyMeasurements(); call in our SDK. This call can be made immediately after your call to platform.connect();, which can happen any time before the user action triggers a queueLaunchRequest();. Version 4.10.2 of our SDK was released today with this change, and it can be downloaded on NPM here: npm: @pureweb/platform-sdk .

CoreWeave Nvidia Driver
The Nvidia driver on our CoreWeave On-demand provider has been upgraded to 552.55.

Stability
A number of stability improvements were made this week, particularly with respect to CloudXR streaming processes and model availability on our on-demand CoreWeave providers. If you have any questions on anything above, please reach out on Discord or at support@pureweb.com. And happy streaming!

Oct 1, 2024

Summary

This release update pulls together a number of updates across various parts of PureWeb Reality which have been updated in the last couple of weeks.

New CloudXR Client
We have released an updated set of CloudXR clients, which can be downloaded here.  This package includes version 7.4.2 of the PureWeb Platform CloudXR client, and version 2.1.9 of the PureWeb Android Streamer Client.  For the time being, we are continuing to focus on the 2-client approach for CloudXR, as we’ve found that this provides the most sustainable path for development, while maintaining the best possible streaming performance.  Specific updates to these clients include:

• Added a refresh button so you if you upload new VR models to the PureWeb platform while the VR client is open, you don’t have to exit / restart your client to see your newly uploaded model.
• Improved error handling for a variety of workflows, including platform connectivity, configuration json file errors, and token errors.
• Added support for local game development, similar to our local workflow for web streaming projects.  Instructions on how to use the local development workflow for CloudXR, check out this link.

Other improvements

• We  added support to the UAE region of AWS as an option for dedicated providers. To make use of this region, you’ll need to be using version 4.10.0 of the PureWeb platform SDK, or newer.
• A number of UI polish improvements on the newly updated PureWeb Reality console.
• Security improvements around token checking.
• Platform stability improvements around model availability accuracy.

Sept 6, 2024

Summary

Today we have released an update to the dependencies in our AWS based on-demand environment. Specifically:

• The Nvidia driver has been updated to 552.55
• Updated the MS VC redistributable to the latest version

Regarding the Nvidia driver, we want to share some insights into how we choose what version to upgrade to. Having over a decade of working with cloud GPUs, we’ve learned that the newest Nvidia driver is rarely the best for our platform / use case. Frequently new driver versions introduce compatibility issues with either the underlying infrastructure, or specific visualization libraries like video encoders or rendering libraries (ex. DLSS). In many cases we’ve tested new driver versions that have seriously degraded rendering performance on on either our on-demand or dedicated system or both. We put a lot of attention and testing into jumping driver versions to ensure that the version we pick is universally compatible across all our provider types, and doesn’t negatively impact any of the workloads that run on our platform.

For reference, our AWS dedicated servers are also using v552.55 of the Nvidia driver. Our CoreWeave based infrastructure will be updated in the coming weeks.

If you have any questions about this update, please drop us a line in discord, or reach out to support@pureweb.com.

Happy Streaming!

Sept 5, 2024

Summary

Today we’re releasing some quality of life and stability improvements in the our SDK artifacts (platform-sdk, admin-sdk, and streaming agent).

• In the admin SDK package (npm: @pureweb/platform-admin-sdk), the authenticate() method, now includes an optional Boolean parameter to tell the SDK whether or not the access token should be automatically refreshed.  The default value for this parameter is true.  In the same vein, we’ve also added a new method called stopAuthenticateRefresh(). This new method allows you have the client access token automatically refreshed until the time of your choosing.  This will give you finer grained control of managing your access tokens when designing a custom auth solution. More on custom auth can be found here.
• In the platform SDK package (npm: @pureweb/platform-sdk ), we fixed a defect wherein the WebRTC peer connection was not properly closed after verifying client side codec support.

Streamer Reset

Additionally, in the platform SDK we’ve implemented new resetLaunchRequest() and resetStreamer() methods. These will allow you to completely terminate a stream from within your client application, and reset the client application state so you can launch a fresh stream. Previously, you had to refresh your web client to launch a new stream. Now if your workflow requires it, you can terminate and re-launch fresh streams from within the same active page without introducing any memory leaks.

To implement this workflow in your client:

In your App.tsx, retrieve the extra reset arguments from the launch request and streamer hooks:

const [status, launchRequest, queueLaunchRequest, resetLaunchRequest] = useLaunchRequest(
   platform,
   modelDefinition,
   launchRequestOptions
);
const [streamerStatus, emitter, videoStream, audioStream, messageSubject, resetStreamer] = useStreamer(
   platform,
   launchRequest,
   streamerOptions
);

In the launch view, you’ll also want to intercept the streamer status to reset the launch view:

if (streamerStatus === StreamerStatus.Completed) { return ; }

Implement your onClose function:

onClose={async () => { 
    if (launchRequest) { 
        await launchRequest.close(platform);
        // Reset the hooks
        resetLaunchRequest(); 
        resetStreamer(); 
        // Reset loading state
        setLoading(false); 
    } 
}}

Call your new onClose function from wherever you want in your client app.

To see a full implementation of these new APIs in action, we’ve also added a new disconnection button into the client template (npm: @pureweb/cra-template-pureweb-client) where you can see the code for yourself.

Default Stream Time Limit

Finally, we introduced a new default stream time limit into the platform via the Streaming Agent (npm: @pureweb/platform-streaming-agent). This is to help combat the rare instances of sessions that keep running or fail to clean up correctly.

The default time limit is 8 hours. If you’re developing locally using the Streaming Agent, this can be configured either as an argument --maximumRuntime or environment variable PUREWEB_MAXIMUM_RUNTIME. The value is passed in seconds.

If the Streaming Agent terminates due to exceeding the configured maximum runtime, you it will return error code ErrorMaximumRuntimeReached (12).

All of these changes are available in version v4.9.0 of the PureWeb SDK bundle.

Check it out on NPM & happy streaming!

Aug 29, 2024

Summary

Today the PureWeb console is getting a major UI overhaul. Our old console UI served us very well for a few years, but we’ve got some big plans for new console features coming down the pipe, and they need a new home.

Our main goal with this UI overhaul was to more readily surface valuable information that was previously buried in various menus and modal dialog boxes, and in general, reduce the amount of clicking needed to move around between different models, projects, and other settings.

Some of the new quality of life changes that we’ve added include:

• Dark mode! Click on your account icon in the top right corner, and select ‘theme’ to toggle modes.
• The home screen now shows all of your available projects along with how those projects are configured (on-demand or dedicated), how many models they contain, and also whether they are Reality projects (i.e. web streaming) or Extended Reality projects (i.e. VR streaming).
• If you have a large number of projects, the left hand bar now includes a project search function.
• The notification icon in the top right corner is now much more informative, also giving you the ability to filter previously read messages.
• We now have a brand new dedicated model page for each model, where you can see and change model settings, and preview your model.

This is just the first step in a series of exciting console updates you’ll be seeing from us, so stay tuned, and as always, happy streaming!

Aug 26, 2024

Summary

We’re excited to announce that as of today’s release, PureWeb Extended Reality, our new VR streaming solution, is now globally available through our dedicated infrastructure providers on AWS.

Running PureWeb Extended Reality on dedicated infrastructure will give you all the same benefits that you’d see for Pixel Streaming or Render Streaming workloads:

• Pick from many area including United States, Australia, UAE, Brazil, Ireland, Germany, England, Japan, India, and Canada.
• Faster time to first frame
• Fully customizable scaling and scheduling behavior
• Ability to reserve infrastructure capacity

Please reach out to support@pureweb.io or your PureWeb account rep to get started. If you’re already using PureWeb XR, you’ll need to update your client applications, which can be found here: https://developer.pureweb.io/metaquest3-cloudxr-setup/

Other Updates

Since our last update, we also made an improvement in our global routing system to more quickly respond to load increases in certain regions and route around any temporary capacity crunches. This should mean less time queueing for a stream. As is the norm, we also made a number of routine security and availability / stabilization improvements.

Aug 14, 2024

While we have been a bit quiet on the release notes front, rest assured that we’ve been hard at work improving and expanding PureWeb Reality over the last several months. With a major release landing this week, we wanted to highlight these recent changes.

CloudXR Support

First and foremost, we’re excited to announce that PureWeb now supports VR streaming through Nvidia’s CloudXR. Work began on this capability in January, and we have been in closed beta since March, and as of today we’re thrilled to announce that the offering is now generally available. This capability will allow you to run graphically intense VR workloads on the cloud, and interactively stream that content to a lightweight all-in-one headset over the internet.

Here are the quick facts:

• As of today, we support the Meta Quest 2 / 3 & Pro headsets. If you are interested in using a different headset, please reach out, as we intend to expand our headset support over time, and we’d love to hear about your use case.
• VR Streaming is supported for both Unreal and Unity models. The currently supported versions of Unreal and Unity are still valid.
• All workloads currently run on our North American based On-demand providers. We will be adding support for our international Dedicated providers in the next two to three weeks. International support for our On-demand providers will be coming at a later date.
• We have a ready-to-use lightweight preview client that can be used to consume your VR streams. This client is easily configurable to either directly launch a VR stream, or browser and launch multiple different VR streams within a given project on the PureWeb platform. We will be releasing a customizable client template later this year which will allow you to build a completely bespoke client experience in Unity.
• VR Streaming is offered a separate license from our existing web-streaming offering. If you’re an existing customer, please reach out to your account manager to learn more about this offering and get set up with access. If you’re not a customer, and interested in learning more, please get in touch.

To learn more about what to expect when using CloudXR on PureWeb Reality, you can check out our Meta Quest - Getting Started Guide.

Streamer IP Address Retention

When negotiating a PureWeb stream, historically, we retained the IP address of the end user in our internal logs for debugging and troubleshooting purposes. However, in an effort to further improve our privacy stance, and only retain the information that is strictly necessary for the operation of the PureWeb platform, all end user IP address information is now anonymized before being written to our logging aggregator.

If you have any questions about what data we do retain, and how we use it, please see this post.

Unreal Microphone Support

As of v4.7.4 of the PureWeb Platform SDK, it is now possible to send web browser microphone inputs to your Unreal application. For more information on how to configure your web client and Unreal application to capture this audio stream, please check out this dev hub article.

Routine Security Updates

As always, we’ve been working hard to ensure that all of the software we use in the development of our platform is as secure and performant as possible. As such we’ve also used several releases over recent months to roll out major updates to key dependencies across our software stack. These updates should be entirely transparent to users.

Jan 26, 2024

Summary

2FA
Today, we’re improving the way you secure access to your PureWeb account, by providing optional two-factor authentication (2FA) for console users.
When you log into the console for the first time, you’ll be prompted to turn on 2FA for your account. Using 2FA is completely optional, but it is an excellent way to improve the security of your account.
For more information on enabling 2FA, please see this article: https://developer.pureweb.io/enabling-console-mfa/.                                  

If you want to disable 2FA, the instructions can be found in this article: https://developer.pureweb.io/disabling-console-mfa/

CoreWeave Security Improvements
We have also released a collection of changes to further improve data security within our CoreWeave based providers. These changes should be completely transparent to end users.

Jan 18, 2024

Summary

Today we’re excited to release an entirely new class of on-demand infrastructure in the PureWeb platform. Taking advantage of the top tier rendering GPUs available in CoreWeave, we are introducing a new High Performance on-demand provider. These new providers, which are available today in both the Western and Eastern United States, are equipped with Nvidia’s powerful RTX A6000 GPUs, 32 GB of RAM, and 10 vCPUs. Capable of powering your most demanding Unreal and Unity streaming projects.

We ran these new providers through their paces with some GPU melting workloads, and we found that our testbench of Unreal applications saw framerate improvements that ranged from 28% to 67%, and our Unity testbench saw improvements ranging from 35% to 48%, when compared to our existing CoreWeave providers.

We also benchmarked against our Nvidia A10G-based dedicated providers, and found that frame rates of the high-performance CoreWeave providers saw improvements ranging from 20% to 132%, depending on the what was being rendered. Depending on the performance characteristics of your model, it can be possible to run these new on-demand providers in a hybrid configuration with a dedicated provider running A10G powered instances for better global reach.

If you’re interested in trying out these new providers, please reach out to your customer success manager, or email support@pureweb.com.

Other releases

We have also released a patch to our Unity plugin (v.3.0.9) available here: https://pureweb-na.s3.amazonaws.com/nightlies/PixelStreaming/com.pureweb.platform-3.0.9.tgz.

This version fixes an issue where touch events from mobile devices were causing a persistent hover effect on UI elements.

Finally, a note about release notes

Over the last year, we’ve rapidly sped up our release cadence. Part of this was done through breaking work down in to smaller releasable units. Due if this, we would often end up publish release notes two or three times a week. Starting in 2024, we’re going to switch to publishing release notes approximately every two weeks. so we can bundle all these updates together, so you’re not inundated with release messaging.

Previous Release Notes

• Release Notes - 2023
• Release Notes - 2022

The client.json configuration file supports the following settings:

projectId

A projectId must always be specified unless an overrideIP is used.

modelId

If the modelId parameter is included in the configuration file, that specific model will be launched automatically. If omitted, the client will present a list of models in the specified project. The user can select one of these to launch.

endpoint

The endpoint overrides the default https://api.pureweb.io.

nativeClient

If nativeClient is omitted, the default NVIDIA CXR client will stream the game.

overrideIP

If overrideIP is specified, the Unity CloudXR client will directly connect to that IP.

launchURL

If launchURL is specified, it will be used to launch a specific model. The projectId must match the model for which the URL was generated. You can generate a shared URL in the model card in our console.

projectToken

projectToken is used for secure projects. You can generate the token using our CLI or extract it from the shared URL. Note that tokens from shared URLs are valid for 7 days, whereas CLI-generated tokens are valid for 24 hours.

Overview

When streaming from Unreal (or Unity) on our platform detecting the type of device consuming the pixel stream needs to be done on the client side in the web browser.

1. Detect the client device type using JavaScript.
2. Send the device information to Unreal using the data channel.
3. Process the received device information in Unreal and enable or disable the virtual joystick accordingly.
4. Enable native touch events for a smoother experience on mobile devices.

Following these steps will help you enable the on-screen touch controllers for mobile browsers while keeping them disabled for desktop browsers.

Detecting the Client Device

The first step is to determine whether the client device is a mobile device or a desktop. You can achieve this using various methods in JavaScript. For instance, you can follow the approaches outlined in this Medium article. These methods generally involve checking the user agent string of the browser to identify the device type.

function isMobile() {
    return /Mobi|Android/i.test(navigator.userAgent);
}

Sending Device Information to the Unreal Application

Once you have the type of client device, you need to send this information to your Unreal application. This can be done using the data channel. Detailed instructions for this process are available in the PureWeb documentation here.

let deviceType = isMobile() ? 'mobile' : 'desktop';
dataChannel.send(JSON.stringify({ type: 'DeviceType', device: deviceType }));

Handling Device Information in Unreal

On the Unreal side, you need to process the received message and enable or disable the virtual joystick based on device type.

void AYourPlayerController::ProcessWebClientMessage(const FString& Message)
{
    TSharedPtr<FJsonObject> JsonObject;
    TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(Message);

    if (FJsonSerializer::Deserialize(Reader, JsonObject))
    {
        FString MessageType = JsonObject->GetStringField("type");

        if (MessageType == "DeviceType")
        {
            FString DeviceType = JsonObject->GetStringField("device");

            if (DeviceType == "mobile")
            {
                // Enable virtual joystick
                bEnableVirtualJoystick = true;
                EnableNativeTouchEvents();
            }
            else
            {
                // Disable virtual joystick
                bEnableVirtualJoystick = false;
            }
        }
    }
}

Enabling Native Touch Events

When using the on-screen joystick for mobile devices, it's recommended to enable native touch events for smoother interactivity. You can enable native touch events in Unreal by setting the appropriate flags or configuration in your project settings.

void AYourPlayerController::EnableNativeTouchEvents()
{
    // Code to enable native touch events
}

By following the steps above, you can effectively enable on-screen touch controllers for mobile browsers while keeping them disabled for desktop browsers. This approach ensures that the user experience is optimized for each device type.

Introduction

When working with the PureWeb Reality streaming platform, particularly with custom web client, developers often encounter issues related to status messages and correct timing of sending messages between the game and web client.

A common problem is the failure to receive the "serviced" status, with the logging stopping at the "ready" state. This guide aims to provide practical solutions to ensure smooth communication between the game and the web client.

Status Messages in PureWeb Streaming

PureWeb's platform provides various status messages to indicate the connection and readiness states of the platform and the stream. These statuses are crucial for understanding the current state of the application and the streaming process. However, it is important to note that these status messages do not convey the readiness status of the game itself. This distinction is vital for troubleshooting.

Key Status Messages

1. Ready: Indicates that the application is ready but does not guarantee that the game and web client are fully ready to communicate.
2. Serviced: Indicates that the web client has successfully connected and is receiving service.

Timing Challenges in Message Communication

One of the most challenging aspects of using PureWeb Streaming is ensuring the correct timing of initial messages between the game and the web client. If either side sends a message before the other is ready, the message will essentially be lost. This timing issue is a common reason for not receiving the "serviced" status.

Example Scenarios

• Game Sends Message Prematurely: If the game sends a message before the stream has fully connected, the web client will not receive it, causing the status to remain at "ready."
• Web Client Sends Message Prematurely: Similarly, if the web client sends a message while the game is still loading assets or starting up, the game may not be ready to process it, leading to missed communication.

Practical Solution: Delaying Game Initialization

To address the timing issue, introducing a delay before the game starts sending messages can be an effective solution. This ensures that both the game and the web client are fully ready to communicate. Below is a sample implementation for Unity that demonstrates how to introduce a delay on startup.

Unity Code Example

public class GameManager : MonoBehaviour
{
	public GameObject objectToLoad; // Assign your prefab here in the Inspector   

	void Start()
	{
    // Use a condition or a delay to control when to load your GameObject
    Invoke("LoadObject", 5); // Call LoadObject method after 5 seconds
	}

	void LoadObject()
	{
    	Instantiate(objectToLoad);
	}
}    

In this example, the Invoke method is used to delay the loading of the game object by 5 seconds. This delay ensures that the game is ready before it attempts to send any messages to the web client.

Conclusion

Understanding the status messages and timing issues in PureWeb Streaming is crucial for ensuring smooth communication between the game and the web client. By recognizing the importance of synchronization and implementing practical solutions such as introducing delays, developers can prevent common issues like missing the "serviced" status. This guide serves as a resource for troubleshooting and resolving these issues, ensuring a more reliable streaming experience with PureWeb.

Our web client template provides some default checks to ensure that a stream is not kept active indefinitely and disconnected if the user becomes inactive.

You can adjust these values as needed to suit the needs of your streaming experience.

• WarningThreshold - when this duration is reached, a warning will pop up to ask the user if they're still there.
• ExitThreshold - is how long it will wait after the warning threshold is reached. If there is no activity the stream will disconnect.

These values can be found in your web client template's App.tsx file, in the IdleTimeout section (around line 165).

The values are in seconds (ex: 300s is the equivalent of 5 minutes).

Note that this method does not fully replicate a CloudXR connection on the PureWeb Reality Platform, and might not replicate all issues you may encounter in building, packaging and deployment.

To identify problems early, use this workflow only after your model is packaged and ready to be deployed, or for troubleshooting if a cloud-hosted model is not working after initial deployment.

Local Workflow

Environment Configuration

Before starting please ensure your local environment is set up as follows:

• A supported VR headset (Meta Quest 2/3/Pro)
• A PC with a GPU capable of running the application (Nvidia A6000 or A10G equivalent recommended)
• The PC has the Steam Client installed
• Both the headset and PC must be connected on the same network (with firewall exceptions opened if necessary)
• Your VR model packaged correctly
• (Optional) PureWeb clients side-loaded on the headset

Install SteamVR

Download and install the SteamVR application to your Steam Client Library.

Launch the SteamVR application.

Add your model as a Non-Steam Game in the Steam client by left-clicking on Add a game > Browse to location

Now navigate to the location of your package VR model's main executable and select it, then click "Add Selected Programs."

Connecting to the Game

Without the PureWeb clients

If you want to connect without the PureWeb clients to test the model or don't have the optional clients setup on the headset:

Connect the headset to SteamVR through Steam Link (Steam Link should be installed by default, if not you can download it from the store).

From the headset Library, under All apps select Steam Link, which should find the PC that has SteamVR running.

Click "Connect" to connect to your PC.

Now you should load into the Grid "Steam waiting room"
(Note: This view is configurable and may look different if it has been updated.)

With the PureWeb clients

If you want to connect with the PureWeb clients to test the model on the headset:

• In the client.json file add a new property:
  "overrideIP": "[IP_Address_Of_Your_PC]"
• Copy this file to the headset similar to the step described in the Configure the Client during Setting up your MetaQuest3 with PureWeb CloudXR Client.

When the overrideIP parameter is included, the client will ignore other parameters and launch directly to this IP address, so be sure to remove or adjust it when reconnecting to the platform to avoid connectivity issues.

• Launch the client from your headset as you normally would.
• In the Steam client, run the model you added earlier from Steam Library .

At this point, your model should either connect or display issues related to the model, build or packaging process. Common issues might include:

• No video stream (black screen)
• The model displaying in a large desktop view.
• Steam errors due to incorrectd model configuration with OpenXR.
• The client failing to connect and reverting to the horizon OS bar.

Should you encounter any of the above, please refer to our CXR troubleshooting guide.

Although we provide a comprehensive ReactJS template for building out a custom web client, you may want to use something different such as Next.js.

Node.js & Next.js

When integrating the PureWeb SDK with a Next.js project, you may encounter issues with certain Node.js modules that aren't intended for front-end use. This article documents a real-world scenario and explains how to resolve such issues, ensuring a smoother integration process.

NOTE: Whether using our React.js template or porting over to a more custom solution such as this, once you begin customizing your web client the responsibility for hosting, maintaining, supporting and testing it is that of the license-holder.

When porting from the React-based project into a Next.js environment you may encounter compiler errors related to Node.js-specific modules (tls and  fs ). These modules are required by the PureWeb SDK but are only meant for server-side use and not for the browser. See below for examples

Module Not Found Error

• Error: Module not found: Can't resolve 'tls'
• Cause: Next.js tried to include the tls module, which is not available in the browser environment.

File System Module Error

• Error: Module not found: Can't resolve 'fs'
• Cause: Similar to tls, the fs module is also specific to Node.js and not available in the browser.

These errors typically arise because Next.js, by default, may attempt to bundle these Node.js modules into the client-side code.

Solution

To resolve these issues, we need to adjust the Webpack configuration in the Next.js project to exclude these Node.js modules from the client-side bundle.

Step-by-step guide

Modify the Next.js Webpack Configuration

Open your Next.js configuration file (next.config.js) and add a custom Webpack configuration to handle the module resolution issue.

next.config.jsmodule.exports = { 
    webpack(config) {
        config.resolve.fallback = { 
            // Preserve existing fallbacks
            config.resolve.fallback, 
            // Add fallbacks for problematic modules      
            fs: false, tls: false, }; 
        return config; }, };

• config.resolve.fallback: This Webpack option allows you to specify fallbacks for modules that cannot be resolved. By setting fs and tls to false, you're instructing Webpack to ignore these modules in the client-side bundle.

This configuration effectively silences the errors caused by these modules and allows your Next.js application to build and run without issues.

Testing the Configuration

After making the above changes:

1. Rebuild your Next.js project,
2. Ensure that the application builds successfully without the previous module errors,
3. Verify that the PureWeb SDK functions as expected in the browser environment.

Additional Tips

• Check Dependencies: Ensure that other dependencies in your project do not also try to include Node.js-specific modules. Apply similar fallbacks as needed.
• Environment-specific code: Use environment checks to conditionally import or use modules that are specific to the server-side, ensuring they are not included in the client bundle.

if (typeof window === 'undefined') { 
    // Code here runs only on the server  
    const someServerSideOnlyModule = require('some-server-side-only-module');
}

Conclusion

By configuring Webpack to handle Node.js-specific modules properly, you can successfully integrate the PureWeb SDK into your Next.js application without encountering build errors. This approach ensures a seamless transition from a standard React environment to Next.js, leveraging the server-side rendering capabilities while avoiding client-side issues.

This solution will help developers avoid similar issues in the future, providing a smoother development experience when working with Next.js and PureWeb SDK.

What do you need?

• a 3D game package developed in Unity or Unreal
• a deployed web client

How does it work?

The PureWeb Reality platform is a fully managed provider of interactive metaverse experiences.  You provide the 3D model, and the Reality platform handles infrastructure management, model deployment, global routing & load balancing, scheduling and much more.

Following upload through our Console, your 3D game package is directly fed into our deployment pipeline. The first time you upload a model, our team will rapidly prepare your streaming infrastructure. Following the initial set up, you can update your model through the console at any time.

Finally, your users will connect to your game package using a web client. Our platform provides a default preview client for rapid feedback and testing. For production deployment, we provide a web client template that can be configured in seconds.

Once deployed, the web client seamlessly handles coordination of streaming session connections, region selection, queuing and more to ensure the fastest time from request to delivery of a stream in the user's web browser.

In short:

1. You upload your prepared 3D game package through the PureWeb Reality console.
2. Optionally create and deploy a custom web client to your desired hosting service.
3. Users can begin streaming from their web browser on their device of choice through your custom client or our preview client.

Scheduling & Auto-Scaling

For timebound events with a large number of users, you'll want to discuss reserving server capacity with your sales representative ahead of time. The PureWeb Reality platform can dynamically scale the number of servers up (or down) over time to adjust to user demand during your event.

What Next?

Head back to our Getting started with PureWeb Reality guide!