Q: Can I use the virtual keyboard on iOS and Android devices to collect user input for my streaming game?
A: Yes...sort of.
In a streaming solution on the PureWeb Reality platform, it's important to remember that the streaming game (whether it be Unreal or Unity) is running in a Windows environment in the cloud. That means the game itself has no information about the browser or operating system being used by the user; unless you specifically send that information to the game from your custom web client.
When streaming to a desktop browser, the web client can process keyboard input through the stream without issue. It can become more complicated when streaming to a mobile device, such as a phone or tablet.
Let's look at an example where we assume the stream is being consumed on a mobile device.
Suppose you wanted Unreal (or Unity) to ask the user for some text when they clicked a "Search" button in the game. Since the user's device has no physical keyboard, and the streaming game doesn't know anything about the browser or user's OS, there is no way for the user to type in their search string.
The game would need to send a message to the web client stating that it requires some type of text input. The web client would need to listen for such a message, and when received, request it on the user's device. After collecting the user input, it would need to then be sent via a different message back to the game so that the game could actually execute the search.
When the "Search" button is clicked in Unreal, it would then send a message to the web client, and would need to contain information for the web client to listen for in the Descriptor.
It might look something like this:
{"Subject": "getinput", "Type":"text", "Variable": "var_search"}Your custom web client can be edited to listen for this incoming message. When received perhaps you might display a window with a text field and a "Submit" button. The user can type the needed data and click the button. Clicking submit would send the user input back to the game as a new message.
Finally, the game would need to listen for this new message and perform the appriopriate actions when it was received.
That's a lot for just collecting some user input, isn't it?
As an alternative, it may be simpler to build a search dialog in your custom web client. That way when the browser asks the user for data, the user's device OS knows what to do, such as display the on-screen keyboard. Once collected, you can send that message into the game and execute it all on that side, reducing the complexity of the messaging needed to do the same thing.