What is screen-scraping?

Screen scraping is the act of copying information from a digital display to be used for another purpose. Visual data can be collected from on-screen elements that appear on the desktop, in an application or on a website. In the case of streaming a game, that game runs on a remote resource, and is rendered on that computer's display, scraped and sent as a video stream to the user's browser.

Screen scraping is usually performed automatically with a scraping program. Due to this, anything that is displayed on the computer might be subjected to scraping, not just the game application being streamed.

What is pixel streaming?

With Pixel Streaming on the PureWeb Reality platform, your Unreal game runs on a virtual machine in the cloud. The game uses the resources available to that computer — CPU, GPU, RAM — to run the game and render each frame. It continuously encodes this rendered output into a media stream, which passes through a lightweight stack of Web services. Users can then view that stream in standard Web browsers running on other computers and mobile devices.

The result for the user is a lot like watching a video stream from YouTube or Netflix, except for two things:

  1. Instead of playing back a pre-recorded video clip, the stream is playing back the rendered frames and audio generated by the Unreal Engine in real time.
  2. Users can control the experience from their browsers, sending keyboard, mouse, and touch events, plus custom events emitted from the web page displaying the stream, back to the Unreal Engine. This allows for the creation of a tightly integrated and custom experience of which the game stream is only one component.

With pixel streaming, the user consuming the stream can only interact with the game application. The user has no access to the cloud machine itself, or information from other applications running on it.

So why choose Pixel Streaming?

Send the right frames at the right time

Screen scraping happens much later in the rendering stack than pixel streaming. An application being scraped must be run visibly on the desktop (which causes a performance hit), the displayed results must be scraped on regular intervals, then fed into a video encoding pipeline to be included in the stream.

By comparison, when pixel streaming an Unreal application, as soon as the virtual viewport has been composed in the engine, that viewport can be sent immediately to a video encoding pipeline, which saves precious milliseconds and provides a better streaming experience.

With screen scraping, the scraper doesn't know when the display is updated; the best it can do is scrape the screen and hope that it has been updated. So you might scrape the screen 60 times per second, but if the display is refreshing at a lower frequency, several of those scrapes are actually sending the same data as the scrape before. If the screen is refreshing more quickly, the scrape would be missing frames, and the stream would seem choppy to the end user.

In contrast, a streaming Unreal game is aware of when it has updated the virtual viewport, and every time it does, can trigger a call to the video encoding pipeline. This means if a game is running at 100 fps, we can stream at 100 fps, and if a game is running at 30 fps, we only encode at that frequency.

Integrate the browser experience

Code level interactivity is a huge advantage of pixel streaming.

In screen scraping, the game is essentially a black box. You can only interact with the game as though it were running on your desktop. You’re simply doing it through your browser. i.e.: You send mouse and keyboard events, but they are ‘virtual’ and go through the Windows APIs, which can result in quirky or unpredictable behaviour.

Since pixel streaming is integrated into Unreal Engine itself, all mouse and keyboard inputs go directly to the game. This has several benefits, including the ability to run multiple copies of a pixel streaming game on the same server at the same time (which is something you can do on our dedicated instances, but you could never do in a screen scraping solution).

Additionally, and probably most importantly, you can send commands directly to the game itself from the web browser and create a fully integrated experience with the game. You cannot create this kind of tight integration in a screen scraped solution. We have several examples of how to do this on the PureWeb Reality platform.