In our 0.3.0-beta.1
SDK release we’re excited to announce the beta for the new ‘Bring Your Own Frame Buffer’ (BYOFB) feature in addition to bugfixes and improvements. In order to participate in this beta release, all SDK runtimes must be updated to 0.3.0-beta.1
to continue to communicate with each other - this beta offering is not backward compatible with our stable 0.2.8
release.
Bring Your Own Frame Buffer
We’ve introduced a new set of functions to the Rainway SDK that allow for direct submission of audio and video data within the context of a Rainway Stream.
These include rainway_submit_frame(...)
, rainway_submit_audio(...)
for submitting video and audio data, and an onStreamInput(...)
callback to process remote input.
We expect that most folks will leverage this new functionality via our C++ SDK, so we’ve included some snippets below demonstrating how it works at a high level, using C++. For example, to submit video frames:
// stream to submit frame / audio to.
// You can obtain a Stream from the SDK's `onStreamStart` event.
Stream stream;
extern winrt::com_ptr<ID3D11Texture2D> renderFrame();
auto frame = renderFrame();
stream.submitFrame(frame);
and to submit audio:
Stream stream;
extern int16_t *audioBuffer;
extern usize_t audioBufferSamples
auto submission = RainwayAudioSubmit {
44100, // samples per sec
2, // channels
RainwayAudioBuffer {
RAINWAY_AUDIO_BUFFER_PCM, // buffer type
audioBuffer, // buffer of samples
},
audioBufferSamples, // number of samples in buffer
};
stream.submitAudio(submission);
Handling stream input is done with the onStreamInput
event:
config.onStreamInput = [&](const Runtime &runtime, const Stream &stream, const Peer &peer, const RainwayInput &input) {
switch (input.tag) {
case RAINWAY_INPUT_MOUSE_ABSOLUTE:
mouseAbsoluteInput(peer, input.MOUSE_ABSOLUTE.x, input.MOUSE_ABSOLUTE.y);
break;
case RAINWAY_INPUT_KEYBOARD:
keyboardInput(peer, input.KEYBOARD_INPUT.keycode);
break;
// ...
}
};
It’s important to note that while we believe BYOFB will be extremely valuable for several use cases, our high-level APIs aren’t going away! You may continue to build applications with Rainway automatically handling capture and input exactly as you would today.
Improved Support for Multiple Peers and Streams
It is now possible for two or more peers to subscribe to the same stream. In addition, we’ve also added the ability to host several streams at the same time, provided you have a graphics card that supports it.
These improvements are available for both our new BYOFB streams and existing full-desktop or App Isolation streams.
New Stream Related Events
Native SDK consumers can now subscribe to several new events that relate to the lifetime of a stream (whether using BYOFB or not).
Event | Description |
---|---|
config.onStreamStart |
A local stream has started. |
config.onStreamEnd |
A local stream has ended, normally as the result of all other peers leaving. |
config.onStreamPeerJoin |
A new peer has joined an existing stream. |
config.onStreamPeerLeave |
A peer has left a local stream. |
config.onStreamError |
An error occured relating to a stream. |
Bugfixes and Improvements
- [Documentation] Documented new BYOFB APIs.
- [Examples] Added a new C++ MP4 playback using BYOFB example demonstrating BYOFB capabilities.
- [Web SDK] Fixed types, making configuration callbacks optional.
- [Web SDK] Added
dispose
method on the runtime, to faciltate Rainway SDK cleanup. - [.NET SDK] Fixed an issue that could lead to an unexpected exception when a stream was terminated.
- [Native SDKs] Added support for querying active stream IDs to peers.
- [Bugfix] Made
createDataChannel
async in all SDK runtimes, since creation does not occur synchronously.