Twitch Extensions: Part 2 – Integration Points

Welcome to part two of the Twitch Extension Series of Posts.

This week, we’ll be talking a little on Integration Points!

What integration points does a Twitch Extension have?

We touched on this in Part 1

Twitch provides developers with three main, two auxiliary, and two Broadcaster only integration points, and of these integration points Twitch lets us pick the HTML (and thus Javascript or CSS) file(s) we wish to load for all of these. (Well except Panel Popout, screw that guy, it owes me a tenner).

Hang on just a minute

What? OH! You want to know how to create an Extension first or where to set the HTML to be loaded for each view? Yes, that would make a little more sense, wouldn’t it!

Extensions can be created, configured, and release via the Twitch Developer Console.

The Developer console can be found at https://dev.twitch.tv/console

After logging in you’ll generally land on the Summary page that will list you current Extensions, Applications and any Games/Categories you may “own” on Twitch from being part of an organization

An example of the Twitch Developer Dashboard Overview
An example of the Twitch Developer Dashboard Overview

We’ll ignore everything else since we are only interested in Extensions.

Twitch Developer Console for Extensions
Twitch Developer Console for Extensions

If you click Extensions then “Create Extension” it’ll take you through a short “Setup Wizard” before presenting you with your ClientID for the Extension. We’ll do that now so we have an Extension to play with during this series!

PRO TIP: After starting the Wizard, please finish the wizard, even if you intend to change everything later.

  • Asks for your Extension Name, then Click Continue
  • Asks you to pick the views/integration points you want, which you can change later
  • Provide a Version number, 0.0.1 will suffice for now
  • Add additional details such as the Extension description and contact details for you
  • Now hit “Create Extension Version”

Twitch will send you an email to verify the provided contact details, so click the link in those email(s).

We’ll cover most of the other fields in a future post, but today we are interested in integration points.

Twitch will now have dropped us on the status page for our Extension

It’s important to note that you cannot use “Twitch” in the name of you Extension, and the name needs to be unique across Extensions and Applications across all of Twitch

So Integration Points?

On the Extension Status page, hit “Asset Hosting”, this will take us to the page to configure our “Extension Views” and what html to load. Twitch will prefill with something sensible, but you can use anything you want

"Asset Hosting" section of a Twitch Extension Console
“Asset Hosting” section of a Twitch Extension Console

At the top is the Testing Base URI, we are going to ignore this for now, but we’ll be covering it next week, when we might actually start building.

Below that comes the section that lets you pick via Checkbox, which views (for the viewer) you wish to enable. And the settings for each view.

Panel

A Panel is rendered below the stream, in the panels section, Broadcasters have “some” control over where a panel extension will appear in relation to the other panels, Twitch liks to jumble things round sometimes, but generally Panel Extensions are pretty sticky and reflect the broadcasters choice.

A panel has the following settings available

  • Panel Viewer Path – the path and file name of the HTML file you wish to load, it is a relative path to the Testing Base URI/final upload URL
  • Panel Height – a panel extension is fixed Width (318px), but the developer can choose what height to use, ranging from 100px to 500px

A Panel extension, can be popped out (one of the Auxiliary Integration points we mentioned), which will start at 318px wide and the specified height, but the Viewer can then resize this window. It’s something to be aware of when building your extension, and offers the ability to use Responsive design

Video – Fullscreen

A Video Fullscreen extension will cover the whole stream, so the developer and designer can utilize the whole stream. Usually it’s safe to assume that the size is 1920px x 1080px, and you can scale as needed, we’ll cover some ways to handle this in a future post

A video Fullscreen only has one setting, the HTML file you wish to load, it is a relative path to the Testing Base URI/final upload URL

Mobile

YES, THE CHECKBOXES GO: Panel, Video – Full, Video – Comp, Mobile but the page goes in a different order

The Mobile view is presented to Viewers using a Mobile device in the Twitch App for that device, such as an Apple iPhone, Apple Table, Android phone and so on.

Like Video Fullscreen there is only one option here, the HTML file to load, it is a relative path to the Testing Base URI/final upload URL

Mobile you’ll need to practice Responsive design as dimensions will vary by device and orientation of the device

Video – Component Viewer Path

A video Component roughly works similar to a Panel, but it defaults to being “closed” and can be manually opened by the Viewer by clicking the relevant icon in the “taskbar”, the Taskbar is presented to the viewer on the right of a Live Stream.

The Extensions Taskbar, left is shown a Video Component Extension in the closed state and right is the open state

The following options are available

  • Video Component Viewer Path – the path and file name of the HTML file you wish to load, it is a relative path to the Testing Base URI/final upload URL
  • Video Component Sizing Parameters – there are a number of options here we’ll cover this more in depth at a future post as it can get fun!

Non Viewer integration Points

That covers all the Viewer intergration points and their options in summary

Next we have the “Broadcaster” Integration points, there are two of them, both have the same available settings, the path and file name of the HTML file you wish to load, it is a relative path to the Testing Base URI/final upload URL

Configuration view

This is the main/one off configuration view, when a broadcaster first installs your Extension they will be prompted to ask if they want to visit the Configuration page or not. This page is used for one off or infrequent configuration

The Extension Config prompt during install
The Extension Config prompt during install

In some cases, developers can also block an extension being activated if the configuration has not been done, usually this would be used for things such as providing an API key to access another API, or asking the broadcaster what their Destiny 2/other game Character name is (for example)

The Twitch Extension configuration page for FlightSimTrack
The Twitch Extension configuration page for FlightSimTrack, this example provides additional setup instructions and the API Access Key for the Companion Desktop App

It can be reached from the install flow for an extension and from the “Cog” Icon on an Extension in the Extension Manager section of the Dashboard

In terms of Dimensions it’s usually Landscape but a broadcaster can resize it pretty narrow, so you’ll need to be responsive or provide a minimum width

Live Configuration View

The Live configration view is to be used by the broadcaster for common tasks or things that need to be done during a Stream, examples of this would be to start a Poll for a poll extension, or a “I’m starting a round” in a mini games extension.

The Live configuration view is reached by the Broadcaster from the “Quick Actions” Section of their “Stream Manager”

A Twitch Dashboard Quick Action Button
An example of a Quick Action Extension Button

When clicked, a new window will open, which will contain the Quick Action/Live Configuration View, it’s resizable by the broadcaster so you’ll need to practice responsive design again here

An Opened Quick Action
EliteTrack’s Quick Action view provides easy access to a summary of the data sent, and prompts the user about Bug Reporting

Summary

And that covers the various integration points in a bit more depth compared to last weeks post!

Next week, we’ll be covering “Getting Started Building” I think and why every time I mention a HTML file did I follow it with: “the path and file name of the HTML file you wish to load, it is a relative path to the Testing Base URI/final upload URL”

BUT MOTHER I CRAVE VIOLENCE

Well, until I write the next part if you want to read more about the Developer Side of Extensions, you can pop a visit over the to the Documentation or take a look at Twitch’s Introductory Page and you can always join us on the “TwitchDev Discord Server”, visit the Developer Support Page for the current invite link!

Why you think you are good enough to even write blog posts on Extensions? I made a one or two of them Extensions of various types.

Twitch Extensions: Part 1 – An Introduction

This is the first part of a I don’t know how many parts series on Twitch Extensions. We’ll cover how to develop them, how to EBS them, some notes on Designing and what to avoid, some practicies for after release care, and touch on some marketing ideas/things for your Extension Website.

So since this is the first part, we’ll cover the basics first. What, Where, Who, How, and Why of an Extension

[caption id="attachment_51846" align="aligncenter" width="740"]When is Gamora Why is Gamora[/caption]

This is the first part of a I don’t know how many parts series on Twitch Extensions. We’ll cover how to develop them, how to EBS them, some notes on Designing and what to avoid, some practices for after release care, and touch on some marketing ideas/things for your Extension Website.

So since this is the first part, we’ll cover the basics first. What, Where, Who, How, and Why of an Extension

When is Gamora
Why is Gamora

What is a Twitch Extension

A Twitch Extension is basically an iFrame that allows a developer to create anything they want, as long as it fits within the Guidelines set out by Twitch, and of course it’s Terms of Service and Developer Agreements

This can range from MiniGames, to QnA/Polling extensions, to Community information extensions, to game information extensions, or play with the Streamer extensions

Some examples include

  • Sound Alerts – Lets users pick a sound to play on Stream, can be free or utilize bits/channel points
  • Crowd Control – Provides plugins for a variety of games, to allow people to interact with the game, make it easier/harder for the Broadcaster
  • Cardboard.live – Lets viewers see what cards are in your current deck, and check the state of the game board, without having to spam/ask in chat
  • Borderlands 3 ECHOcast – Lets viewers check out your Borderlands 3 character, and let viewers win extra loot for their own character when the Broadcaster opens Red Chests in game
  • Detroit: Community Play – Ask the community to pick/vote on the option when a multiple choice question appears during the game play of Detroit: Become Human.
  • The Cohhilition – A community interaction extension, that provides access to various Community things (in this case for a single channel), without having the viewer leave the comfort of the Twitch page

Some of these descriptions are super simple to cover the salient points and many will do more things than my summary covers

Check out more extensions at Twitch’s own Extension Discovery

Where is a Twitch Extension

Twitch being a live streaming site, provides to the Broadcaster, a page, and that page will consist of a number of elements.

Depending on if the streamer is live or not the elements on the page will vary slightly. If the Broadcaster is live, you’ll land on the video/chat page, if the Broadcaster is not live, you’ll land on a “Home”/index style page

Now the part that we care about is the “Chat”/live view page, and on that page you will find a number of sections

An example of a Twitch Channel Page
An example of a Twitch Channel Page
  • The Video Player
  • The live Chat
  • The Stream information section – The Title and Category
  • A small about the Broadcaster section
  • The Panels section

A Twitch Extension can be added to a couple of these sections, and has 3 main (and two auxiliary) integration points.

The Main Integration points

  • Video Overlay – The Extension can cover/utilize the whole of the video player
  • Panel – The Extension appears in the panels section below the stream, and has width of 318px and a maximum height of 500px
  • Video Component – Basically a panel that appears over the video player but is locked to the right hand side of the player, it can utilize a varied amount of the player space

The “main” integration points are mutually exclusive, an Extension can only occupy any one of those slots at once

The Auxiliary Integration points

An example of a mobile extension on iOS
An example of a mobile extension on iOS
  • Mobile Panel – The Extension is available on mobile for mobile users to interact, it will replace the chat, and dimensions wise basically similar to a panel on PC (in terms of ratio), but you would have to consider landscape views on tablets as well.
  • Panel popout – Panel Extensions can be opened in a new window and can be resized by the user at will

An extension can be in one of the “main” integration points, and the mobile point.

Who is a Twitch Extension

As part of being on Twitch, extensions are able to use a number of Extension Features, as well as doing more “regular” Twitch stuff. You could run a regular chat bot that runs with your extension, the Twitch Extension Timeout with bits does this in order to run the actual timeout commands on users

So aside from the “regular” stuff like chat bots, Twitch Extensions have access to some additional features

  • Bits Support – Allow Viewers, to exchange bits (a digital good) for various “digital goods” inside Extensions, this could range from an extra vote in polling extensions, or picking a victim in “Timeout With Bits”, or a cool cloak for your character in a game. Revenue generate here is split 80/20 between the Broadcaster/Developer.
  • Subscription Support – Allows the Extension to check the subscription status of a viewer on the channel the extension is installed to, avoiding the need for the Extension Developer to get and maintain oAuth access tokens from the Broadcaster “separately” to the install process of the Extension
  • Identitiy Link – Allows viewers to “login’ to your extension, we’ll cover this more in a later post in the series
  • Chat Capabilities – Allow the Extension to send chat messages (via a HTTP POST request), usually used as a notification system to prompt viewers to perform an action in the extension, like a new poll has started, go vote, for example
  • The Configuration Service – We’ll cover this in a later post as well, but it’s a way to store data on Twitch’s server that you can use in the extension, this might be something like, the name of the Broadcasters Character in a game that you would use in an API request to get information about the character
  • Streamer Allowlist – allows the Extension Developer to restrict whom can install the Extension to their channel

We’ll cover each Capability/feature in future blog posts in the series

How is a Twitch Extension

We’ll cover this more, in depth in later posts, a Twitch Extension is a bundle of files uploaded to the Extension CDN (Content Delivery Network). This needs to include your HTML, JS, CSS, and any static images you want to store on the CDN (pretty handy for background images for panel extensions). Twitch has some restrictions on what an extension can load from external sources, but essentially images are fine, CSS/JS is not, CSS/JS must be local/included.

Those files are uploaded to a sub domain of Twitch, into a particular sub folder tree on that sub domain, which we will cover more in depth in a later post, when we talk about building extensions and a suitable way to test them and some related gotchas.

All Twitch Extensions have their bundles uploaded to the Twitch and before they are released (or updated) to the masses, the Twitch Extension Review team will review the Extension, to ensure it works as intended, there is no major bugs effecting activation, the Extension compiles with the Guidelines and Terms of Services, and most importantly contains nothing malicious to interfere with the Twitch website or the viewer using the extension Computer/device

A Twitch Extension is allowed to communicate offsite, the resource just has to be secured over SSL, this is commonly referred to as an EBS or Extension Backend Service, we’ll cover this more later as well!

Why is a Twitch Extensions?

But Why is a Twitch Extension (any use) I hear you cry?

A Twitch Extension provides ways for the Streamers Community to perform rich interactions, without leaving the Twitch Broadcasters page, which means you keep the Viewer watching the Stream or interacting with Chat, with relatively easy access to Twitch API’s, without long additional steps for Viewer Authentication

Summary

That is it for Part 1 in this series on Twitch Extensions, I’ve cover the basic What, Where, Who, How of Twitch Extensions

Parts will either be weekly or bi-weekly, we will see how we go!

BUT MOTHER I CRAVE VIOLENCE

Well, until I write the next part if you want to read more about the Developer Side of Extensions, you can pop a visit over the to the Documentation or take a look at Twitch’s Introductory Page and you can always join us on the “TwitchDev Discord Server”, visit the Developer Support Page for the current invite link!

Were the sub headings supposed to make sense? No not really.

Why you think you are good enough to even write blog posts on Extensions? I made a one or two of them Extensions of various types.

Twitch API Examples

I spend a lot of time on the Twitch Developer forums and Discord helping out other third party developers. That among other things led to me being asked to become a Twitch Ambassador, which is probably a story for another post.

As part of spending a lot of time helping of Forums/Discord, it become useful to write up some examples in various languages for people to refer to, since some people prefer code examples over documentation, and it’s easier to demonstrate how to tie multiple calls/endpoints together for the desired result.

To that end my GitHub Repo at barrycarlyon/twitch_misc now exists and holds examples from Authentication flows (from Implicit to server access and regular user in-between), extension config/pubsub, and examples for Webhooks and the new Eventsub (which is worth a look!). So if you are looking for some examples do checkout the Repository. Some of the examples can even be tested on GitHub itself via GitHub pages, the examples available are listed in the readme and at the Github Pages site.

Twitch also recently made the requirement that all calls to helix (aka the New API) need to be Authenticated using a Bearer, which made it difficult for Extensions to get the viewers details. So to that end I created a basic example of how to do that in an Extension with a “User Profile Extension” example. Which is at BarryCarlyon/twitch_profile_extension. So this covers a good way to handle that flow.

Right now most of the examples are nodeJS, or PHP, but there are some in Python kicking about!

I’ll be looking at adding more examples and other examples in other languages as we go!

I’m usually really bad at commenting my code as I prefer reading the code, but I made a conscious effort to add useful code comments on these repos!

Google oAuth and offline access

Been doing a lot of various stuff and things for CohhCarnage and some of that stuff has involved building an achievements tracking system for the website.

One of those achievements, is for YouTube Subscription. Where the achievement is awarded to the logged in user, if the user has subscribed to a given YouTube channel, in this case Cohh’s YouTube.

In order to make sure that people can’t “cheat” the system, we ask them to link their Google/YouTube account with the website and use the relevant API to look up their Subscription status.

Initially this worked fine, but I ran into some issues where the oAuth token stored has expired and thus I can’t do a status check, for cases where the user links their YouTube to their Cohhilition Account then doesn’t subscribe on YouTube until after 24 hours later (or some caching issue with Google).

So, the simple fix using Googles PHP Library for oAuth’ing is to just do a

<?php

$client->setAccessType('offline')

Now, this works fine for the most part, you happily get a refresh token, and can thus renew your token.

Then comes a hiccup, if for whatever reason you have offline access type on, and the user has previously authorised the application and it’s offline permission, you DON’T get a refresh token in some cases. Some user cases include:

  • you’ve lost their token,
  • or got a bad one
  • or the user managed to find the authentication loop (again) when they shouldn’t, and thus a new code/token combo is generated

Normally you are using something like:

<?php

$client = new Google_Client();
$client->setClientId($client_id);
$client->setClientSecret($client_secret);
$client->setRedirectUri($redirect_uri);
$client->addScope("email");
$client->addScope("profile"); 
$client->setAccessType('offline');// last forever/give me a refresh

But in order to make sure that you get a refresh_token EVERY time someone goes through the authentication loop, you have to adjust as follows:

<?php

$client = new Google_Client();
$client->setClientId($client_id);
$client->setClientSecret($client_secret);
$client->setRedirectUri($redirect_uri);
$client->addScope("email");
$client->addScope("profile"); 
$client->setAccessType('offline');// last forever/give me a refresh
$client->setApprovalPrompt('force');// force a refresh token return everytime

Apparently, using

'offline'

is supposed to imply

'force'

according to some Stack Overflows posts, but this doesn’t seem the case.

In the end my full Google_Client setup looks like:

<?php

        $client = new Google_Client();
        $client->setAuthConfig($consumer);
        $client->addScope('profile');
        $client->addScope('email');
        $client->addScope('https://www.googleapis.com/auth/youtube.readonly');
        $client->setAccessType('offline');// asks for a refresh token
        $client->setApprovalPrompt('force');// forces the refresh token being returned
        $client->setIncludeGrantedScopes(true);
        $client->setRedirectUri($callback);

Just an odd thing I came across recently that I thought I would write up. Most of the notes here are from Stack Overflow post on the subject

PHP5.6+ cURL and file uploads

Came across something odd today, and thought I’d condense down my Tweets on the subject into a blog post.

Basically, I use cURL and some wacky wacky stuff to upload files to a site over HTTP POST. And since I’d just grabbed PHP 7 from HomeBrew, it had overridden my PHP 5.5 install that comes as standard on OSX 10.10 and thus I cross checked the script with my MacPorts PHP 5.6 install and found the same. (Yes THREE different PHP versions for science…)

“Traditionally” the method for this would be something along the lines of:

<?php

    $ch = curl_init('SOMEURL');
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36');

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $data = array(
        'some_file' => '@' . $some_path
    );
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

    $r = curl_exec($ch);
    curl_close($ch);

On PHP 5.5 and previous that works file, using a @ at the start of a POST entry would instruct PHP/cURL to treat the data/string as a File Path to Upload.

This behaviour is controlled by the PHP cURL constant of CURLOPT_SAFE_UPLOAD. In PHP 5.6 this constant changed from default FALSE to default TRUE, setting to TRUE means that a string starting @ is treated as a String and not a File Path to upload. The changes are documented on the PHP.net website, but the primary trip up is that most of us just use the defaults and we get tripped up when things change.

So, after trying to set this to FALSE under PHP 5.6 it still wasn’t working, and under PHP 7 you are thrown an error to indicate that you are not allowed to change this constant any more for security reasons, which is fine.

The solution is to use the CURLFile class, which is pretty straightforward:

<?php

    $ch = curl_init('SOMEURL');
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36');

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $data = array(
        'some_file' => new CURLFile($some_path)
    );
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

    $r = curl_exec($ch);
    curl_close($ch);

This is the truly lazy edition, just chuck a new CURLFile($path) at it, instead of the @. I’m sure CURLFile does more useful stuff, but this was enough to get me back up and running!

Thus endeth this blog post!