Twitch App Access Tokens can do MORE!

In the beginning App Access Tokens could just load public data, that didn’t need any oAuth scopes, from Twitch API. And so any given developers project had to maintain a user access token(s) for users of their application.

Then extensions were introduced, bundled in Extensions are Subscriber support, allowing extensions and their EBS to read a given viewers subscription status to a channel, outside of the Extension JS Helper function to check the status in client side code. That then leaves how does an Extension EBS (aka Backend Service) perform the same call (since you should never trust the client) to perform the same check.

And so Extension generated App Access Tokens, could call Get Broadcaster Subscriptions. The first endpoint that is scoped that an app access token could load on behalf of a user.

Since then various aspects of Twitch Developer Tools have come and gone, we lost WebSub, and we lost PubSub, and we lost both (over time) in favour of EventSub. There is a bunch of other stuff that happened but the important part here is that with the loss of PubSub, Eventsub achieved “featured parity” with PubSub and a number of other things introduced Chat over Eventsub.

EventSub

An Extensions ability to grant the ability for an app access token to call subscriber information is baked into the “install an extension” work flow.

An extension that offers the Subscription and Chat permissions (chat here has it's own endpoint using JWT), but subscriptions grant the power for app access tokens to be able to read.

But EventSub introduced the dual (or even triple) legged auth approach.

First a user will need to perform a user oAuth loop, either Implicit, Authorization Code, or DCF (Device Code Flow). This then creates a link between the user and the ClientID along with the scopes requested.

Combine that with the recently added Get Authorization By User endpoint to check what scopes your ClientID has to a given user, you can check permissions before trying to create EventSub Subscriptions for Webhooks or Conduits.

So for eventsub the pattern is:

  • Check the conditions and scopes needed for the EventSub Type/Topic
  • For the user’s in the condition get that user to user authenticate with the needed scopes
  • Generate an active app access token (if you don’t have one already)
  • Create the Subscription

As an aside you can use “user.authorization.grant” to easily monitor for new user authentications and create subscriptions as a follow up, if say your user token gen is air gapped from your Eventsub handler. This is a pattern I use on the IGDB Extension to monitor for authentications and create subscriptions to stream.online/offline and channel.update (which yes are public topics but become cost 0 _if_ the user is authenticate to the Client ID, and I use the user.authorization.revoke to unsubscribe since I only want to keep cost 0 subscriptions), becuase the IGDB Extension config page launches an implicit flow since I don’t need the user token ever I just need to auth so creating one with a refresh token is overkill, but I digress….

So that briefly is how EventSub dual legged auth operates.

Chat Over EventSub

Then along came Chat over Eventsub and a bunch of separate topics essentially providing IRC chat as a already JSON-ifed consumable feed.

This also came with Conduits and the ability for chat bots to scale to as many channels as needed, because the broadcaster auth’s the chatbot to be present on their channel. It is similar to how adding Discord Bot to a Discord Server works from a broadcaster point of view.

But chat over eventsub specified two user ID’s in the condition. (which yes can be the same ID but we’ll skip that for simplicity).

So first a developer would

  • Create a Client ID (generally owned by their own Twitch profile but the other of a Client ID is irrelevant, it’s always whom auths to the client that is important)
  • Create a Twitch account with the username of the bot they want to use
  • User Authenticate the bot to the Client ID with the relevant scopes
  • Then authentication any number of broadcasters to the Client ID
  • Now you can rejoin all channels with one API request (update shard), great for restarts or outages, as restart the websocket, call update shard, and then the bot is back in all rooms, no more slow roll rejoins or rate limit joins/etc.

So that is briefly the triple legged auth of chat over eventsub.

So What about App Access Tokens on the API?

Send Chat Message was introduced at the same time as chat over eventsub and is the first “proper” endpoint that worked with App Access Tokens following prior auth from the user(s) involved, (the broadcaster_id and sender_id)

This meant that for “general” operations a chat bot still needed to load and maintain either the broadcasters user token for Start Commercial for example, or the bots own token for Banning a User from chat.

So a bot would be juggling and maintaining up to three tokens,

  • A broadcasters token for broadcaster based operations
  • A bot token for doing moderation actions (for example), or sending chat
  • An app access token for subscription management (or sending chat)

But, now, as of yesterday! For the most part that is no more!

Well Spill the Beans then!

TWENTY ONE API endpoints now support this pattern of using an App Access Token after performing prior user(s) authentication(s). Look for the Beta tag under the Authorization section of each endpoint.

Here is the list:

Anything marked in bold with a * has additional scope requirements compared to the user token version, please check the docs, but I suspect your bot will likely already have obtained the scopes.

So now a chat bot broadly doesn’t need to maintain or even keep on file a user access token for itself or the broadcaster, as most general operations a bot is gonna do will work using the App Access Token. This also comes with the bonus that a user access code flow token is only good for 4 hours but an app access token is good for much longer. (Remember to utilise the Validate Token Endpoint to check time remaining on a token)

I’ll note that “Get Clips Download” supports this pattern, but that API was introduced with the pattern.

But a Sadness

However, the obvious missing endpoint that doesn’t support this pattern is Send Chat Announcement which I hope is supported soon, I think that’s the only one I need to token juggle for at the moment.

Summary

Thats it! Go forth an App Access Token! And hopefully more of the API will support this pattern over time!

If you need further help please join as over on the Twitch Dev Discord or Support Forums!

Bonus Thought

Cronjobs that need to do something on multiple users/broadcasters, the script starts, loads the token, does the job and just switches the user ID no token switchin’!

DEATH TO KRAKEN

Todays the day!

Twitch’s Kraken API gets turned off in checks watch just over three hours!

I know I’m just about ready, but if you are not, and you already missed the shutdown test windows, heres some notes!

There is a v5 Migration guide – https://dev.twitch.tv/docs/api/migration

DO NOT FORGET this also effects the Extension API endpoints (such as Send PubSub message and Send Extension Chat message), this doesn’t affect the Extension JS Helper!

For most server to server operations you can use a Client Credentials Access Token

Most people will likely be using Ajax calls in website front ends and are now confused how to migrate, there are two routes for this problem. You either ask the user to login with Twitch to provide a token to use (this is how my GitHub examples work) but not very user friendly, you don’t want to prompt the user to login to your website JUST to show them if your stream is live or not. The second route is your frontend needs to call your server and the server uses a Client Credentials token to call the API with.

Alternatively if you are doing something “simple” such as “check my stream is live” then you should consider moving from API calls to EventSub instead. Then when a channel changes Title or stream status, Twitch will tell you in (near) real time. Collect and store that data in a database, and now your website doesn’t need to call the Twitch API each time someone loads your website.

For further help migrating or to join us for the DEATH TO KRAKEN party (bring your own confetti) join us on the Twitch Dev Third Party Developers Discord you can find a link on the Twitch Dev Support Page

For reference here is the Dev Forums link with the shutdown window timings: https://discuss.dev.twitch.tv/t/legacy-twitch-api-v5-i-e-kraken-shutdown-reminder-february-28-2022/36589 the shutdown originally announced back in July.

If a feature is missing or theres something you need thats not in Helix, don’t forget to file a UserVoice. I know a lot of people need/want Clip Offsets so make your voice heard with UserVoices and votes on existing issues. See also my mega UserVoice sheet. And checkout the GitHub for known issues.

Latest Twitch Extension Release: IGDB.com Game Information

This will be the first post in a two/three parter on this extension since it also serves as a good example on how to utilise a number of Twitch and IGDB products/API’s to achieve the end solution! But today just an announcement post!

One of the features of the Dropped Frames Extension, that is used during Dropped Frames on itmeJP’s channel on Wednesday’s, is that I’ll push a Game Information box to the extension that shows information about the game the panel is talking about. That information box will collect information from the IGDB.com API and then present that information to the viewer. Heres a quick example screenshot for Mind Scanners:

Dropped Frames example of the IGDB.com information widget

You can see on the left the “box” with the cover art appears on the left over the Game name that is baked into the overlay and on user mouse over of the box the game information box is then displayed. Title, brief synopsis, cover art and store information are presented.

When I showed this off in the IGDB discord (prior to E3), I was asked, if this was a publicly available thing, and I replied, sorta but not really. Which then led me to go and create a seperate extension for anyone to use, during a smidge of down time I had recently!

So I present the IGDB.com Game Information Extension!

The first IGDB.com Game Information Extension screenshot

It is availble to both Twitch Mobile and Desktop users. On Desktop both Component and Panel are supported. I usually recommend the Component slot as it saves users having to scroll down, and it’s auto collapsed out of the way until a user clicks on the [IGDB] taskbar icon. And generally speaking most streamers have the Component slots free anyway. ITS FREE REAL ESTATE MAN!

The three views of the Extension. Information, Screenshots, Stores and Platforms.

Via the power of Twitch EventSub, it will even automatically self refresh if you change games during a stream and will gracefully fail back if you pick a game that is not on IGDB.com or a category that is not a game, such as Special Events or Just Chatting, it’ll just show the Twitch Box art/title instead.

Streamers will just need to link their Twitch accounts via the Extensions Configuration view, or the Extension Mini site, to enable auto updating!

Examples using Microsoft Flight Simulator, Elite Dangerous and the non game category Just Chatting.

To checkout the extension, or to see live streamers running the Extension, and to see the install instructions pop over to the mini site for the extension over at twitch.extensions.barrycarlyon.co.uk/igdb/ or visit the extension directly on Twitch.

The beauty of IGDB.com is that it POWERS Twitch’s categories and being crowd sourced, if the information is wrong, outdated or missing, anyone can submit a fix or change and await an IGDB Admin to accept the change.

I also wrote about the Extension over on the Twitch Reddit and on my Twitch Extensions Twitter, so feel free to spead the word!

And as a final note: this is a third party Extension not affiliated with IGDB.com, I just got permission to use their brand assets!

MineCraft Math

So, I tend to now and again do random bits and pieces to MineCraft Servers.

Of late I have been working with ZimDoorCraft, took the existing relatively static site, and flipped it over to a Mini PHP Powered Content Management System, that I wrote and roll out to smaller sites.
Couple of MySQL powered bits to make updates easier (Staff list and the like).

In time more complex and interesting things will be added (it was for a while running a Login with Mojang/MineCraft account, but that means you type your MineCraft login into the site…), how that works is a subject for another post. As is what ever I build next 😀

The MineCraft server in this case is running a derivative of Bukkit, which means it supports plugins, so we are using the JSON Api plugin, and making a call to the getWorld() data resource.
Which passes back information about the world in a nice handy format.

Heres an example

    [world] => stdClass Object
        (
            [remainingWeatherTicks] => 14670
            [hasStorm] => 
            [time] => 4964
            [environment] => normal
            [isThundering] => 
            [name] => world
            [fullTime] => 2903932964
        )

If your running Vanilla you can grab the same information just by decoding the level.dat file in the World Directory, however opening a file currently being written to, is not often a good idea, and the level.dat is written to a lot.

Like a lot of games, Minecraft uses ticks, in this case operating at 20 ticks per second, and running to 24000 ticks per Minecraft day, which means a Minecraft day is about 20mins.

Just to be interesting 0 ticks is Sunrise, rather than midnight. And midnight itself is 18000 ticks.

Also contained in the data packet is a count down to when the weather changes, in this case there are 7500 (ish) ticks to go.

Which works out as

14670 / 20 to get Real Seconds = 733.5 Seconds, and then its arbitrary to flip this to a human readable time. We have about 12 minutes, or half a MineCraft day, until the weather changes.

Reading out a more recent packet

    [world] => stdClass Object
        (
            [remainingWeatherTicks] => 7339
            [hasStorm] => 1
            [time] => 3827
            [environment] => normal
            [isThundering] => 
            [name] => world
            [fullTime] => 3902379827
        )

Looks like there is a storm on…. But its only Rain no Thunder and Lighting. With 6 mins to go.

On a side note!, watch out for FullTime, as its likely to run away, on a long running server, to something a 32 bit system might not be able to handle! Refer to my Post on Facebook and the Order ID on some thoughts about dealing with that.


Using Tick to Seconds math, means that it is also possible to display the current server time and weather on the website.

I’m using a bit of CSS, and the MineCraft time piece (craftable in game) to show the current time visually. In this case I’ve taken the 16×16 PNG’s and expanded them (by hand) to 32×32, finally converting the time from ticks to the relevant chunk of 360 degrees, offsetting as needed to account for the fact that Sunrise is 0 instead of Midnight.

I do this by adding 6000 ticks and from there treating the ticks complete as a percentage and scaling to meet 360 degrees. Some of the code involved needs a little tweaking and tidying to improve efficiency, but the angle is being calculated by a CRON Job powered script and stored with the full World JSON packet, in an extended object and cached in a file on the file system ready to be read by the web server when a request comes in. So it’s not a top priority to clean up yet.

I was finding that after taking into account the addition 6000 ticks to offset for Sunrise, the whole Sun Moon picture was upside down. So I just added 180 degrees, and corrected it when the angle was greater that 360. Since why bother rotating thru a whole 360 degrees?


$time = $world->time;// grab from data packet from JSON Api Call
$time += 6000;// add 600 for sunrise

// work out the scale factor 360 degrees / ticks in a Minecraft day
$scale = 360 / 24000;
$angle = $scale * $time;// calculate the <span class="hiddenGrammarError" pre=""><span class="hiddenGrammarError" pre=""><span class="hiddenGrammarError" pre="">angle
$angle</span></span></span> = number_format($angle, 0);// clean up
// correct
$angle = $angle + 180;// the image is upside down :-(
if ($angle >= 360) {// clean up again
	$angle -= 360;
}

The rotation component was quite difficult to initially decide how to do, PHP wasn’t quite cutting it so switched to a CSS Transform.
HTML wise it consists of a pair of divs, a pair of images, and a chunk of CSS.

<div class="worldtime"><div class="inner"></div></div>


WorldTime being the one behind contains the Sun Moon Picture as a background image.


And Inner contains the Watch overlay also as a background image.

Apply the CSS transformation to rotate the Sun Moon Picture is just a little nasty, because in rotating the Sun Moon picture it also rotated the overlay (inner).
I decided to apply rotation to the overlay, in the reverse direction (so just sticking a negative sign in front).
Works quite well, but if you can think of a better way to spit out this with the rotation please comment below.

Heres my current Style Code

<style type="text/css">
	.worldtime .inner {
		width: 32px;
		height: 32px;
		background: url('large_watch.png') left top no-repeat;
	    /* CSS3 then proprietary code */
	    rotation: -336deg;
	    -webkit-transform: rotate(-336deg);
	    -moz-transform: rotate(-336deg);
	    filter: progid: DXImageTransform.Microsoft.BasicImage(-rotation=4);
	}
	.worldtime {
		float: left;
		width: 32px;
		height: 32px;
		margin: 0 10px;
		background: url('large_inner.png') left top no-repeat;
	    /* CSS3 then proprietary code */
	    rotation: 336deg;
	    -webkit-transform: rotate(336deg);
	    -moz-transform: rotate(336deg);
	    filter: progid:DXImageTransform.Microsoft.BasicImage(rotation=4);
	}
</style>

The whole WorldTime Div is then wrapped in another Div and jQuery used to once per minute go and fetch the updated time, weather and CSS rotation.

At first I did try using the PHP function to rotate the image, but the quality drop off when rotating by an angle not divisible by 90 was shocking.
So ended up resorting to CSS, which does work a hell of a lot better.

This only covers how to do this using the JSON API, in a future post I’ll talk about decoding level.dat and other Minecraft Files.

Facebook Credits, the Order ID, and dealing with large integers in 32 bit PHP

So, currently at work I’ve been rewriting all our Payment Gateways for Your Members. Making them more better Class based an abstracting out the common functions to a base class, to save memory footprint and code etc.

And came across a problem when rewriting the Facebook Credits handler.

The order ID that gets passed around, as well as the Application ID is too long for a 32 Bit PHP installation to handle when treating it as an Integer.

So a number like “239724439419867” was ending up as “1.7592246582797E+14” when stored/processed, or converted to a string.
Now the obvious way to process this is to convert it to a string another way, which is somewhat difficult since before you even start it is in a format that you can’t handle.

The saving grace here is the fact that data is passed to you as a JSON packet (aside from the fact the Order ID is in the $_POST variable and thus a string, so I could of used it from there).

This led to me to look at the JSON packet, which starts as a string but when decoded the large integers are still a larger integer than can be processed (whether you json_decode to a Array or to an Object, the problem persists). So I thought about how to process the raw JSON packet and make sure the order ID, and other large integers are treated as strings.

If you look at a Raw JSON packet you can easily spot whats a string, integer, object or array.

Take this example Facebook Credits JSON Packet, (wrapped for readability):

{"order_id":239724439419867,"buyer":197803678,"app":148748711865470,
"receiver":197803678,"amount":5,"update_time":1320075413,
"time_placed":1320075408,"data":"","items":[{"item_id":"<item_id>","title":"Post",
"description":"A New Purchasable Post",
"image_url":"<some url>","product_url":"<some other url>",
"price":5,"data":"<data>"}],"status":"<status>"}
  • order_id is an integer as it has no ” around it
  • status is a string as its surrounded by “
  • Items is an array as its surrounded by [, in this case containing a single object, entries/items are comma separated.

So after thinking about this I decided the best way to sort this out was to convert the integers in the raw JSON packet to strings before decoding.

In pseudo code.

Lop off the { and } from the start and end
Explode around ,
array walk each item
split on :
check if there are no " in the second bit
if none wrap in "
glue back together

Something alone the lines of:

function largeint($rawjson) {
  $rawjson = substr($rawjson, 1, -1);
  $rawjson = explode(',' , $rawjson);
  array_walk($rawjson, 'strfun');
  $rawjson = implode(',', $rawjson);
  $rawjson = '{', . $rawjson . '}';
  $json = json_decode($rawjson);
  return $json;
}

function strfun(&$entry, $key) {
  $data = explode(':', $entry);
  if (FALSE === strpos($data[1], '"')) {
    $data[1] = '"' . $data[1] . '"';
    $entry = implode(':', $data);
  }
}

I’m not sure in terms of memory footprint if its cheaper to do a substr of $data[1] and check if its a ” or not.
I suppose it could test the string length of the detected integer to see if its invalid/unable to process, but then you would be performing a string function on an integer and again the problem would arise.

But as a block of code is does the job, its obviously not ideal for all situations, since in this case I want the integer as a string, I’m not using it for math, but if I did, we probably need to do some bizarre unpack-ing or something.

Any opinions or improvements give us a comment below 😀