A few days ago, I wrote about Olympics Streaming and the challenge of quickly navigating around streaming/downloaded sports programs with typical streaming or cloud DVR services.

I tend to make heavy use of "jump" buttons which skip forward or back amounts like 10, 15, 30 or 120 seconds. With a local disk DVR (and some very good streaming ones) this is done with super fast response time and a live preview, so it's easy to move around a program to what you want.

But what I did not discuss, and should, is just what do you want? The most common thing is to skip commercials, and some DVRS (typically those not under control of networks or cable providers) offer automatic commercial skip, or a button that seeks to the end (or start) of the commercial break. They even do this on near-live recordings. The problem is that TV shows don't want you to skip commercials, and work to get this feature out of DVRs. A solution that keeps everybody happier is to pay extra for commercial-free, though some things like sports can't do this as the live event has gaps for commercials.

With sports, though, there are tons and tons of short boring gaps, and that's really obvious in the Olympics. Indeed, there are too many, which can mean a complex UI. The best simple UI is usually a button or two that skips 10 to 30 seconds, with a "back" button that skips a shorter amount of time, so you can skip forward, and if you overshoot, go back 1-2 times. But why not be exact.

In hockey, for example, I found found a 10 second button works well from blowing the whistle to a face-off. In football, there's a similar gap. In Olympic judged sports, there's a fairly standard gap from watching a performance to seeing the judges scores, usually filled with slow-motion replays and video of waiting athletes.

One answer would be for the producers of a video to encode the various points of interest, so you can seek exactly to them. This could be done by a human director who is placing them as the recording is made. You probably want 2 levels, for short-term events (like whistle-blows) and bigger things like chapters. The location of these varies greatly depending on the program. The marks could be tagged, which might allow viewers to custom program what marks they are most interested in, probably on a one-time per-event basis.

Indeed at one could also get marks to make a "highlights" version of a game. Producers already do this, but they do it by editing together the highlights. If the highlights are simply marked in the stream, viewers could pick their own, or just say, "I want a 10 minute highlight version" or "I want a 20 minute version." The most common highlight edits focus only on scoring, but you could have editors deliberately also tag interesting non-scoring events. The viewer would ask to see some of those at random to increase surprise -- if you know you're always going to see nothing but scoring, it's effectively a big spoiler.

While editors could do these marks, and they already do for highlight versions, it turns out computers could do most of the work for the, and in fact it should be possible with AI tools to generate these mark points entirely automatically. In fact, the viewer could do it themselves, though it's more efficient if one party does it and publishes the list of timestamps.

Clock and Score

Things like the score, clock and game state are already encoded in the feed. They have them directly in the studio but a tool to extract them from the video isn't hard. You know when the clock stops and restarts. You certainly know when scoring happens. You can also usually detect penalties and many other game events, and make marks for them (in some cases, like scoring, the view wants to seek to a minute or two before the scoring, and also to sometimes get some almost-scores to remove the spoiler effect.)

Another easy tool is the audio. It's simple now to turn it into text, and the announcers are then describing everything that's going on. You could also read excitement in their voices with an AI tool. But it should be hard to go from the transcript to a "good parts" list of bookmarks. All these tools would be best to just produce an initial list of bookmarks for somebody in the control room to oversee and tweak. The system can even put labels on the bookmarks so that a viewer can very easily browse through a list of them to pick the ones they want. It's really not a big problem to have occasional mistakes, bookmarks for things that aren't that interesting. The viewer can just skip again. The labels would be written to avoid spoilers.

Done well you get a very simple UI, though, "just watch this abridged version, and press a button if you get bored."

Another easy UI is to put up a grid on the screen of thumbnails and tags, and let the viewer quickly move through it to see what they want. Some streaming services replaced fast forward with an "array of thumbnails" interface which works well when you don't have a live local recording with its ability to do instant seek.

A sophisticated tool can also do more than seeking, it can switch into different fast forward modes. Because these modes are planned, they could be pre-computed super-smooth fast forward at whatever speed makes sense. As such you might watch something like the 5 hour 250km bike road race with a mix of real-time snippets with smooth fast-forward middle sections when not much is happening. There could even be an AI generated summary of what the announcers said while playing this. This sort of editing technique is now common in videos, when we know there is a less exciting section, and it suddenly jumps it speed, but very smooth speed. Another advantage of smooth FF is you can be less worried you missed something, and if the bookmarking AI made a mistake, you can just go back and watch it.