I used Stb-tester to generate a textual representation of the Apple TV’s screen, fed that to GPT-3, and fed GPT-3’s response back to Stb-tester. It’s super flaky —I don’t need to worry about my job security just yet— but it’s pretty neat to watch when it does work. Here’s a quick demo:
