2017-04-13: On speedruns, cheating, and emulators
A thing has been on my mind for a bit. Since I have this space now I figure I'll brain dump on it.
Last week, the Ninja Gaiden II leaderboard added some new rules geared toward making it harder for emulator runners to cheat. This came on the tail of a four hour long debate on what's reasonable to expect of runners, whether emulator makes it too easy to cheat, and if it's fair to expect a higher standard from emulator runners as far as proving they are legit goes. This all started when a run was submitted that the community deemed too good to be legit. The runner, when asked to prove his ability in a live stream, deleted his run and disappeared from the community.
The NG2 community came up with similar answers as their rules now state:
For sub 10:20 runs, frame counter and input display are required. If this is not followed, contact the mods/community and tell us the reason.
Why is this a good solution? Because on most emulators, the frame counter will simultaneously make both TAS playback and splicing a thousand times more difficult, and the input monitor will further address other concerns of turbo use, TASing, and such. The frame counter making splicing harder boils down to this: you now have a number in the game feed that indicates how many frames of video there should be at any given time. If that number suddenly jumps forward or backward, you have a splice in the video.
A side bonus of the frame counter is, in most emulators with TAS playback capabilities, the counter will change in a very obvious manner when a TAS file is being played. FCEUX and Bizhawk will both change to display the number of frames in the movie file as well as the current frame, making it a dead giveaway that a TAS is being played back. snes9x-rr will display an indicator that a movie is playing, etc.
With TAS playback as ruled out as we can get it, the input display just further reduces the possibility that tasks are automated somehow. With a frame counter ticking, you can slow down the video and see if a runner is doing something impossible like many chained one-frame taps (an accusation that came up in the NG2 situation above). I personally would opt to use an external input viewer for two reasons though: 1.) Getting as much crap out of the game feed as possible is a goal of mine and 2.) It further rules out Lua scripting, TASing, etc since now an external program is corroborating your inputs with the game window.
These two rules together are a very good, honed strike against TAS playbacks. In addition I'd suggest requiring streaming, but that's not practical as there are still runners out there who cannot stream to services we can use to verify runs. Runners in China are a good example of such. However, streaming does even further reduce the window of doubt when it comes to the validity of runs.
However, I'm a little worried about leaderboard mods getting complacent that, because the frame counter says no movie is loaded, there can be no TASing going on. FCEUX, Bizhawk, snes9x, et al are all open source and it'd be trivial for someone with the knowledge to alter the emulator. This raises the barrier, but does not eliminate it. Judgements will have to be made still on if certain things seen in a run are impossible for a human, and the appropriate skill checks made. Still it's a good start.
I'm a huge proponent of reducing the amount of stuff displayed over the game. Until now I said I'd fight to the death any requirement to put something in the game window to make your run acceptable but you know what? Requiring the frame counter makes a lot of sense as far as combating TASing and splicing goes. In FCEUX and snes9x-rr it's a giant eyesore, but in Bizhawk it's a very discreet, small number in the corner of the screen. I'm all for it.
Just as long as people stop requiring capturing the window border of the emulator. I've proved that that does absolutely nothing to combat cheating.
tags: speedrunning, tech