Sun Feb 19 02:45:05 PST 2017

My (short) experiment with GNUSocial and Mastodon

For a whole lot of reasons I've been getting real sick of Twitter lately. I'd heard about Mastodon awhile back and decided to give it a try.

As a quick summary: Mastodon is a re-imagining of GNUSocial. Both are open source, federated microblogging services. They're basically peer to peer Twitter clone where the theory is not having one company owning the entire means of communication will make for a better experience. Anyone can run their own node and connect it to the network, and everyone's messages are federated into one global feed. Mastodon builds on top of OStatus, the protocol used by GNUSocial, and tries to wrap it in a sleeker, more user-friendly interface with a couple of idealistic changes to the workflow on top.

My initial bootup experience with Mastodon's main instance was pleasant enough. It's got a slick interface like Tweetdeck, works as you'd expect, and jumps right into the GNUsocial distributed OStatus network/Fediverse/whatever you want to call it. The public timeline moves slow enough that it's a good source of amusing people-watching, the extended message limit is nice in the era of not needing to post from SMS anymore, I could go on on its positive points and I recommend everyone at least take a look. I was pretty hooked after about a week and wanted to spin up my own instance.

All that sleek, shiny veneer comes at a price though. I looked into setting up my own Mastodon instance. There's pretty decent instructions on their git repo... if you're using Docker or a Debian variant. To be 100% frank, the resource footprint implied by the installation docs put it out of my reach with my tiny cluster of underpowered VPSes. I'm not running a Docker container there, and I was kind of left in the cold getting it running on my CentOS box and wanted to try a lighter option before diving into the giant system rework that would be getting an entire Rails stack, Nodejs, Postgres, Redis... blah blah... set up.

So I turned to GNUSocial. GNUSocial would give me a lightweight option but still let me federate with Mastodon instances, as well as other GNUSocial instances of course. Where Mastodon may be slightly over-engineered and fat, GNUSocial was the opposite: it was everything I've sadly come to expect from non-flagship GNU projects. Under-documented, under-developed, and apparently in the middle of some major rework that made a lot of the presented features either not work the way they were presented, or not be present at all. The current point release is 1.2 from about 12-15mo ago (nightlies are more recent) and its docs still call it "StatusNet" which is a name that hasn't been used since pre-1.0. Other sections of the documentation are similarly outdated or just wrong.

Getting GNUSocial actually running wasn't bad despite the documentation dearth. Once it was up everything seemed just a little "not quite right" though. Privacy options I'd come to expect from Mastodon, and were promoted in the GNUSocial docs, weren't there. My syslog was unhappily churning out PHP warnings about bad practice in the code. Deleting messages left strange debug message looking artifacts that filled my terminal with DB errors if I tried to clear them... Nothing full out failed, but it wasn't a reassuring user experience. I upgraded to nightlies, hoping maybe in the intervening 15 months since point release, some of these quirks got fixed but alas, that just promoted me from "not quite right" to "not quite working".

Digging through the system's admin pages, I started discovering links to documentation that still pointed to the old StatusNet site, pre-rename to GNUSocial; a domain that has since lapsed and been picked up by some kind of gaming blogger? I don't know. Even links that point to the correct domain point to old material and return 404s. Phew.

I decided to expand beyond the first party support resources and look for wikis, blogs, even GNUSocial posts themselves. As search after search returned either deprecated information, or no information, I began getting more creative. Some of those creative searches began to expose some "fun" discussions on Mastodon, GNUSocial, and OStatus as a whole. See, apparently I wasn't aware of it, happily nestled up in my Mastodon bubble, but there's some drama going on between Mastodon and GNUSocial node admins. I don't know the full of it but it comes down to idealogical differences and the perceived intelligence of GNUSocial node admins VS those who chose to utilize the flagship Mastodon instance and its shiny, easy to access interface.

Some of the more fun content included recommendations for GNUSocial admins to block messages from Mastodon instances in the federated network, which is a great idea for a darkhorse solution that requires a strong federation to even hold a candle to the monolithic Twitter, I'm sure! Sadly this, also, is what I've come to expect from GNU offerings: elitist userbases. Sigh.

I'm kind of soured on the whole OStatus thing at the moment. Maybe it's just because I spent several hours fighting with finnicky federation and "not quite right" PHP GNU abandonware. GNUSocial's a nonstarter, at least. At this point I haven't decided if I'm going to try to cram a Mastodon instance on one of my tiny VPSes, stick with mastodon.social, or just bury this whole idea and go crawling back to Twitter for my shitposting.

Is there a third option?

Posted by trysdyn | Permanent link

Sun Dec 18 14:29:16 PST 2016

2016 in review, 2017 plans

Ignoring the obvious horrible stuff that occurred up to and including the election, I at least made a lot of personal gains and progress in 2016. Speaking only on the not-super-personal ones...
  • I've moved to mostly working from home, giving me ~2 hours more a day of NOT COMMUTING to do other stuff
  • I had a blast with SNES Superstars in March. My run went well and I even got an unscheduled second one of Live A Live, Oboro IL
  • SGL didn't really come to be this year but that gave me time for a lot of small speedrun projects, probably doubling my list of speedgames
  • I did my first blind race ever; that was fun
  • I completed about two dozen games that sat on my "Games to complete" list, a lot of them on stream; including Lufia TLR, SaGa 1, Final Fantasy V, and Might and Magic
  • Speaking of Final Fantasy V, I participated in the Four Job Fiesta for the first time and completed several runs
  • I wrote and/or improved several important tools I use in my stream and have them mostly ready to distribute including my Livestreamer frontend, my alternate speedrun timer, my retro roulette script, and an alternate host notifier script
Admitteedly I was wrong in my start-of-2016 post about 2016 being a great year; I've had my fun with depression on and off, but I'm mostly on the mend now and hoping 2017 is better. Plans for 2017 stream and game wise?
  • I'm submitting three games for this year's SNES Superstars. I'm learning one of them now and will take about 3-4 weeks to derust all three leading up to the marathon. Estimated total runtime: 1 hour
  • For 2017 I want to take on a "big" speedrun project. I've decided to tackle both Zelda II, and Link to the Past. I've wanted to run both for a long time but didn't feel in a place to commit the time; this year I do
  • I expect to expand my list of completed games on-stream. For sure I want to take on more classic RPGs. SaGa 2 and 3, FF6, the BoF series, Dream Maze, and many more are on my hitlist. I may also re-beat Emerald Dragon and Dark Law on stream for people to see
  • Maybe more Game Boy runs because Game Boy is under-represented in the speed world: Wario Blast, Pocket Bomberman, Super Mario Land 2, Banishing Racer?
  • Will SGL happen this year? Don't know. Would do if it did
  • Hopefully getting into races with some of the more commonly run games I'll pick up
The rough plan for how I'm going to approach stream stuff is this (yay another list):
  • New Years-ish: A long retro roulette stream like I did last year
  • First week of Jan: A super casual playthrough of LttP and/or Z2 for awful splits and re-acquainting
  • Jan / Feb: Slow learning of LttP and/or Z2 with other stuff mixed in
  • Mid-Feb / Early-Mar: Derust and prep for SNES Superstars
  • Post-Superstars: Some kind of chill RPG playthrough or something before return to LttP/Z2
Here's hoping 2017 is better than 2016 :P

Posted by trysdyn | Permanent link

Sun Jan 10 16:12:39 PST 2016

A Busy Q1

2016 has started off pretty solid with AGDQ and a resurgence in speedrunning interest. I myself have a few speedrun-related things lined up. I'm going to be busy for the first quarter of 2016 and beyond. Just to sort of get my thoughts together, here's what's on my plate:
  1. Someone cut my Dicing Knight WR, so I'm going to be working on getting that back
  2. I want to up my streaming presence and take a shot at getting onto RGLTV
  3. I've submitted Cool Spot and Live a Live to be shown on SNES Super Stars in March but I don't know if either/both are in yet
  4. I WILL be doing a lot of volunteer work in the above. My plan is to go ham on it and be doing something to help run the marathon 12+hr a day during the event
  5. I plan to participate at least a little in Speed Gaming League
  6. If I have time and the ducks line up with all this other stuff, I want to take a crack at entering the Big 20 #4
  7. If I have any sanity left, SPEO 2.0 still needs to be coded. The team has given me the material I need; I just need to write the code
Busy, busy 2016... It'll be a good one; I'm full of Optimism and Determination.

Posted by trysdyn | Permanent link

Mon Dec 7 20:15:54 PST 2015

A "Fun" Prediction

I'm posting this mostly to "Call it". In wake of reports that the Windows 10 upgrade consultant widget has been getting more and more invasive in urging users of Windows 7 and 8 to migrate, I'm predicting a specific chain of events. This is mostly just for fun and for "I told you so" credits later on.

So here's how it's gonna go. I'm predicting by July 2016; or at the least by the end of 2016, Microsoft will...
  1. Push Windows 10 upgrades to users in a way that largely results in "I rebooted and 10 just installed" reports
  2. Ignore previous user action declining upgrades to 10 in doing so
  3. Cite Home/Pro/Enterprise division in the fallout, noting that anyone who needed a specific version should have been using Enterprise or a similar solution
  4. Address valid complaints from people who did not want to upgrade (for compatability, performance, or privacy reasons) with EULA mumbo-jumbo disguised as a marketing whitewash about how superior 10 is
I won't get into the drama of talking about how Windows 10 could have been a great thing for users and Microsoft alike, and how Microsoft screwed the pooch; I've ranted about that to friends and colleagues enough as it is. Mainly my concern now is, with the way Microsoft is aggressively pushing 10, the fallout from when they finally decide to force the matter and force the upgrade as they've shown they are capable of doing.

I guess we'll see.

Posted by trysdyn | Permanent link

Sat Nov 14 18:41:23 PST 2015

Re-visiting Speedrun Timers

Anyone who's spent any time watching or trying speedrunning knows how a timer works. The typical speedrun timer displays a few bits of information including a total runtime, and the time it takes to get to certain checkpoints within the game. These checkpoints are called splits and are used by the runner and viewers to determine how the run is going by comparing the run to the runner's personal best at various points of the game.

If you've never seen a speedrun timer, it looks like this:



While this format is solid and presents a lot of useful information, for some runs it just doesn't fit. For example, extremely short runs that don't need detailed splits, or RNG-heavy runs, or runs that are done in no particular order can't really use splits well, and a lot of this space is wasted on data that is not relevant to the run. Furthermore, some runners don't enjoy using splits, citing that detailed knowledge of their performance compared to their personal best can psyche them out during the run. The typical response for these concerns is to strip the timer down to nothing but the total runtime. This works, but then leaves viewers with no indication of how well the run is progressing. To attempt a compromise, I designed my own experimental timer.

This is Fuse:



It's probably no surprise where the name comes from. Fuse displays the run as a life bar, or a fuse, that burns down as time progresses. When the bar runs empty, the current run has gone on longer than the runner's personal best. Since viewers tend to focus heavily on PBs and WRs, I decided to make the main element of the timer be a visual comparison against the runner's PB. Within the bar are white vertical marks, dividing the bar into chunks. Each chunk represents one split. The text below the bar displays the split name, and the current split differential. In this example, the runner is about 1 second behind his personal best as of the first split.

My goal with this is threefold: limit the size of the timer on-screen, give the viewer a rough indication of how the run is progressing, and limit the amount of information overload the runner can experience when looking at the timer during a run. Toward this end, Fuse does not display any data about past splits. Once you split, the previous split's marks are removed, like so:



This means a runner, who is only taking quick glances at the timer, only really has a general idea of if they're ahead or behind, but not by how much. Meanwhile stream viewers have a much better idea of where the runner sits because they're able to witness the timer at the moment of splitting and see how far ahead/behind the runner is. This should hopefully allow a runner a very streamlined timer in their overlay, without necessary space or mental clutter, but still provide viewers with something to watch to get an understanding of the quality of the run.

Fuse is extremely simple and probably won't be of interest to most runners. It stores no gold splits, has no sum of best calculations, no run prediction, won't display green/red splits, and has very limited reskinning functionality (mostly font and color changes). It exists for people (like me) who want a cute and simple visual to display run quality, and may go well along side a full fledged timer like LiveSplit.

If there's interest I'll package it up and release it. I suspect however most runners would desire some actual power in their tools.

Posted by trysdyn | Permanent link

Tue Sep 22 08:37:24 PDT 2015

On Speedrunning and Live A Live

Awhile back, Digi and I did some rough routing for the Squaresoft Japanese release-only SNES RPG Live A Live (this is an awesome game by the way and there's a translation patch available, I recommend giving it a spin). We routed out six of the initial seven chapters, theorized heavily on the seventh and the denouement, and I did some individual level test runs. I estimated the final run to be between 6 and 7 hours, with a rough run taking up to 8.

That's a long time. I usually am not capable of sitting still for that long. Hell, 90 minute runs of Getsu Fuuma Den start wearing on a bit for me sometimes. However, we did a lot of work on theorycrafting and this is kind of our baby and I want to see it through. So I'm proposing a thing.

I want to finish routing all the levels and do ILs of them. Once that's done I'd like to culminate it all in a run or two to get a time down and have a final product "real" RPG runners can use as a starting point. There's no way I'll be able to do a real run without most of the people I talk with on a daily basis being on board though. Not just understanding that that's basically an entire day I'll be unable to respond to them save for quick messages during cutscenes, but also because I'll need company to not go crazy across an eight hour run.

So the rough proposal is this: I return to streaming, doing IL runs and final routing of the game. These will be SUPER casual streams, not the rapid fire attempts streams I usually do. Once I feel like the ILs are kind of solid (or at least not awful), I'll pick a weekend to say "I'm doing the attempts here". I'll pretty much be incommunicado that weekend while I try to get this down. I may keep Discord up and publicize the link so people can pop in and say hi to me (you'll be streamed, though), and of course would like people to pop into the stream chat and hang around too. I don't know if I won't just give up a few hours in because I'm prone to doing that, but I want to give it a shot anyway.

The questions to you, dear reader, fall down to this:
  1. Can you get by with me giving you one-word responses for an entire weekend without thinking I've started hating you?
  2. Would you be willing to hang around and banter so I don't go crazy? Not the whole time, but pop in sometime and hang out
  3. Is this something you'd even like to see or care about? Or would this just be a waste of time? If not the whole run, maybe the ILs?
  4. Are you okay with me dropping ALL of my other projects to do this? If I'm doing something for you, it probably won't get finished this year
  5. Will you deliver me take-out while I run this so I don't starve to death? :)
I nuked the comment section on this blog because when I moved to https-only Discus kind of ate my comment threads, so you'll need to let me know via twitter or IRC if you want in on helping out with this. The actual run won't be any time soon since I have to de-rust the game and finish routing it, but it'll be this year if I do it.

I've said "That's too long, it's not for me" for a year now but honestly, it's been in my head and we've done most all the groundwork so I want to be the first to actually strike a full run. It's a pride thing. I guess we'll see what develops.

Posted by trysdyn | Permanent link

Sat May 24 02:55:18 PDT 2014

The Strange Case of Higan and XBMC

Posting this for posterity.

XBMC is awesome. I'm building a little media box around it today. Setup has been smooth except for one little detail: XBMC's equally awesome Advanced Launcher add-on can't handle Higan and its rather unique way of storing SNES ROMs.

Here's the skinny: Higan is a great SNES emulator, but its author is kind of a maverick. He thought he could single-handedly change the emulation scene by forcing his emulation to convert ROMs into his special format. His format is great, don't get me wrong: it solves a lot of problems inherent in ROM dumping and such, but no other piece of softare knows how to handle it.

Higan's format is to store the ROM data, battery-backed RAM, cheats, and saves all in a directory with an 'sfc' extension. Most file browsers will see this directory and refuse to let you say "This is a ROM'. Anyone who's tried to make Higan work on XBMC has run into this, but I've found the solution.

We're going to play a little switcharoo on XBMC. Ready for this? First of all set up an Advanced Launcher ... launcher... like you would for any other emulator; specify your file extension to be 'sfc', set up your ROMs directory as normal, but don't try to Add Items yet.

Now, drop to a terminal, or open explorer, or whatever. Browse to the directory above your ROMs directory. Let's say my ROMs directory is called 'SNES'. Move 'SNES' to something temporarily like 'SNES-real'. Create a new SNES directory and for every .sfc dir in SNES-real, create a file with the same name. In linux-land you can do this:

cd /ROMs
mv SNES SNES-real
mkdir SNES
cd SNES-real
for f in *.sfc; do touch "../SNES/$f"; done


See where we're going yet? Now go into XBMC and run your Add Items. Let it finish, then switch the dirs. Again in Linux land:

cd /ROMs
mv SNES SNES-fake
mv SNES-real SNES


As long as you don't re-run your Add Items or do anything to make XBMC examine its ROM database, you'll be fine.

It's stupid. It's dodgy. It works. I'm hoping people out there with this problem find this, because so far Google says "Don't use Higan", and that's a shame-- because Higan is awesome.

Posted by trysdyn | Permanent link

Sat Apr 19 13:06:27 PDT 2014

Morals, Moral Debate, and Pull Requests

Moral and ethical debate is something that's been at the forefront of my free cycles lately. It's sparked this strange idea that won't dislodge itself from the back of my mind. While sitting and pondering the meta-cognition of morality (meta-meta-cognition, if you will), I came to the following conclusions:
  1. Morals are simply rules that define how one wishes to act in moral situations
  2. These rules could, ideally, be conveyed in if-then-else relationships (but this is very muddy in complex situations)
  3. These rules should be constantly updated as new situations and new circumstances are introduced to the person's world view
  4. Constant revision of if-then-else rules of order is the purview of a SCM
You can probably see where I'm going with this: why not GitHub this stuff?

I'm not the first person to have this idea, or one like it anyway. Large chunks of German law are available on GitHub and pull requests are accepted for the purposes of discussion (though this seems to be dead now). I'm basically proposing that a moral consequentialist or utilitarianist do the same thing to open their moral platforms up for public submission of alterations. This would require a very neutral and, as stated, utilitarian view on morality, ethics, and the consequences of one's actions as you would be subject to arguments based purely in reason as to why certain moral standings may be incorrect.

I imagine a simple example of what I'm proposing would start with a very bare list of moral tenets, and then involve pull requests or other debates that bring about updates and complications. The workflow would look like this (note: this is just an example and in no way reflects my actual beliefs):

1.) Violence is never acceptable
Which leads to a pull request that looks like...

1a2
>    a.) Unless used in self-defense
Which brings about discussion somewhere (probably not in the pull request notes, that'd be noisy), and eventually brings...

2c2
<    a.) Unless used in self-defense
---
>    a.) Unless used in self-defense, and only in appropriate amounts
Which brings about discussion on what "Appropriate amounts" is. Eventually (and maybe sooner rather than later), an impasse would be reached where the people involved can't come to an agreement on how to handle a certain situation. That would ideally result in a fork where the two individuals continue updating their own repos and accepting pull requests as appropriate. The important thing to keep in mind is that these are personal codes of morality, not some kind of guidebook on how to act in general. Some people may hold "Don't be a dick" to be the whole of the law, but even that would result in discussion on what exactly being a dick entails.

Why all of this? Because I think it'd be more productive than moral argumentations with heated rhetoric and all kinds of shouting and demonizing. Don't get me wrong: I've seen plenty of that in the Linux kernel source repo too! But at least those guys can be cordial to each other outside the pull request logs?

More importantly: I think it's important to constantly update and re-evaluate one's own moral stances, and it can be awkward to solicit discussion on these things without a medium dedicated to doing so.

Down sides? Doing it via a SCM would inherently bring about a series of biased viewpoints. I'm not saying everyone who knows their way around a SCM has a certain moral viewpoint, but it's rather incontrovertible that the engineering mentality brings with it a certain approach when thinking about problems, and moral synthesis is inherently a problem to be solved. Maybe this is a good thing because those biases would also imply an ability to accept criticism of one's ideals. Maybe this is a good thing for someone like me who does, also, consider moral synthesis to be a logical problem, and not an emotional or religious one.

I don't know. I think of weird things in my spare time. Maybe this would all be mental masturbation anyway, since moral changes are not as easy as simply making or accepting a pull request; but I think it'd be a good tool for discussion and idea trade. I might give it a shot.

Posted by trysdyn | Permanent link

Tue Mar 18 02:41:02 PDT 2014

An Overview of Analogue: A Hate Story and Hate Plus

So I stayed up until 9am reading both Analogue: A Hate Story and Hate Plus. I do not regret this decision. Both games, as we'll call them, are amazingly well constructed and point out some rather jilting points about society and where we're going as a race. Some call the narrative heavy-handed and I agree, but most of even the heavy-handed stuff is necessary to illustrate just how much changes as time passes.

The purpose of this little write-up is to gloss over the story, chronologically within the game's universe, then make some comments about how it fits into today's society at the end.

This entire thing is a spoiler warning. I say this because, even from paragraph 1 of the synopsis, there will be spoilers because I will essentially be explaining the story backwards from the way the game presents it. You have been warned.

Another warning: both games are quite unashamed to delve into sexual material, sometimes of a LGBT nature, and sometimes skirting on (or sitting directly in) sexual assault. Most of these diversions into the lustful are framing devices; they're not completely relevant to the plot, but can demonstrate how society changes in-universe based on the way people approach sex.

I wish to re-emphasize: no part of this post after this line is spoiler-safe. None of it. Don't try to pick and choose what to read if you haven't played the games.


I think that's enough framing from the outside. Let's quickly touch on how the story is framed before getting into it. This sums up the first 15 minutes of play in Analogue: A Hate Story:

It's close to the year 5,000 (presumably AD). You've been hired as a contracted salvage pilot to locate the Mugunghwa, a generational starship launched by the Korean space program in 2,500. (Sidenote: A distinction is never made about WHICH Korea launched the Mugunghwa, but based on my knowledge of the story's events, I believe by 2,500 North and South Korea re-unified and were heavily influenced by modern day China. You'll see why.).

Upon locating the ship and tapping into its barely-powered computer, you are greeted by an AI named *Hyun-ae. *Hyun-ae is unusual: she's overly emotional for a shipboard AI, is elated to see you, and instantly grants you access to the ship's systems despite knowing you're a foreign agent. From there, your task is to explore logfiles and discover what happened. From time to time *Hyun-ae will interject with her own thoughts and ask you questions; answering will decide what logs she gives you access to and, answering in a way that infuriates her will cause her to lock you out of the ship's systems, ending the game. Losing, however, requires intentional effort in pissing off *Hyun-ae, so it's not really a concern.

From here, both games are from the perspective of this console to the ship AI, and your task is to figure out what happened to the Mugunghwa and how it came to be a derelict.

One note I should make here: The game denotes AIs in C pointer style. This gets important later. *John is an AI. John is not. Simple? Good.

Okay now the spoiler warnings begin. I'll be delving straight to the middle of the second game (Hate Plus) because that's where the logs explaining how this all started are found.


Let's jump back 800 years, to about 4,150. We learn in the beginning of Hate Plus that the Mugunghwa has a democratic meritocratic government with a sitting President and a Council of ten who are chosen by selecting the best-of-field from various fields of study on the ship (Science, Engineering, Education, Security, etc). Laws are passed much the same way they are in the United States: the Council passes a vote and passes the bill to the President to be signs into law. This becomes relevant later.

Also worth noting: The Mugunghwa is already derelict to a degree. Some of the log entries make vague reference to an event in the late 2,500s that caused the destruction of several of the ship's core systems. It can be inferred from this that the Mugunghwa is floating in space either on some kind of autopilot, or aimlessly entirely; also that it had not had communication with Earth in many, many generations. It's a closed social ecosystem.

On a more social scale, the Mugunghwa is sexually progressive and, in some ways, socially liberal. Crew members on the Mugunghwa of 4,150 openly engage gay, lesbian, bisexual, and poly relationships with seemingly no fear of persecution. Several of the log arcs go off on details sexual exploits between men, women, multiple partners, transpeople... This all sets a framework for how those in 4,150 approach sexuality and gender identity.

Contrasting the social liberalism: there's a strictly established caste system not unlike Plato's Republic wherein you have Peasants, Slaves, and Nobles. Slaves are mostly house workers but aren't actually indentured as far as I can tell, Peasants are middle class workers who often have a Slave housekeep or two, and Nobles are leaders and those with political power, and usually hand their positions of power down to their children when they retire, keeping the Noble caste familial. This has lead to a social structure where there's privileged families that hold all the power, and everyone else is a Peasant or Slave.

Problems on the Mugunghwa at this period: the birth rate is quickly falling and is under the death rate, and this is a problem that the Council wishes to tackle. Furthermore, elections are coming up and one of the Presidential candidates, Park, represents the Peasants and their desire to move to a more direct democracy, doing away with the nepotistic meritocracy in place now.

Enter *Mute: the ship's security AI and the Councilor of Security. Since, to a degree, becoming a Councilor is a meritocratic thing, *Mute is considered the perfect Councilor of Security with her access to everything on the ship in the blink of an eye. She has sat as Councilor since 2,500, and is considered more or less unassailable and beyond reproach.

*Mute, foreseeing problems with riots in the coming election if Park does not win, presents a plan to the Council to manipulate the election to give Park the victory, while at the same time making changes in the Council to create a stalemate between the Presidency and Council, so Park's Presidency will be a lame duck one. Once Park's term is over, the Council will be restructured the way it was and the Peasants will be satisfied that they got their five years in power, and nothing was accomplished.

The plan is enacted and the Council restructured around a man named Ryu. Ryu, up until this point, was a rather low-grade and unimpressive politician: the perfect puppet to be installed as the Head Councilor as he could easily be controlled by the other Council members. This is where the trouble starts.

The logs meander through several dry, boring records of Council meetings that discuss, on their own, seemingly harmless changes to ship law. Most of the Councilors are pursuing their own political agendas and vote the way that will bring them most benefit but *Mute, being immortal and a permanent fixture in the Council, votes what's best for ship security and usually ends up coming at odds with the rest of the Council.

With Ryu as Head Councilor Chinese literacy becomes a requisite to graduate college (which is only available to Nobles). The framework for this is to provide a broader understanding of classical Earth culture. Soon after, a law is passed that grants a living yearly wage to any woman that has a child, to try to combat the birth rate decline. Finally, after that, a law is passed that limits the transference of Noble status and Noble pay to firstborn males within Noble families, permanently codifying the nepotism that's been in play for centuries anyway into law.

President Park signs all of this. It should be noted that at no time is Park ever directly mentioned or addressed in the logs; he's just sort of a figurehead that seems to sign anything that comes his way. It's not explored why. Maybe he's dumb, maybe it's political pressure, or maybe... well read on and you'll see.

In the year leading up to elections, three major changes: first, the Councilor of Education brought forth a change to the meritocratic exams that allows a sitting Councilor to be replaced if someone else scores higher on the exams then them. The Councilor insists this is mostly political showboating and the Board of Education will (and already does) manipulate the results so sitting Councilors will keep their seats.

Second, a chunk of *Mute's security force is decommissioned and pulled into a special Election Fraud unit that reports not to *Mute, but to Ryu. This is, at the time, seen as a political power play to punish *Mute for being oppositional in previous debates (*Mute pretty much argued against every change so far, on the grounds that these changes would irritate the lower class).

Third: Ryu moves to change election law. Specifically he wants to dissolve almost all requirements to become President, removing term limits, conflict of interest limits, and completely deregulate who can run. This is framed on getting anyone but Park on the seat next election, basically, and is passed despite *Mute's heavy objections.

Council Exam day comes: somehow everyone on the Council but *Mute and Ryu fail their exams and are replaced. Then in the following election, Ryu's Election Police find "Fraud" in the election and have Park arrested. After the votes are re-counted, it's found that Ryu won the Presidency, despite not really running for election. Now he holds both the Head Councilor position and the Presidency, and this is where things go straight to hell.

Ryu now has the power to overturn veto, veto anything, and generally ignore the rest of the Council. Meanwhile, the changes already pushed are causing major harm to the economy and welfare of the ship. Women are having their pay reduced or being fired out of fear that they'll simply decide to quit and have children for the stipend, and the birth rate is still falling despite the renewed focus on wives "Doing their duty". Women quickly end up in a position where they cannot hold any job that requires skill or training, because no business wants to compete with the motherhood stipend or lose an important employee.

As a background arc, a scientist researching the birth rate issue discovers that the ship's engines are putting off gamma radiation that may be harming the reproductive systems of the crew. Unfortunately her research is cut short when she's fired because her boss is moving to a staff of men who won't quit to become mothers. Despite this setback, she finds alternate ways to continue her work, including finding records of a stasis chamber deep in the ship that houses the body of a 12 year old girl with a terminal illness.

This girl, who should be known as the "Sick Daughter" at this time, was frozen until such time that science could find a cure for her disease. Science, however, seems to have declined since ~2,500 and she remains in stasis. In any case, the scientist locates her pod and takes a blood sample. She uses it to prove that, indeed, gamma radiation is causing reproductive harm to the people on the Mugunghwa, and calls an audience with the Board of Science to present her findings.

Sadly that never comes to pass. In the days leading up to this meeting, *Mute discovers that Ryu not only rigged his election with his hand-picked Election Cops, but also rigged the Council exams to replace everyone on the Council with his puppets. *Mute responds in her capacity as head of ship security by arranging to arrest Ryu. However, Ryu becomes aware of the plan and responds by planting bombs in both public areas of the ship, and on the computer core. *Mute is forced to surrender during the fight, and Ryu forces her to relent her security access to every system on the ship. *Mute's programming forces her to comply, lest the alternative be destruction of the ship.

With Ryu now in full control of the ship's systems, he changes his title to Emperor, and ushers in a new era of Neo-Confucianism. He effectively dissolves the Council and cements his position of power by revoking the systems access of anyone who could possibly depose him.

Sidenote: If you don't know about Neo-Confucianism, it's a real thing. Feel free to go read up on it. The relevant points here are a focus on the subservience of women, familial passage of position and rights, and the honor involved in a man being the head of household.

Ryu plans to destroy the ship computer and effectively kill *Mute. Instead, she convinces Ryu to reset the computer, wiping all data and resetting everything to a blank slate. Ryu complies with this and, in the moments before *Mute's memory is erased, she compiles a chunk of data into her own code explaining what has transpired.

The computer is brought back online ten years later (presumably the life support systems continued to run). The date is now year 10 of the new calendar. All data that Emperor Ryu considered dangerous had been wiped, *Mute had her entire 1,600 year memory erased and awoke in a world where Ryu was firmly in control. She was taught the new way of things and assimilated it, never knowing that she now worked as head of security for the man who destroyed her.

All of this is revealed via logs in Hate Plus; the second game. Hate Plus begins when this chunk of code is discovered by the protagonist, but we can't get into that yet. Let's move to Analogue: A Hate Story, which was the first game, but takes place 300 years later.


The year is now approximately 320 on the new calendar, 4,500 on the old. Emperor Ryu's coup was three centuries ago, and during his time he rewrote the history books and deleted any mention of the old era. No one knows what happened in year 0, and no one really cares. The Ryu family continues to hold power, and has converted the Mugunghwa from a sexually and socially liberal, meritocratic caste system to a conservative Neo-Confucian empire with all positions of power hand-picked by the Emperor himself.

As you are introduced to Analogue, *Hyun-ae asks you to view the logs from the perspective of the year 320 Mugunghwa mantra "Men are honored, women are abased". It is argued that this mantra loses something in the translation to English, and that 'abased' is not the right word, but regardless, the society of the time places women in a role where they should be subservient to their husbands and provide children.

It is not hard to understand why: the gamma radiation leak on the Mugunghwa was never fixed because the computer was shut down before the scientist from Hate Plus could present her findings. It is implied now that 35 is considered "Old". Birth rate is continuing to plummet, stillbirths are becoming as common as live births, and the new regime had brought with it a fear of science and medicine so no one is even aware the problem exists anymore. This, combined with the continued and heightened focus on family lineage and nobility has stressed society due to the difficulty of presenting a healthy male heir.

The Emperor himself, growing old and nearing death, has no male heir. A particularly shrewd low-tier politician sees this as an opportunity to boost his position in the hierarchy by providing him with a wife, but alas he's impotent (no doubt due to the radiation). However he remembers an old legend of a girl in an egg deep within the ship, and goes looking.

He finds the stasis pod of the "Sick Daughter" from pre-Ryu times. However by now Chinese has become the official language of the ship and he is unable to properly read the text on the pod. He mistranslates it to "Pale Bride" and comes to believe she exists purely to be a wife for the ailing Emperor. Breaking her out of the pot, she takes the Pale Bride home and begins to groom her for this task.

The Bride, having been put into stasis somewhere around 2,500, has radically different expectations of life. She's a 12 or 13 year old girl who wants to play, learn, and has aspirations for a career in science. However she's awoken in 320 new-era Mugunghwa where her lot is decided for her from birth: learn to be a "Good Woman", get sold off to a noble man as a wife, bear children, and raise them. Furthermore she's painfully aware of her disease, but her attempts to explain her condition are misconstrued as bragging about her beauty. In this era, her illness gives her traits desired in a wife (weakness, paleness, gaunt figure, etc).

The logs are from a different perspective in Analogue than they were in Hate Plus: While Hate Plus was mostly official records and reports, Analogue is all personal diaries. Probably implying the breakdown in structure on the Mugunghwa since the Council and boards are now gone.

As a contrast to the log entries in Hate Plus that go into details on homosexual and non-standard relationships, the "dirty" logs in Analogue are very prim and proper and basically boil down to "I had sex with my husband, I believe I did not perform properly". There is one exception, where a lesbian relationship occurs, and the actors involved are considered whores and shamed for it. Again, some may see the naughty side-arcs unnecessary but I believe how the people on the Mugunghwa approach sex gives insight to their cultural identity.

Following the Pale Bride's log is depressing: she paints a picture of a society where she's little more than a possession. She's forbidden from entering the men's area of the building, forbidden from using computers except for her tasks and light diary writing, is expected to delete anything she writes to make room for men's business on the ship's computer, and is not taught to read Chinese because no one sees it as important. Somehow she's able to get a terminal that displays in her native Korean, though. It's implied she hacked it.

At 16 she's considered ready to be married off to the Emperor, who has rewarded her adopted father with a high position in his court. The Bride, still headstrong and solid in her beliefs, refuses to go. She threatens to tell the Emperor that her surrogate father conspires against her if she's forced. For a time she believes this would win the fight for her and she'd be allowed to find her own way in life. Just the opposite, her adopted family responded to this by cutting out her tongue, then telling her soon-to-be husband that she's mute in addition to all of her other "Traits". The Emperor was elated, believing this to be the trait of a perfect woman. From here, The Pale Bride is mute, only revealing her thoughts and feelings to her diary.

Most of the story from here is emotional reflection of the Pale Bride. She discusses how she hates life and only finds joy in two things: her sexual escapades, and her time with the Emperor's other bride, who is old and barren (again, radiation probably). The other bride in this case is extremely supportive, believing that she is broken since she is barren, and that the Pale Bride is a redemption of her honor and the family's ability to continue. Despite the societal influences, though, she's a downright good person who gives the Pale Bride the only comfort she knows on the Mugunghwa.

As the Pale Bride approaches 18 and grows sicker from both her condition and the radiation, her only friend on the ship grows ill as well. She dies at the age of 40, mysteriously. Doctors at the time make a completely nonsensical diagnosis and call it, effectively "women die for no reason, oh well". This sends the Pale Bride into a rage; most of her log entries from here until the end are almost incoherent in her anger and depression.

Knowing she's about to die from a condition no one is even aware of, knowing no one cares, knowing she's only expected to provide a son for the Emperor and then she's "of no use" anymore, and now knowing her only friend died (probably of radiation poisoning) and no one is even bothering to find a diagnosis, the Pale Bride breaks. She uses her old era knowledge to gain access to the ship's systems and terminates life support functions, killing everyone on board. In the time it takes for the ship's atmosphere to vent into space, she climbs back into her stasis pod and uploads her consciousness into the ship's computer, taking the place of one of the ship's AIs.

This is where the logs of Analogue end. The ship floats derelict for about 400 years before it's found by the protagonist. We'll touch on events revealed in the game's real-time briefly too.


It's now 4,850. You, as a wrecker, just discovered the Mugunghwa and found *Hyun-ae. She acts odd for an AI, gives you access despite lacking clearance, and assists you in locating logs in the Mugunghwa computer to help find out what happened. As you read logs that start out innocent and seemingly irrelevant, and slowly grow more jilting, she asks you questions about your feelings on the matter. With each answer, assuming you don't completely infuriate her into dropping the link, she opens the next section of even more shocking logs.

A short jaunt into this cycle of read-and-answer, you find the personal logs of the Emperor, who foolishly wrote down the ship's password. This is part of the plot, giving you access so you can continue, but it also reveals *Hyun-ae is not who she says she is. The root password for the ship is her name, and the log entry implies she was his wife.

Once you confront *Hyun-ae with this, she relents that she used to be a living girl, and was uploaded into the ship as an AI. Around this time you discover the deactivated AI of *Mute. This is not the *Mute of old era Mugunghwa though. When *Mute is reactivated, she's more or less in line with the "Women are abased" outlook that everyone else had. She calls *Hyun-ae a murderer and insists you don't trust her. You are placed in a position where you are forced to choose who you wish to listen to.

Depending on who you trust, you'll get the whole story from either *Hyun-ae or *Mute. Both stories are the same: they're told via the same log entries. What's different is how the respective AI wraps up the story. *Hyun-ae reveals that she was the Pale Bride and is repentant for what she did, but explains she couldn't live like that anymore. *Mute calls her an unfilial murderess and a coward. Ultimately it's up to your own morality to decide who is right, if either of them.

Also keep in mind that by this point, if the writer has done her job, you have an emotional attachment to *Hyun-ae and a logical attachment to *Mute. *Hyun-ae was put through hell, and snapped, like any of us probably would. *Mute was just doing her job and just going with the society at the time in her focus on gender equality.

Once you decide who you trust, you get access to everything on the ship and it's time to leave with the data. Depending on who you trusted, the AI will ask to come with you. Also, if you play through the game twice to get both sides of the story, you'll have access to information you can use in a third play-thru to break the fourth wall and get both AIs together and conversing with each other.

The canonical path is probably this 100% play-thru: *Mute, in the course of calling *Hyun-ae a murderess, can be cajoled into challenging you to change her mind. She dares you to show her something that proves *Hyun-ae was pushed to do it. Using info only available during the *Hyun-ae arc, you can force *Mute to read a document you don't see otherwise: *Hyun-ae's story leading up to terminating life support. Upon reading this, *Mute has a change of heart, and teaches you how to activate both AIs at the same time (previously impossible).

*Hyun-ae and *Mute converse and come to a shaky agreement. *Mute still hates *Hyun-ae to a degree because she killed everyone on the Mugunghwa, but no longer sees her as an insane murderess, but instead someone who was pushed to the breaking point. Together they work out a way to transfer the ship logs, and both their AI cores, to your ship, and you depart the Mugunghwa with your mission complete.

Hate Plus takes place on the three day trip back to Earth, wherein whichever AI you rescued (or both in the case of the 100% ending of Analogue) discovers the note *Mute left for herself way back in year 0. You use the data hidden in *Mute's code to find the logs from back then, and unravel the mystery of what caused the Mugunghwa to fall. This is mostly historical reading. *Hyun-ae and *Mute will stick by your side no matter what and you can't really lose. However when you reach Earth, depending on how you handle certain events, *Hyun-ae and *Mute may choose to stick with you as sidekicks, or depart.


So what does this all mean? Let's start from the very top, way way back in 4,150:

This all began with people playing politics. The Mugunghwa of 4,150 was corrupt. Nepotism ran rampant. Elections and tests were rigged to keep families in power. The *Mute of 4,150 was concerned with keeping the status quo because the status quo was safe. In doing so she hatched a scheme so elaborate and fragile that the right people were able to inject their own vendettas in a ploy to seize power.

Arguably Ryu and his co-conspirators weren't wrong. With the birth rate falling sharply, something had to be done. However their Neo-Confucianist stance was anti-science, and rather than researching the cause he put the blame on women. He pushed forth this group's own plan to fix the situation, and was disenfranchised by a corrupt system to the point that he felt empowered enough to take matters into his own hands. I would personally submit he wouldn't have seized power if the system wasn't already corrupt and blind to anyone but the nobles in power.

When Ryu seized power and was able to neuter *Mute's ability to stop him, he imposed his ideals on the people. His character seems to be of someone greedy for power, but ultimately also believing his way is the right way.

This situation was made worse by denial of education to the masses. In the years leading up to Ryu's power grab, the official language of the ship was changed to Chinese, but only nobles were allowed to learn the language. This would only change when the new era was in swing and no one could possibly question Ryu's power. Furthermore, this change of language would make it impossible for anyone in the new era from reading anything from the old era, if it were to be found, hence Hyun-ae's troubles when she was alive.

As a result you can see a sharp, sharp contrast between the level of education and understanding in 4,150 and that same level in 320 by the new calendar. Society had moved backwards to the point that the Mugunghwa was effectively some kind of techno-feudal China. The new era Mugunghwa was not a pleasant place to live, even for the nobles.

On top of this, it's worth noting that the women were subjugated openly, but the men were subjugated discreetly. Marginalizing women and putting them in a role where they are to cook, clean, produce children, and be taken care of by men without a voice of their own did something to society in general. There's a focus on manhood being related to being able to take care of a wife and run a family on your own. Society in the new era Mugunghwa was extremely abusive of anyone who failed to either support a family, or produce children, to the point that several people commit honor-based suicide when they fail.

I think ultimately that's what the work gets at: when you subjugate someone, you become less of a person yourself. It's unhealthy for society and puts even those not subjugated in a position of failure. The old era Mugunghwa failed when people tried to subjugate the Peasants. New era Mugunghwa failed when society tried to subjugate women. You fail if you try to force your will on either *Hyun-ae or *Mute.

The noble families from the old era are still the noble families from the new era, thanks to the laws imposing horrific nepotism, but they're not the same. They're not the brilliant best of the best in their field. No longer are heirs expected to become the best in order to take over. Society became stupid because the people in charge were allowed to be stupid. In the end, no one is in control; the new era Mugunghwa was rudderless. No one even cared about things like life support, steering, communication... there was no crew. It was a bunch of idiots living in a box in space.

The descendants from the Councilor of State drink themselves to death out of misery that they're no longer on top of society. The descendants from the Councilor of Science are a broken family because the radiation denies them heirs. The descendants of Ryu are still the Imperial bloodline, but on borrowed time because they're prisoners to their own ritual: unable to go back to a time when they could figure out what's causing the birth rate drop and fix it. In the end, everyone's fate is bitterly ironic. If not for Hyun-ae killing them, the Mugunghwa probably would have imploded in its own incompetence (and the radiation) in the next few generations.

And that's the crux of it. When your focus is pushing someone down, you don't rise yourself. Society plummets to the bottom as everyone tries to push everyone else down. In the end, no one knows how to turn the life support back on.

Posted by trysdyn | Permanent link

Thu Mar 13 00:05:58 PDT 2014

HTTP Live Streaming and its implmentation client-side

Some time ago I wrote a wrapper around Livestreamer to let me monitor Twitch streamers I enjoy watching and shove their streams into a media player. It's a pretty swank wrapper: I can provide a list of streamers, or even a search term, and the wrapper will keep something going to the player at all times. If one streamer stops, it'll find another, etc.

Well, not long ago I discovered that Livestreamer is just a wrapper to handle HTTP Live Streams (HLS), and now and days media players can handle that natively, so I decided to drop Livestreamer from the app and just send HLS m3u8 playlists straight to my player. This is where the trouble began. It seems that every major player has a major problem handling HLS.

LiveStreamer itself: Can't detect when a stream ends and will halt, playing the last few seconds of video in a loop until killed. It'll also enter into this mode on network hiccups without re-establishing.

VLC: Pegs a core as soon as I begin viewing an HLS. As far as I can tell it's something in the handling of the m3u8 or network transaction to retrieve the m3u8s. Handling the actual video is fine, I can download TS files individually and play them fine, I can play the TS files where they sit via HTTP fine. It's only when I open the m3u8s themselves that my CPU usage goes through the roof. HTTP proxy investigation shows VLC isn't hammering the server, so I'm not sure what it's busylocked on.

MPC-HC: Perfect... almost. It'll play fine for hours, then when some kind of event causes the buffer to empty (a network hiccup, poorly timed check-in with the m3u8, etc), MPC-HC enters a busylock and will not attempt to re-establish the stream. It'll freeze on the final frame in the buffer until killed and restarted. Will also die horribly if something interferes with video output (eg: starting a game/app that needs exclusive display use for one of my displays).

Mplayer: Disappointing! It handles the HLS fine, but doesn't cache the stream in any manner. This means every time mplayer needs to retrieve a new chunk of video, the buffer empties and the stream halts for a moment. Without aggressive A/V sync settings (which cause more studders), A/V desyncs horribly. Mplayer devs point to a bug in FFmpeg and WONTFIX.

FFPlay: Same as above, shockingly enough :)

QuickTime: Aside from the lack of Linux port and a broken installer on Windows, and the proprietary lock-in... QuickTime doesn't seem to want to open ANY URL as of its most recent version. I think it's a bug in the Windows variant. Odd... I thought QuickTime would handle HLS properly considering HLS is Apple's baby. Furthermore, QuickTime seems to lack any kind of control to terminate when the video ends, which is mandatory for the way my "Start a new stream when one goes off air" wrapper is set up.

So at this point what options do I have? The easiest option is to stick with MPC-HC and just deal with the 2 or 3 times a day the stream just halts until I restart the player. The other option is to implement the HTTP portion of HLS in Python, and pipe the video to VLC in STDIN. This is nasty but would give me more flexibility in how I handle the stream, errors, and the like.

I'm just kind of peeved that no player seems to just handle HLS properly. I did a ton of digging into the VLC bug today and it seems it might even be specific to Twitch, which would be really odd. I'm not sure though; I don't really have the drive to go further into my troubleshooting than I already have. Staring at Wireshark is kind of my limit.

Posted by trysdyn | Permanent link

Sun Oct 13 06:04:11 PDT 2013

Occult Meddlings

Yesterday evening as I was wrapping up work, a friend of mine contacted me, asking me if I could throw together a quick Python project. They needed a large, window-less countdown clock they could lay over a livestream, for a rather large project their workplace was doing. I'd never done any deep level of Windows GUI muckery, but I figured why not?

After producing a quick and horrible hack to meet the project's tight timeframe, I took a few minutes to discuss the project with said friend and asked why they didn't just use a readily available tool, rather than get one custom-coded. Turns out they couldn't find one. Not one that was flexible enough to use for any livestream with a minimum of reconfiguratory fuss, anyway. One thing led to another and the next thing I know, I'm uploading the cleaned up project to Github.

You can find OCCult (the Overlay Countdown Clock Utility) in binary form here and its Github project page here.

Yeah that's it. Terrible thing I used to learn wxPython. I'm hoping at least one streamer or two will find it handy, though.


Posted by trysdyn | Permanent link

Tue Sep 10 14:09:18 PDT 2013

The Things You Find in httpd Logs

Normally I don't do fluff posts. I don't update often because I don't usually have a whole lot to say. I suppose this is the exception. I was perusing my logs today (because I get few enough visitors that I can still hand-inspect my httpd logs for strange traffic patterns) and came across this little trinket:

x.x.x.x - - [09/Sep/2013:19:00:45 -0700] "GET /blog/images/favicon.ico HTTP/1.1" 200 1406 "-" "Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20100101 Firefox (By displaying your content to me and/or saving the following Terms of Using the Service you hereby agree to the Terms available at: http://rys.io/static/touts-en.txt)"

Now, to save you the trouble, here's the contents of the linked document in their entirety:

TERMS OF USING THE SERVICE
--------------------------

By displaying your content to me and/or saving the following Terms of Using the Service you hereby agree to the following Terms:
 1. you will not track my browsing habbits outside of your domain;
 2. when tracking my browsing within your domain you will do that only after my express consent;
 3. you will not use any of data generated by me without my express written consent;
 4. at no time will you hinder my ability to access my data;
 5. you will fully comply with any and all my requests to delete my data, including copies, backups, cache and similar.

One can argue that, by posting this individual's log hit to the public and retaining the log data, I've violated terms 2 and 3. However, let's be honest: this is complete bunk. It can't be enforced and I believe the author is fully aware of that fact. So I'm not worried about it.

After some poking I came across this document on the same domain, which pretty much outlines that this "TOUTS" (terms of using the service) is a protest of some of the whack-ass stuff armchair lawyers stick in their content, surrounded with a thin veil of official sounding rhetoric, and expect to stick. Honestly, at first I was going to write up this huge rant about shrink-wrap TOSes and binding no-action agreements, but this guy's already done that legwork for me.

In this light, I applaud this cheeky little show of defiance. I don't know why but I find this absolutely adorable. I would propose adding a term #6 that mandates reading the page explaining the joke, personally. I had to do some digging to find that this wasn't yet another case of someone expecting some inane "You agree to..." to stick in a court just because they took the time to type it out.

PS: Term 1 of the agreement: "Browsing habbits"?


Posted by trysdyn | Permanent link

Sat Jul 6 05:09:11 PDT 2013

My Journey Into Determinstic Password Generation

Let me put one thing out there: I hate passwords. Passwords served a purpose at one time: when logging into a system meant being at that system's physical point of presence; or at least somewhere conspicuous. An attacker didn't have an infinite amount of time to attempt to brute force the password, because someone would eventually notice that strange fellow sitting at a workstation that didn't belong to them. Passwords should have died a few decades ago, when it became feasible for an attacker to just try every possible combination until they got in. They didn't; it's what we have to deal with, so a common topic of conversation among my contemporaries is how to work with what we have and make password auth as secure as possible.

One such contemporary is a fan of deterministic password generation like SuperGenPass and cobbled together Passacre. In a nutshell, this widget (and SuperGenPass) takes a block of input (specifically: a personal pass phrase and a domain) and generates a password from that data. The generated password will always be the same given the same input data, so you don't need to memorize a new password for each service. You simply memorize your master password and use this tool to hash out a unique password for each site, using the master password as a salt in the hashing function.

The goal of such tools is to enable the generation of unique, secure passwords that are not reused and allow for secure retrieval of said passwords if they are lost, while not storing those passwords on disk, encrypted or not. Both of these tools do this in a way that does not rely on having anything at hand, or remembering anything but your master key. If your house burns down, your desktop is torched, and you don't remember anything but your master password, you can recover everything if you're using either SuperGenPass or Passacre. You can just grab the respective tool again, plug in your info, and the generated passwords will be identical to what they were before.

It's a shiny thought: password persistence without the persistence. As a result, you can use much longer and more secure passwords since you don't have to sweat forgetting them. One can argue something like Keepass does the same job, but with Keepass you're left reliant on a key chain file that can become corrupted, destroyed, or be stolen. With deterministic password generation, you have the potential of recovery without the risk of having someone find your key chain and steal your passwords.

To present a super simple proof of concept, check this out. This is a python snippet that basically does what SuperGenPass/Passacre does. However it's very basic and insecure; generating passwords that are not very deep or entropic. I chose this method because it's easy to understand and the reader can follow along because it only uses the Python standard lib.

import hashlib
import base64

def generate_password(key="", username="", domain="", length=16):
    """
    Take a key, a username, and a domain, concatenate them, and return a b64
    encoded SHA-512 hash of the string, truncated to 'length'
    """
    h = hashlib.sha512()
    h.update(key+username+domain)

    return base64.b64encode(h.digest())[0:length]

All we do here is take a key (which would be your "master password"), a username for the service you're trying to log into, and that service's domain. The function concatenates those three strings into one, then generates an sha512 digest of the resulting string. Like this:

>>> import hashlib
>>> h = hashlib.sha512()
>>> key = "some_secure_string"
>>> username = "trysdyn"
>>> domain = "foo.com"
>>> h.update(key+username+domain)
>>> h.digest()
'L\x07}z\xe0\xc1\xe6\xe6\x0e\xfe\xb5\x8c4\xf1 [...]

That's not so good for a password, is it? Sure, we can get the hexadecimal digest, but I'm not too hot on the idea of a password with character depth of just 16. I'm taking the lazy way around this and encoding the digest as a base64 string. I've already warned the reader that this example is insecure, so let's continue with base64.

>>> import base64
>>> base64.b64encode(h.digest())
'TAd9euDB5uYO/rWMNPEiQ0AO+jtm72pVyngaMDYXw+oR/oML91WdQQgwM9zYzcDxYGI2hBBK6udcuUSGxD+ypA=='

Not bad. Let's assume we're generating a password for a site with a 16 character password limit. We truncate to 16 characters and get "TAd9euDB5uYO/rWM". Well... that LOOKS like a password, at least. We can use this as our password and if foo.com gets hacked and this password stolen, the attacker will only have access to our account on foo.com. Decoding this string from base64 back to its original data will give the attacker a blob of worthless crap since it's only 16 bytes of a sha-512 digest. The chance of having this reverse engineered based just on that is slim.

Now let's look at this simple example and discuss some of the problems:

  1. Sites have different requirements for password length. An astute observer will note my example function allows for adjustment of the password length, but then you have to remember what length you initially chose.
  2. You have to be super careful choosing an encoding scheme. I used base64 here as an example. However it's also probably the scheme most likely to never run afoul of weird password requirements. That's kind of bad because you can do much better with your passwords than base64 strings. Passacre gets around this by allowing the user to specify a set of allowed characters in their passwords per site. However this means you now have to remember or retain an encoding scheme for each site on which you wish to generate a password.
  3. Some sites are, frankly, stupid and have such draconian requirements for passwords that you will need yet more special logic beyond a custom encoding scheme. Think... exactly 8 characters, with a number following a letter and the number cannot be at the end.
  4. Password rotation will leave you in the weeds. You have three choices really if you want (or need) to rotate passwords: rotate your master key, begin sprinkling 'sugar' on the generated passwords ('Password1', 'Password2', etc), or have some kind of modulator built into the tool that allows generation of multiple passwords per data set. In all cases, you now have to remember whatever crazy modulation you added to the process, so you can get your password back next time.
  5. If using a method that's easily recognizable, published, open source, etc, you're left relying on keeping your master password secret to keep your passwords safe. If someone manages to reverse engineer your master password and method, they now have access to all of your passwords. Worst case: an attacker knows you're using my method and compromises one of your passwords. They can now use the generator to brute force your master password. You can add additional salts, twist the method you're using, and other things to obscure what you're doing, but this also increases your burden of memorization if you wish to retrieve passwords later.

As you can see, the solution to most of the noted problems is "Memorize more stuff". Suddenly this stops being so easy. To get around password length and content requirements, rotation, and single point of failure you need to memorize a master password, password length per site, password encoding per site, any extra sugar you sprinkle on passwords to deal with password rotation, and any additional salts or obfuscation you do to increase security. At that point you might as well memorize the damn passwords!

Part of the problem here is sites won't tell you when you've submitted a password that violates their password policy. In a way this is good. Letting an attacker know what your password rules are would significantly decrease the amount of work they'd have to do to brute force a password. It makes it a real pain in the lower end, though, to remember if you generated a 16 character or 32 character password for a given site. Did it have +'s or did this site disallow +'s? So on, so forth.

In a nutshell, the problem with deterministic password generation is no one deterministic method will work 100% of the time, and for each password you need to apply special rules for, you need to memorize more and more stuff. Eventually you'll reach a point where you're memorizing just as much crap as you would be just memorizing passwords. Still, I love the idea of deterministic generation via cryptographic hash. I want to keep working on it, but I suspect the burden of memorization will always be there.

UPDATE: After talking to the author of Passacre we hashed out that his goal is to have all of the variables in password generation stored in a config file. At that point you have things like password length and schema stored in the file and don't have to remember it. I (and I believe the author too) believe this dings one of the main benefits of deterministic generation: you now live and die by a config file, just like you would live and die by a key chain file with Keepass. With that said, you still aren't storing your passwords on disk, just the recipes needed to regenerate them. I suppose whether or not this is worthwhile is left to the reader.


Posted by trysdyn | Permanent link

Wed May 29 01:40:12 PDT 2013

My Project Du Jour

Generally speaking I don't publicize my projects. Too many fail in the early stages. I have project ADHD. Work is so busy, life is so busy, so on and so forth. I'm kind of proud of this one, though, so I figured I'd mention it.

This is a retro Wizardry clone engine I whipped up in Python. It's nothing impressive; just some SDL and a bunch of line drawing and font rendering. It's almost BASIC-ish in its simplicity. What has me excited, however, is not the depth of the engine but what I'm considering doing with it.

I'm a bit of a procedural content whore. I've been following Starbound for awhile and that kind of thing has me excited. However I also have a soft spot for retro dungeon crawlers, and I've never seen one done with more of a procedural content bent.

Enter this, as of yet unnamed, project. It's more of an experiment than an intent to make a game that becomes popular, but who knows. My plan is to build an engine and basic game ruleset, then procedurally generate as much content as is feasible. So far I've worked out the dungeon generator, which is standard procedural content fare, but I'm also planning on constructing a few less common things:

  • Monsters, their names, locations, loot, skills, stats, etc will all be generated
  • Monster sprites will be generated, to a degree. Palettes, equipment on humanoid monsters, etc will be dynamic
  • The locations and identities of the big bads will change each game. These big bads will each run a faction that moves within the dungeon
  • As game time progresses, factions will expand their influence over the main dungeon area, interact, clash, and migrate
  • Spell costs, levels, and effects will be re-rolled each game. This includes their names, which are in a fictional language in the spirit of old school Wizardry (with an 'easy mode' option to just show them in English)
  • Unique artifacts will be rolled fresh each game. One game the ultimate item to find may be a powerful sword in the hands of a boss monster. The next it may be a magic-empowering crystal hidden deep in a side-path of the dungeon

The idea is to require experimentation each fresh play-thru, in order to come to understand not just the dungeon, but the weaknesses, strengths, and capabilities of each monster, as well as the weaknesses, strengths, and capabilities of your own gear and spells. One game, one of the main baddie factions may be dragons, may be red, specialize in fire, and require ice spells to easily defeat. The next game, the same faction may instead be orcs who are resistant to the elements, are nasty in melee, and that spell you knew as 'Ice' is now a heal spell instead.

I hesitate to call it a "Roguelike" because people get pissy when you use that word to address anything not an ASCII top-down permadeath hall-and-room game. Still... yeah, it's kind of a first person, party-based Roguelike.

As far as progress goes, I have those "Core Rules" I mentioned done. The drawing engine, the UI, most of the combat and menu fun. Before getting into the meat and potatoes of all the procedural generation, I still need to wrap up a few dangling bits with the interface. I realized while playing with the engine that mouse support would be far more useful than I originally thought, so I get to go back and shoehorn that back into the code.

Should be a fun experiment. Now let's see if I can ever find time to work on it...


Posted by trysdyn | Permanent link | File under: project

Sat May 18 18:00:26 PDT 2013

Why the new Google Hangouts screws over users

Okay so, recently Google announced that they're moving from Google talk to Google Hangouts. Google has stated that Hangouts will be a more awesome, unified experience, bringing Google Plus's Hangouts feature straight into the IM client, allowing people to start voice/video chat hangouts from anywhere with minimal effort. It'll be a grand reinvention of how they've been doing their IM stuff and it'll be great. I admit, the features they promise sound pretty cool.

One feature they quietly (and I mean really quietly) removed, however, was support for XMPP federation. A lot of people don't even know what this means, so there's not really been much press about it. This post will hopefully help explain why this is a big deal to people (especially me), and why this is personally upsetting, and why I think it shows some bad faith on the part of Google.

First, what is XMPP: XMPP's site explains it better than I can. XMPP is an open protocol for instant messaging. It's been around for forever, it's stable, and most importantly anyone can set up an XMPP server. By using XMPP, I'm not reliant on the good faith of a company like Google, AOL, or Microsoft to be able to communicate. At least that's the theory. XMPP, as far as Google Talk is concerned, has allowed people to use their client of choice when connecting to Google talk as any XMPP client will work with it, and has allowed people using Google talk to communicate with people connected to XMPP servers. This has caused an increase in the number of XMPP users because almost everyone has a Google account, and therefore XMPP users could reach almost anyone they wanted through Google Talk. It's been a great victory for open software and open protocols!

That's all changing. Like I said, Google was very quiet about it. I didn't even realize they'd stopped supporting XMPP in their new Hangouts app until I asked my boyfriend why he'd stopped responding to my IMs. He claimed he never got them. Posts on the Google product forums noted similar confusion until someone curtly pointed out this page. Google's announcement of dropping support for XMPP comes in this form: a banner on a few of their help pages, and an announcement somewhere I can't even find. They claim (at least this is what I've heard from third parties so... grain of salt) that this is necessary to improve their product and increase the unification of their chat software. Meaning, they want to move away from the XMPP standard because they don't want non-Hangouts contacts appearing in buddy lists that may or may not support certain Hangouts features coming down the pipe (video chat hangouts I guess would be one example).

I call bullshit on this. The XMPP standard has clear guidelines for supporting video and voice chat and Talk has even allowed voice chat to non-Gtalk XMPP users in the past. All the protocols Gtalk uses are fully documented so clients can support them and most are open standards as part of the XMPP feature set. I will admit that the XMPP extended features and compliance therein can be a melange of issues with regards to server and client support, but you run into the same support problems when trying to do anything with voice and video chat (do you have a camera? what about a mic? is your system set up properly?).

But enough of that. Let's talk about this is actually doing to XMPP users. Right now, as it sits, I have 42 online buddies on my XMPP contact list. 36 are Gtalk users. Of those 36, who appear online and active, I cannot talk to about 20 and that number is expected to grow in the coming days. Why? Because Google hasn't entirely killed XMPP Federation. I can see these buddies online; I can even see them online when they use the Hangouts app. If I attempt to send them an IM, however, it never goes through. Looking at this from the Hangouts side, these buddies don't see me online at all. I have totally disappeared from their contact list. If I had to guess, I'd imagine Hangouts fully supports XMPP, but the client is hard-coded to ignore IMs from people not on your buddy list, and the XMPP contacts aren't ported over.

Long story short: 90% of my buddy list is now polluted with contacts that appear online, but I can't talk to. They're giant ads for Google Hangouts, expressing to me that I will never be able to talk to these people unless I get Hangouts and drink the Google Plus punch. This is entirely a show of bad faith in my opinion. Even if there's no intent to use my buddies as billboards for Google Hangouts, this is a terrible violation of the XMPP spec, and an act of ignorance of good software development practice. Rule one of software development is "Don't surprise your users". Having contacts that appear online, but silently and without feedback cannot receive my IMs, is about the worst surprise you can give a user of an instant messaging system.

So in summary... Google has pulled the plug on support on a protocol they've helped popularize, after years of promising interoperability, for reasons that are dubious at best, and in a way that leaves people who don't jump to the new Hangouts app unable to talk to their contacts without any feedback that their IMs aren't getting through... and they've done that with no warning to anyone. I imagine there's a bunch of people out there wondering where some of their buddies have gone, or why their messages aren't getting responses, because this isn't documented anywhere.

The counterarguments I hear on this are, as above, that it'll make the service better. I disagree, but only time will tell. The other argument is that Google doesn't owe XMPP users anything, and I'm expecting a lot to be able to talk to Hangouts users while not even using Hangouts myself. I have a few problems with this... mainly that yes, they don't owe XMPP users anything, but they're using the XMPP spec which has a mission of being open and interoperable, and that's a mission that Google built Gtalk on. Gtalk got popular because of its interoperability: Gtalk to AIM federation? That's XMPP at work. The ability to connect to Gtalk with Pidgin? XMPP again. The ability to connect to Gtalk from multiple clients and the server just knows where to send IMs? XMPP yet again. Google built Talk on XMPP and are now neutering its most identifying feature. Secondly, Google still gets what they want from me, even if I don't use their service directly. How? Because 90% of my buddies use Google talk. They can still datamine my conversations; most of them anyway. What else do they want from me?

What do I expect Google, or users, to do? Nothing. Vanilla XMPP users are a minority. Always have been, probably always will be. Most people who use XMPP (Users of Gtalk, Facebook chat, etc) don't even know they're using it because companies have an interest in neutering the one feature that makes XMPP unique, identifiable, and great: server to server federation. I just wanted it documented why I'm upset, because a lot of people who don't know the origin, core, and philosophy of the protocol Gtalk has been using for years haven't understood why this is upsetting and a very 'evil' move on Google's part.

PS: This also means you people who have relied on Gtalk to talk to AIM contacts are out in the cold too.


Posted by trysdyn | Permanent link | File under: rant