2017-06-29: The great OStatus experiment, concluded

I've been doing experiments into Mastodon and GNU Social for about two months now, with my own GS instance running for about two weeks. In that time I've become pretty familiar with the OStatus protocol that powers both, the differences between the two, how they inter-operate, and the political issues that both led to the division, and continue to spark contention between the loyalists to the two. Sadly, I made the decision today to close my GNU Social node and instead do my microblogging on a Mastodon server a friend owns. I think it important to share why for people confused about the state of the "Fediverse" in the future.

Getting out our history books

To fully understand this, some historical rehashing is required. You can probably skip this if you don't need the whole story.

Ages ago, OStatus was developed as a protocol for federated "distributed updates" (microblogging). Instead of having a monolithic server like Twitter, you'd have dozens or hundreds of servers each serving some users and sharing posts between them. Even if one server fell offline, the posts would survive since they were federated out to peers. Furthermore, server admins could run their instances in the way they chose, instead of relying on a central service to make those decisions for them. This protocol actually powered identi.ca in the beginning-- for you Twitter OGs.

Through several hand-offs and changes of leadership, OStatus fell under the umbrella of the GNU Project and so came the flagship implementation of OStatus: GNU Social. Meanwhile the original leaders in OStatus left and moved on to form a competing federation standard: Pump.io. In the end, the GNU Project ended up making decisions for what became known as GNU Social; that's the important part.

GNU Social mostly served as a social well for people who either felt Twitter was too monolithic and had too much control over their data, people who wanted to try something different, or people who got banned from Twitter. The "Fediverse" as they call their federated network was alive but didn't have nearly the inertia to be a Twitter competitor.

Mastodon in a nutshell

Fast forward to 2016. Mastodon launched. I don't claim to speak for the founding folk on their mission, but judging by development direction it's clear one of Mastodon's targets is to give users and instance admins better tools for combating toxicity and harassment, as well as generally making a less centralized alternative to Twitter. Mastodon uses OStatus, which means it can federate with GNU Social instances. This gave Mastodon a pre-established user base to tap into, to make some headway against the user base critical mass problem. Mastodon also has a sleek Tweetdeck-like interface that makes it easy to get in and start posting, probably attributing to their growth.

On a UX level, Mastodon offers several things Twitter doesn't: enhanced per-post privacy options, content warnings and cuts, and an extended post length limit. These features (aside from the length limit) exist as an extension of the OStatus protocol, meaning they're new and any software using the base OStatus implementation does not support them. This includes GNU Social and its forks, which means the pre-Mastodon installed base is all running software that has no support for features the flood of new users come to expect as a global mainstay of the system.

New users coming in largely aren't aware of this, though, since they joined "Mastodon", not "OStatus" or "The Fediverse". Which brings us to...

The problem

Since the Mastodon devs extended OStatus and launched their flagship software and instance with these extensions, while still federating with older vanilla servers, what happens when a post jumps the gap? Most of the features Mastodon offers as extensions to OStatus will just fail silently when those posts are federated out to GNU Social instances. This includes privacy. Yep you read that right. If a post is federated out to a GNU Social instance, all privacy options chosen by the poster are discarded and the post is displayed in the clear, globally, on that instance.

Once the post has passed to a GNU Social instance, it can be boosted (retweeted), replied to, or forwarded by anyone on the node it arrived on. Those boosts and replies are also in the clear and can be acted on by anyone following the booster/replier. Essentially once a post touches a GNU Social instance, its privacy settings are completely moot and the post is world-readable.

This also applies to content warnings. If a Mastodon user places a post or image behind a content warning, it will arrive on a GNU Social node in the clear. Conversely, GNU Social users are not capable of CWing their posts at all, which can cause some miscommunications about their intent and decorum to Mastodon users.

Why is this the case? Because the extensions to OStatus that Mastodon offers as major killer features are unofficial, and unsupported outside of Mastodon. I'm unsure the path the Mastodon devs took to get to this point. Did they try to get their extensions integrated into OStatus? It would probably be moot if they did because looking back at who was there before Mastodon came to town, privacy would not be large on their list of killer features to add to the protocol. OStatus, and GNU Social really, at this point is ancient both in age and approach, too. Updates are slow and small, development has almost entirely ceased; a major protocol change would take ages to implement. What's more, you're still trusting the instances you federate with to honor those options.

So we're left with two implementations of OStatus that disagree on how to handle certain features, and fail open and insecure when such a disagreement occurs.

Where we are now

At the moment the fediverse is largely split into three groups. There's the blissfully unaware, who maybe aren't even aware of the glaring privacy and protocol problem, but are certainly not aware of the dirty politics of the divide between Mastodon and GNU Social. There's the largely old school folks who were here before Mastodon, who either see Mastodon as a gateway for undesirable people to get into their pristine fediverse at best, or see the Mastodon developer as a bully that came into their house and demanded to change the music by implementing his own OStatus extensions. Finally there's the people who think that GNU Social old school types are all out of touch neckbeard bros who are ignorant of the struggles of those who need privacy and content options. This is, of course, a gross oversimplification of how nuanced things really are.

Obviously the latter two groups are kind of at each other's throats. To exacerbate this, due to either the protocol issues or just politics, several instances have flat out made it policy to not support the other side of the issue. In addition to this, several Github issues have implications that the devs will be giving different treatment to Mastodon and GNU Social instances in the future, as a requirement for developing new features and enforcing their privacy controls.

In short, the devs are pushing toward what may become a protocol split and several instances are refusing to federate across the gap anyway. We're looking at Mastodon breaking off from OStatus and becoming its own thing sometime in the not-too-distant future if I had to guess. At least on some level. The Github issue linked above only talks about handling private or direct messages, but I suspect fully enforcing the privacy controls will require more. If not, GNU Social will only continue to fall behind in feature support and it will become even more in the best interest of Mastodon instances to not federate with them.

I personally do not see any effort from the GNU Social side of things to alleviate any of this. I suspect the problem, and the divide, will only grow with time.

What does this mean for a user/admin?

This is the important part. If you're thinking of either hosting or using an OStatus-powered service and federating, there's some decisions to make that aren't as obvious as technology choice. Also some things to be aware of:

First, whether you use GNU Social or Mastodon, some instances will refuse to federate with you. You won't be able to interact with people on those instances. The reasons for this are wide and varied and I don't feel this is the place to discuss their merit. Just be aware that's a thing. With the boiling political climate and coming changes in Mastodon 1.5, I expect this divide will only grow. If you absolutely have to reach someone on an instance that refuses to federate with Mastodon, you'll have to use GNU Social and vice versa.

Second, if you choose to host or use a GNU Social instance, you will already be missing several features that let you see the posts from Mastodon users as they were intended to be seen. Due to this missing context, you may chime into threads you're not supposed to, see images that you may not want to because their content warnings have been stripped, and miss parts of messages because they were part of OStatus tags GNU Social just ignores. This feature gap will, again, only grow with time.

Third, by the same token, Mastodon users should be painfully aware that the privacy features are merely suggestions. Not only will a GNU Social instance ignore them and blissfully blat your followers-only posts into the public timeline where they're world-readable but at this point it's trivial to mod a Mastodon server to also ignore privacy controls. You should carefully vet people who follow you before making a followers-only or unlisted post because once someone follows you from either a GNU Social, or bad-acting Mastodon instance, your posts may become world-readable via that instance.

Fourth, as an extension of the above but deserve of its own point. If you use a GNU Social node to follow a Mastodon user, you will be implicitly violating their expectation of privacy due to how GNU Social handles (or rather mishandles) follower-only posts. That should weigh on your conscience a bit.

Finally, no matter which software you use, enforcing Mastodon's privacy model will require sending fewer messages to GNU Social nodes. This means just by merit of using GNU Social you may start being forced out of certain threads and conversations or certain message types. This is a guess and only time will tell. 1.5 will bring the start of this and we can see where it goes from there.

As for my experience with GNU Social...

I'll be honest. I prefer GNU Social's default UI over Mastodon's Tweetdeck clone. I like that GNU Social is simple scripts while Mastodon is either a Docker container or a huge Rails and Node.js stack. I like GNU Social's admin tools and its simplicity stays out of my way. GNU Social better exposes groups as a feature while Mastodon hides them. In almost every way I prefer GNU Social, and yet I'm closing my node and moving to Mastodon. Why?

Looking at the list of people I either follow or want to follow, they all use a Mastodon instance. Every single one. This means a few things for me. First, that they have an expectation that private posts will be private, and by receiving them I'm violating that expectation because of GNU Social dishonoring privacy tags. Second, some of those people are on instances that don't federate with GNU Social or are considering ceasing federating with GNU Social; simple enough. Third, I'm incapable of posting content warnings from GNU Social while my peers expect it-- so I'm just unable to post certain content. Fourth, the GNU Project's mission is fundamentally incompatible with what I want from a social service and, while I can work around that, do I really want to support them implicitly? Finally, no matter how I feel about Mastodon's execution on implementing the OStatus extensions, they are a net positive for the health and usability of the fediverse and by merit of that I am obligated to support that endeavor.

So while I like GNU Social, to continue to socialize with the people I came here to socialize with, and to support the people I feel are actually working in benefit of my peers and friends, I should move to the Mastodon side of things before the inevitable split happens.

What would I like to see happen? I'd like to see Mastodon just fork OStatus, drop compatibility, and implement all the things the devs want to implement. At this point GNU Social users are in the minority; I feel Mastodon would survive if they did this. This would address the criticism that Mastodon showed up to an established party and tried to change the rules on its own, and would free the devs to do what they want without the shackles of backward compatibility.

We'll see what happens I guess.

tags: tech