Reimagine the Internet Day 5: New Directions in Social Media Research

This week I’m attending the Reimagine the Internet mini-conference, a small and mostly academic discussion about decentralization from a corporate controlled Internet to realize a more socially positive network. This post is a collection of my notes from the fifth day of talks, following my previous post.

Today’s final session was on new directions in (academic) social media research, and some well thought-out criticisms of the decentralization zeitgeist.

An Illustrated Field Guide to Social Media

Several researchers have been collaborating on an Illustrated Field Guide to Social Media, which categorizes social media according to user interaction dynamics as follows:

Category Description Examples
Civic Logic Strict speech rules, intended for discussion and civic engagement, not socialization Parlio, Ahwaa, vTaiwan
Local Logic Geo-locked discussions, often with neighborhoods or towns, often with extremely active moderation, intended for local news and requests for assistance Nextdoor, Front Porch Forum
Crypto Logic Platforms reward creators with cryptocurrency tokens for content and engagement, and often allow spending tokens to influence platform governance, under the belief that sponsorship will lead to high quality content Steemit, DTube, Minds
Great Shopping Mall Social media serving corporate interests with government oversight before users (think WeChat Pay and strong censorship), community safety concerns are government-prompted rather than userbase-driven WeChat, Douyin
Russian Logic Simultaneously “free” and “state-controlled”, stemming from a network initially built for public consumption beyond the state, then retroactively surveilled and controlled, with an added mandate of “Internet sovereignty” that demands Russian platforms supersede Western websites within the country VKontakte
Creator Logic Monetized one-to-many platforms where content creators broadcast to an audience, the platform connects audiences with creators and advertisers, and the platform dictates successful monetization parameters, while itself heavily influenced by advertisers YouTube, TikTok, Twitch
Gift Logic Collaborative efforts of love, usually non-commercial and rejecting clear boundaries of ownership, based in reciprocity, volunteerism, and feedback, such as fanfiction, some open source software development, or Wikipedia AO3, Wikipedia
Chat Logic Semi-private semi-ephemeral real-time spaces with community self-governance where small discussions take place without unobserved lurkers, like an online living room Discord, Snapchat, iMessage
Alt-Tech Logic Provides space for people and ideas outside of mainstream acceptable behavior, explicitly for far-right, nationalist, or bigoted viewpoints Gab, Parler
Forum Logic Topic-based chat with strongly defined in-group culture and practices, often featuring gatekeeping and community self-governance Reddit, 4chan, Usenet
Q&A Logic Mostly binary roles of ‘askers’ and ‘answerers’ (often with toxic relations), heavy moderation, focuses on recognition and status, but also reciprocity and benevolence Yahoo Answers, StackOverflow, Quora

The authors compare platforms along five axis (affordances, technology, ideology, revenue model, and governance), with numerous real-world examples of platforms in each category. The table above does not nearly do the book justice; It’s well worth a read, and I’d like to dedicate a post just to the field guide in the future.

The Limits of Imagination

Evelyn Douek, Harvard law school doctoral candidate and Berkman Klein Center affiliate, had some excellent critiques of small-scale decentralization.

Changing Perceptions on Online Free Speech

She framed online free speech perspectives as coming from three “eras”:

  1. The Rights Era, where platforms are expected to be financially motivated, and will maybe engage in light censorship on those grounds (copyright, criminal content, etc), but should otherwise be as hands-off as possible

  2. The Public Health Era, where platforms are expected to be stewards of public society, and should take a more active role in suppressing hatespeech, harassment, and other negative behavior

  3. The Legitimacy Era, where platforms are directed by, or at least accountable to, the public rather than solely corporate interests, bringing public health interests to the forefront of moderation and platform policy

Under this framing we’re currently in the “public health era”, imagining an Internet that more closely resembles the “legitimacy era”. Reddit is expected to ban subreddits for hate speech and inciting violence, even if they don’t meet the criteria of illegal content, the public demands that Twitter and Facebook ban Donald Trump without a court order asking them to, etc. We ask major platforms to act as centralized gatekeepers and intervene in global culture. When we imagine a decentralized Internet, maybe a fediverse like Mastodon or Diaspora, we’re often motivated by distributing responsibility for moderation and content policy, increasing self-governance, and increasing diversity of content by creating spaces with differing content policies.

Will Decentralization Save Us?

Or is decentralization at odds with the public health era? This is mostly about moderation at scale. Facebook employs a small army of content moderators (often from the Philippines, often underpaid and without mental health support despite mopping up incredibly upsetting material daily), and we cannot expect small decentralized communities to replicate that volume of labor.

Does this mean that hatespeech and radicalization will thrive in decentralized spaces? Maybe, depending on the scale of the community. In very small, purposeful online spaces, like subreddits or university discord servers, the content volume is low enough to be moderated, and the appropriate subjects well-defined enough for consistent moderation. On a larger and more general-purpose network like the Mastodon fediverse this could be a serious issue.

In one very real example, Peloton, the Internet-connected stationary bike company, had to ban QAnon hashtags from their in-workout-class chat. As a fitness company, they don’t have a ton of expertise with content moderation in their social micro-community.

Content Cartels / Moderation as a Service

There’s been a push to standardize moderation across major platforms, especially related to child abuse and terrorism. This often revolves around projects like PhotoDNA, which is basically fuzzy-hashing to generate fingerprints for each image, then compare them against vast databases of fingerprints for child abuse images, missing children, terrorist recruitment videos, etc.

This is a great idea, so long as the databases are vetted so that we can be confident they are being used for their intended purpose. Finland maintains a national website blocklist for child pornography, and upon analysis, under 1% of blocked domains actually contained the alleged content.

Nevertheless, the option to centralize some or all moderation, especially in larger online platforms, is tempting. Negotiating the boundary between “we want moderation-as-a-service, to make operating a community easier”, and “we want distinct content policies in each online space, to foster diverse culture” is tricky.

Moderation Along the Stack

Moderation can occur at multiple levels, and every platform is subject to it eventually. For example, we usually describe Reddit in terms of “community self-governance” because each subreddit has volunteer moderators unaffiliated with the company that guide their own communities. When subreddit moderators are ineffectual (such as in subreddits dedicated to hatespeech), then Reddit employees intervene. However, when entire sites lack effectual moderation, such as 8chan, Bitchute, or countless other alt-tech platforms, their infrastructure providers act as moderators. This includes domain registrars, server hosting like AWS, content distribution networks like CloudFlare, and comment-hosting services like Disqus, all of whom have terminated service for customers hosting abhorrent content in the past.

All of this is important to keep in mind when we discuss issues of deplatforming or decentralization, and the idea that users may create a space “without moderation”.

Conclusion

The big takeaway from both conversations today is to look before you leap: What kind of community are you building online? What do you want user interactions and experiences to look like? What problem are you solving with decentralization?

The categories of social media outlined above, and the discussion of moderation and governance at multiple scales, with differing levels of centralization, add a rich vocabulary for discussing platform design and online community building.

This wraps up Reimagine the Internet: A satisfying conclusion to discussions on the value of small communities, diversity of culture and purpose, locality, safety, but also challenges we will face with decentralization and micro-community creation. This series provides a wealth of viewpoints from which to design or critique many aspects of sociotechnical networks, and I look forward to returning to these ideas in more technical and applied settings in the future.

Posted 5/14/2021


Reimagine the Internet Day 4: Building Defiant Communities

This week I’m attending the Reimagine the Internet mini-conference, a small and mostly academic discussion about decentralization from a corporate controlled Internet to realize a more socially positive network. This post is a collection of my notes from the fourth day of talks, following my previous post.

Today’s session was on building platforms in hostile environments, and community-building while facing censorship and dire consequences if deanonymized.

MidEast Tunes

The first speaker, Esra’a Al Shafei had spent some time building news and chat sites in Bahrain (which has a very restricted press), but quickly and repeatedly fell afoul of censorship via ISP-level domain blocking. This was before widespread use of proxies and VPNs, so even if the service could have stayed up hosted remotely, the userbase would be cut off.

Instead, she settled on a music site, sort of indie-west-african-eastern-asian-spotify, MidEast Tunes. Music streaming was harder to justify blocking than text news sites, but still provided an outlet for political speech. This grew into a collaboration system, sort of a web-based GarageBand, where users could supply samples and work together to create tracks. This spawned cross-cultural, international, feminist connections.

Ahwaa

Years later, now that proxies are prevalent and domain blocking is more challenging, she’s returned to making an LGBT+ positive forum. While the censorship evasion is easier, the site still faces many problems, from anonymity concerns to trolls.

Anonymity is secured by forbidding most/all photo posts, representing each user with a customizable but vague cartoon avatar, and providing only the broadest user profiles, like “lesbian in Saudi”.

Infiltration is discouraged through a Reddit-like karma system. Users receive upvote hearts from others for each kind message they post, and site features like chat are restricted based on total upvote hearts. In the opposite case, sufficient downvotes lead to shadowbanning. Therefore, infiltrating the platform to engage in harassment requires posting hundreds or thousands of supportive LGBT-positive messages, and harassers are automatically hidden via shadowbanning. Not a perfect system, but cuts down on the need for moderation dramatically.

Switter

The second speaker, Eliza Sorensen, co-founded Switter and Tryst, sex-worker positive alternatives to Twitter and Backpage, respectively. After FOSTA/SESTA, many US-based companies banned all sex workers or sex work-related accounts to protect themselves from legal liability. Others aggressively shadow-banned even sex-positive suggestive accounts, hiding them from feeds, search, and discovery. Not only is this censorship morally absurd in its own right, but it also took away a valuable safety tool for sex workers. Open communication allowed them to vet clients and exchange lists of trustworthy and dangerous clients, making the entire profession safer. Locking down and further-criminalizing the industry hasn’t stopped sex work, but has made it much more dangerous. Hacking//Hustling has documented just how insidious FOSTA/SESTA is, and the horrible impacts the legislation has had.

Fortunately, Australia has legalized, although heavily regulated, sex work. This makes it possible to host a sex worker-positive mastodon instance within Australia, providing a safer alternative to major platforms. This is not a “solution” by any means - FOSTA/SESTA criminalizes a wide variety of behavior (like treating anyone that knowingly provides housing to a sex worker as a “sex trafficker”), and that social safety net can’t be restored with a decentralized Twitter clone. Nevertheless, it’s a step in harm reduction.

Conclusions

Both speakers stressed the importance of small-scale, purpose-built platforms. Creating platforms for specific purposes allows more care, context-awareness, safety. Scalability is an impulse from capitalism, stressing influence and profit, and is often harmful to online communities.

This seems like a potential case for federation and protocol interoperability. Thinking especially of the Switter case, existing as a Mastodon instance means users outside of Switter can interact with users on the platform without creating a purpose-specific account there. It’s not cut off, and this helps with growth, but the community on Switter is specifically about sex work, and can best provide support and a safe environment for those users. In other cases, like Ahwaa, complete isolation seems mandatory, and reinforces context-awareness and multiple personas. Click here for my notes from day 5.

Posted 5/13/2021


Reimagine the Internet Day 3: Adversarial Interoperability / Competitive Compatibility

This week I’m attending the Reimagine the Internet mini-conference, a small and mostly academic discussion about decentralization from a corporate controlled Internet to realize a more socially positive network. This post is a collection of my notes from the third day of talks, following my previous post.

Today’s session was on platform compatibility as an alternative to regulation, and as a strategy for community bootstrapping.

Protocols, Not Platforms

Users don’t favor competition for idealized capitalistic notions of the “free market”, but out of a desire for self-determination. We generally like having choice about our own experiences; see the complaints when Twitter and Facebook switched feeds to non-chronological, complaints that Twitter doesn’t provide a feed of only people you follow without retweets and likes.

We often speak about the power of the “network effect”: I join Twitter because so many other people are on Twitter, and if you made an exact clone of Twitter’s functionality no one would move there because it lacks the compelling population. This explains why new social media behemoths are so rare, and tend to be limited to ‘one per category’, in that there’s one big Reddit-like platform, one big Twitter-like, etc.

However, even more powerful is the “switching cost”. The network effect may attract users to a new platform in the first place, but the cost of moving to a new platform, losing their social connections, conversation history, and community spaces, is what keeps them there.

The easiest way to create a new platform is therefore to interface with an older platform, lower the switching cost, and make it as easy as possible for users to migrate en-masse. Trivial examples include Telegram and Signal using phone numbers as IDs so your existing contacts immediately appear on the new platform (although this has serious privacy concerns, especially paired with notifications when an existing contact joins the platform). Jabber (now XMPP) is a more complex example: they ran a full network bridge to ICQ, AIM, and other contemporary instant messengers, allowing their users to write to their friends on all other platforms.

Maybe the best example is email, where you’re not firmly rooted to a single email server, but can export all your email, make an account on a new email server, import your mail back in, and continue writing to all the same people. Changing servers effectively changes your username, but otherwise has zero disruption to your social network.

Limits of Regulation

Legislation and regulation move at a glacial pace (see the CFAA), while technology is quickly iterated on. If we legislate social media companies, for example requiring that Facebook expose particular APIs, then they’ll work to undermine those APIs until the regulation no longer has value.

For a very real example, auto manufacturers use a standardized diagnostic port to share information with auto-mechanics. However, manufacturers started extending the protocol, making some information only accessible using custom software, so the dealership had access to more information than general mechanics. They can’t legally mandate that only the dealership be allowed to repair cars, so they made it impossibly difficult for mechanics instead. Eventually regulation caught up and required all information sent over the diagnostic port to be in plaintext via a publicly available API. So auto-manufacturers switched to sending the extra information over wireless diagnostic interfaces, bypassing the regulation.

Reverse Engineering as Praxis

Cory Doctorow suggested legalization of reverse engineering as an alternative to regulation. If mechanics could reverse engineer the secret diagnostic port protocols, then they could continue repairing cars. If the manufacturer changes the protocol in their next model of cars then they need to send updated tooling to all their own dealerships or risk disrupting their own repairs. We’d develop a cat and mouse game between mechanics and manufacturers each year, hopefully with more positive outcomes for consumers.

Returning to the broader topic of social media, we can apply the same logic to Facebook. Rather than mandate that they open APIs to the public, we would be free to reverse-engineer the Facebook mobile apps and web-interface, uncover the undocumented, proprietary APIs, and use them in third party software. This would allow the kind of content-export we’re familiar with from email, contact-syncing with other platforms, even message bridges between Facebook and other social media.

Limitations of Federation

Pivoting topics, let’s talk about some limitations of federated servers that we might link to existing networks like Facebook.

Duplication of labor is hard: if different servers have different content policies then they cannot inherit moderation decisions from one another, and must each moderate the same messages. This is especially difficult in multi-lingual scenarios, where moderators must translate and then moderate messages, and can easily miss cultural subtleties. This is less of an issue in smaller and more purpose-specific communities, which can have specifically tailored content policies with more room for context and less ambiguity.

We often frame censorship or moderation in terms of free speech, and leave out the importance of freedom of assembly. Echo chambers aren’t just where hatespeech and radicalization take place; they’re also where we organize BLM, and LGBTQ+ spaces.

Conclusion

This was a good session for broadening horizons beyond either “reign in Big Tech” or “reject Big Tech entirely and build all-new platforms”. These are good ideas for piggy-backing off of existing large networks to boot up better solutions. There’s also good perspective here on what we’re losing with decentralization. Moderation and scale are hard, and this further amplifies previous discussions on the need for a plethora of small purposeful communities over large homogeneous ones. Click here for my notes from day 3.

Posted 5/12/2021


Reimagine the Internet Day 2: Misinformation, Disinformation, and Media Literacy in a Less-Centralized Social Media Universe

This week I’m attending the Reimagine the Internet mini-conference, a small and mostly academic discussion about decentralization from a corporate controlled Internet to realize a more socially positive network. This post is a collection of my notes from the second day of talks, following my previous post.

Today’s session was mostly on cultural patterns that lead to susceptibility to misinformation and conspiratorial thinking.

Scriptural Inference

I was not raised religiously, and rarely consider religion when analyzing online communities. One of today’s speakers is an anthropologist that’s spent extensive time in southern conservative groups, which include a very high number of practicing Protestants. They drew very direct comparisons between Protestant behavior - namely personal reading, interpretation, and discussion of the Christian Bible - with conservative political practices, especially with regards to:

  • Close readings and interpretation of original documents (the constitution, federalist papers, tax law, Trump speech transcripts) over “expert contextual analysis”

  • A preference for personal research over third party expertise or authority, in almost all contexts

This aligns with constitutional literalism, a mistrust of journalists, of academics, of climate science, of mask and vaccine science, of… a lot. It’s a compelling argument. There are also very direct comparisons to conspiracy groups like QAnon, which feature slogans like “do the research.”

It’s also an uncomfortable argument, because mistrust of authority is so often a good thing. Doctors regularly misdiagnose women’s health, or people of color. There’s the Tuskegee Study. There’s an immense pattern of police violence after generations of white parents telling their children the cops are there to protect everyone. There are countless reasons to mistrust the government.

But, at the same time, expertise and experience are valuable. Doctors have spent years at medical school, lawyers at law school, scientists at academic research, and they do generally know more than you about their field. How do we reconcile a healthy mistrust of authority and embracing the value of expertise?

One of the speakers recommended that experts help the public “do their own research”. For example, doctors could give their skeptical patients a list of vocabulary terms to search for material on their doctor’s diagnosis and recommended treatment, pointing them towards reputable material and away from misinformation. I love the idea of making academia and other expert fields more publicly accessible, but making all that information interpretable without years of training, and training field experts like doctors to communicate those findings to the public, is a daunting task.

Search Engine Reinforcement

For all the discussion of “doing your own research” in the previous section, the depth of research in many of these communities is shallow, limited to “I checked the first three to five results from a Google search and treated that as consensus on the topic.”

Of course, Google results are not consensus, but are directed both by keyword choice (“illegal aliens California” and “undocumented immigrants California” return quite different results despite having nominally the same meaning), and past searches and clicks to determine “relevancy”. This works well for helping refine a search to topics you’re interested in, but also massively inflates confirmation bias.

Google Knowledge Graph

Google knowledge graphs like the one above change a Google result from “returning documents” to “returning information”, blurring the source and legitimacy of information, and further partitioning the web into distinct information spheres, where people can easily find “facts” supporting their existing positions.

Data Voids

“Data voids” are keywords with very few or no existing search results. Because search engines will always try to find the most relevant results, even if the signal is poor, it’s relatively easy to create many documents with these “void keywords” and quickly top the search results. This makes it easy to create an artificial consensus on an obscure topic, then direct people to those keywords to “learn more”. A pretty simple propaganda technique utilizing a weakness of search engines.

Generational Differences

The speakers ended with a discussion on generational differences, especially in how media literacy is taught. Older generations had “trusted sources” that they went to for all news. Students were often told to “seek .org and .edu sites for trustworthy citations”, or before then were told that nothing on the web could be trusted, and print media and academic journals were the only reliable sources. Obviously these are all outdated notions; anyone can register a .org domain, there’s plenty of misinformation in print media, traditionally “trustworthy” sources often fall sway to stories laundered through intermediary publications until they appear legitimate. The “post-social-media youth” see all news sources as untrustworthy, emphasizing a “do your own research” mentality.

Conclusion

I really like this framing of “institutional trust” versus “personal experience and research”. It adds more nuance to misinformation than “online echo-chambers foster disinfo”, or “some communities of people are uneducated”, or “some people are racist and selectively consume media to confirm their biases.” Some people are not confident in their beliefs until they have done research for themselves, and we’ve created a search engine and echo-chamber system that makes it very easy to find reinforcing material and mistake it for consensus, and there are people maliciously taking advantage of this system to promote misinformation for their political gain. Click here for my notes from day 3.

Posted 5/11/2021


Reimagine the Internet Day 1: Pioneering Alternative Models for Community on the Internet

This week I’m attending the Reimagine the Internet mini-conference, a small and mostly academic discussion about decentralization from a corporate controlled Internet to realize a more socially positive network. This post is a collection of my notes from the first day of talks. The full recording is available on the Reimagine the Internet site.

Great summary quote from Ethan Zuckerman: “It’s not too late to fix things, but we have to stop fixing what we have now and imagine what is possible.”

What We’re Rejecting

Large scale networks usually follow one of two models:

  1. Surveillance and Advertising: Social media, generally any “free” services, which sell user data to advertisers for profit

  2. Transactional Sellers and Buyers: Amazon, eBay, Patreon, any service that directly sells a product, or facilitates explicit financial transactions where the platform takes a cut

Large scale networks follow these models not only because they’re financial models that can scale, but because market forces demand scale. Public for-profit corporations are by definition growth-oriented, but there are stable non-growth models outside this dichotomy.

A great example is Wikipedia: they’re donation driven, but mostly sustained by small non-ideological donations. People find Wikipedia valuable, for their homework or quick-references for questions in any context, and they’re willing to throw a few dollars the encyclopedia’s way to keep that product available to them. In this sense Wikipedia is extremely product-centric - their “growth” (mostly adding new languages and outreach in poorer, less Internet-dense countries) does not earn them profit, and is subsidized by providing a useful, slick product to their English-speaking base.

Small Communities

Facebook is ungovernable because it is not a community, it’s millions of communities bumping into each other with incompatible needs. Reaching consensus on content policy for three billion users is impossible, and a foolhardy goal.

Creating the largest community possible should rarely be the goal. Instead, we should create countless small communities, with their own content policies and standards of acceptable behavior. Users are in many such communities concurrently; we’re good at context-switching. Reddit and Discord are great examples of this: larger subreddits and discord servers are rife with abuse, but there are countless micro-communities with every imaginable focus that thrive and are generally respectful and well moderated.

Purpose-based networks are especially successful: Networks where users are all gathered for a specific shared purpose. Social networks are hard to design because many people will have diverging opinions on what constitutes “social behavior”. Wikipedia’s mission of “making an encyclopedia” is much more clear-cut. They have disagreements over policy, but everyone in the project is generally there for the same reason, and that agreement has made it easier to govern and scale.

The Fallacy of “Free and Open”

A lot of early Internet action-groups, including Wikipedia and the EFF, were based around a cyber-libertarian concept of “free and open communication”. Basically the ultimate expression of the “marketplace of ideas” - you can communicate with anyone on the Internet, regardless of race, nationality, sex, or age, because no one can see what you look like, and the Internet knows no borders. The liberatory potential of “radical openness” is outlined in the Declaration of Independence of Cyberspace and The Conscience of a Hacker (also known as The Hacker Manifesto), and is echoed in countless other messages from that era.

This vision of utopia has fallen short. Erasing race and sex and age, has (surprise surprise!) not eliminated bigotry, nor has it consistently led to a brilliant meeting of the minds. In particular, “free and open” amplifies existing power structures: the loudest, most agreed with, most online voices tend to get their way. This tends to mean mainstream communities with enough financial security to spend lots of time online.

Community Norms

Maybe one third of community behavior or less is governed by explicit rules; most behavior is governed by unstated norms of what constitutes “respect” or “acceptable behavior or content.” As an example, there’s no law preventing a student from standing in the middle of class and debating a professor over each point of their lecture, but this is generally unacceptable given the roles or student and educator, and is highly unusual.

When building platforms, we should keep in mind what rules and norms we expect our community to abide by, and design features with those norms in mind. Returning to the previous example, a web conferencing system for education may allow the lecturer to mute all students while they are presenting, requiring students to raise their hands before speaking. This reinforces social norms and makes it much more difficult to misbehave within the confines of the platform.

Reinforcing social norms goes a long way towards community self-governance, limiting the need for explicit moderation, and making a community more scalable and less chaotic and disrespectful.

Coexistence with the Corporate Net

Alternative networks are often presented in opposition to the corporate dominated “main” Internet, but this doesn’t mean that Twitter and Facebook need to burn for better communities to thrive. In fact, existing mainstream networks can be a valuable bootstrapping system to spread broad messages and find peers to join micro-communities. Alt-nets don’t need to be completely self-sustaining or discoverable at first.

Conclusion

A very exciting first day! While the talk was non-technical, I think bringing these ideas of small-scale, self-governance, norm-driven, mainstream-bootstrapping communities to peer-to-peer network design will be rewarding. Click here for my notes from day 2.

Posted 5/10/2021


View older posts