Abstract
The increasing centralisation of large social media platforms under a few companies has created an expectation of opaque data practices, manipulative algorithms, and predatory policies. This centralisation is a product of the appeal of a single, unified interface for all interactions, however it can be another way.
This project proposes creating a unified social media platform (starting with a forum or reddit-like interface) hostable by any individual or organisation on decentralized but interconnected servers. By creating a flexible protocol for communication of actions and data between servers, users can maintain the benefits of a single platform while having the freedom to choose hosts that most benefits them - rather than hosts which have a monopoly on the market. Such an open and extensible protocol should also allow much easier interoperability of different styles of social media application, and make developing supporting tooling and performing analytics easier.
Author
Name: McArthur Alford
Student number: 46967307
Functionality
Frontend/User Experience
- Typical user flow:
- Sign up (with email verification), then log in (2FA available).
- Browse or search posts (no login needed). Allow for filtering of posts and sorting by various metrics (upvotes/downvotes, chronological, date ranges).
- If logged in: create posts, reply, up/downvote, follow accounts, manage account (password reset, server migration, deletion). A focus is extensibility, so this is not an exhaustive list.
- Markdown editor with preview for posts and replies.
- View replies, user profiles, and post histories.
- Frontend is to be independant of server, and communicate exclusively using an API (no coupling).
Communication Protocol
- Allows servers to share actions (replying, up/downvoting, posting, following, direct messaging, referencing, etc.).
- Supports user encryption for privacy.
- Extensible with feature flags: servers can skip actions they don’t wish to implement (e.g., following another user).
Server
- A reference server is provided, though hosts may use their own implementations.
- Server stores user accounts and locally produced content.
- Uses vector embeddings for easy content search.
- Can link to other servers (one-way link to subscribe/index remote posts for browsing, two-way link to allow inter-server actions such as replies or upvoting).
- Features a basic recommendation system based on post recency and up/downvotes, with filtering/sorting by date and approval.
Administration
- Additional capabilities for server administrators:
- Delete or edit posts.
- Ban users (temporary or permanent).
- Create priority posts (seen first).
- Fine-grained privilege controls.
Scope
- Communication Protocol
- Implement basic inter-server actions: posting, replying, up/downvoting.
- Simple Server Implementation
- Provide vector-based search.
- Offer an API for account management and the minimal set of actions.
- Allow linking to other servers (one-way or two-way).
- Sorting and filtering by date or popularity over a set period.
- Frontend
- Basic web interface where users can:
- Create accounts (email verification), log in (no 2FA in MVP).
- Browse posts with sorting/filters.
- Search by keywords.
- Create posts, up/downvote, and comment.
- View user histories.
- Administration
- Not part of the MVP. (Needed for real deployments but excluded for this demonstration.)
Quality Attributes
Extensibility
With multiple hosts offering different features, the protocol must flexibly handle unsupported actions or custom extensions. Hosts should be free to adopt or ignore new features without breaking compatibility. This means:
- The protocol remains understandable by all hosts even as it grows.
- Actions a host does not support can be gracefully blocked, filtered, or translated into a compatible format. Likely achieved with a great deal of metadata.
- Data representation and message content are decoupled, enabling diverse frontends.
Modularity and Interoperability
The degree of decentralization is best solved by breaking infrastructure into a variety of modules. All of these modules—server, frontend, and any additional modules—must be interchangeable and interopereable. Interoperability is achieved using a common, well-defined protocol and API so that:
- Hosts function independently (entirely decoupled) with no knowledge of the behaviour of other hosts yet still communicate effectively.
- Developers must be able to confidently update or replace modules without having to make any changes to the surrounding network.
- Frontend must be able to support arbitrary backends.
Reliability
A real reliability concern for a decentralized approach is desynchronisation of state between hosts. Such desynchronisation may occur if the receiver of an action is unreachable briefly, or actions are recieved in an incorrect order. To resolve this:
- Hosts should confirm acknowledgement of actions.
- Data should only be stored on a single “owner” host, preventing conflicting updates. Other hosts can cache or index that data, but retrieval or updates always route back to the owner.
- Actions should be timestamped and stored for some duration so that ordering issues may be resolved.
Scalability
Scalability is important for any social media site - traffic to individual servers may fluctuate heavily based on the time of day, over the year, or during certain events. Decentralization helps distribute the load, however that is not enough if one particular server is seeing spikes in usage. This should be solved using horizontal scaling for the reference server implementation.
Evaluation
The procedure described here can be re-used for testing of several quality attributes.
Testing will involve creating several instances
- Scalability
- Method: Evaluated using load testing. This involves sending increasing levels of traffic (up to some maximum greater than expected in practice in the worst case).
- Metrics: Response time, request throughput, cpu/memory usage, and error.
- Goal: To keep response time and request throughput within a comfortable range. Verify that high stress does not cause errors.
- Modularity and Interoperability:
- Method: Set up a small network of hosts with slight variations. Variations include a host sending/responding to messages incorrectly, supporting special functionality, or producing errors. Send a suite of automated actions (post creation, voting, etc) to the various hosts.
- Metrics: Compare the final state of all hosts with some expected final state for the given set of automated actions. Track mismatches and errors.
- Goal: To ensure that all modules individually interact in a consistent and predictable manner with no errors regardless of the implementation details of other modules. To ensure the API is used properly for communication and works in unexpected combinations.
- Reliability:
- Method: Repeat the process described in point 2. As part of this, shut down a host during testing, or prevent messages from receiving the destination. As part of the implementation process, manually verify that the server implementation only stores data local to itself (this can not easily be tested, but can be prevented if accounted for in development).
- Metrics: Compare the states of hosts.
- Goal: Ensure there is no lost or duplicatae data.
- Dogfooding and user testing may also be utilized to verify all behaviour is as expected in practice.
- Extensibility:
- Method: Attempt to extend the protocol for a single host in the testing network (e.g. introduce a new action). As part of developing the MVP protocol, add in each new action one at a time to ensure extensibility is on the mind of developers.
- Metrics: Verify correct error handling, contemplate the developer experience for extending and how it might be improved. This is a very subjective metric.
- Goal: Smooth out pain points for developing extensions. Ensure the protocol is easy to extend, and that the server implementation can correctly handle unsupported messages gracefully.