Envy ~ Distributed Habbo Retros

[Leader]

Envy

Introduction:
Throughout my career, I've worked on several projects focused on creating a suite of libraries to streamline the development of Habbo retro applications. Along the way, I've accumulated invaluable insights and started contemplating the future of retro architecture—specifically, how to resolve persistent issues from the past.

Initially, retros were developed with the straightforward goal of reverse-engineering Habbo. Developers didn't fully consider the long-term implications or how to make these projects scalable. As a result, hundreds of emulators have been built, with only a few gaining significant community traction. Most, however, suffer from the same core problem: monolithic architecture.

MMOs, such as Habbo, are designed to handle thousands, if not millions, of concurrent users. While modern retros don't necessarily aim for those numbers, the consequences of monolithic design quickly surface. The retro community remains restricted by outdated architectural decisions, with development often shaped by constraints like the revision, CMS, HTML5, app, or SWF. A well-designed emulator shouldn't be hindered by these variables.

The solution is distributed architecture based on microservices. With a modern stack you get a system that is scalable, resilient, and performant.

---

Project Goals:

• Develop a distributed architecture capable of scaling to millions of users.
• Provide common types and libraries for both front-end and back-end.
• Provide an external GraphQL Websocket API for CMS and plugin implementations to support real-time updates.
• Use an internal Kafka messaging system for efficient microservice communication.
• Ensure strict separation of concerns in the codebase.
• Enable database distribution for optimal scaling.

---

Scope Goals:

• Provide the essential foundation to help you implement your preferred retro revision
• Accomplish this without dictating how users connect or interact with the system.
• Scale Up will provide its events and API calls, which can be integrated into your business logic or packet structure.
• Offer a full implementation example using Nitro.

Tech Stack:

• Golang: High-performance, efficient language for building scalable microservices.
• Kafka: Distributed messaging system for reliable, asynchronous communication between services.
• Protobuff: Lightweight and efficient serialization format for structured data transmission.
• Postgres: Robust, relational database for storing structured data and ensuring ACID compliance.
• MongoDB: NoSQL database for flexible, unstructured or semi-structured data storage.
• Redis: In-memory data structure store used for caching and real-time performance.
• GraphQL: Query language for APIs, enabling flexible and efficient client-server interactions.
• WebSockets: Protocol for real-time, bi-directional communication, critical for live updates.

Architecture:

• Internal-Facing Services: All microservices are designed to be internal, with no direct external API exposure.
• Backend for Frontend (BFF): External interactions, whether from the CMS or the game client, are routed through a BFF layer. This isolates the core services and optimizes interactions tailored specifically for client needs, ensuring security and performance.

 

[webbanditten]

Hey @Leader,

I'm amazed to see some innovation here on this forum. A step toward distributed architecture could be a real game-changer. A few quick thoughts:

1. Skill Transition: A guideline for developers to smoothly shift their skills from monolithic to microservices would be great. We're all ready to learn and adapt!

2. Testing & Debugging: Could you share any plans for testing and debugging tools? These will be vital in a distributed environment.

3. Inter-service Communication: NATS API for microservice communication sounds interesting. It'd be great to hear how potential network latency and data consistency issues would be managed.

4. Database Management: How does the project plan to handle distributed database management challenges, like transactions and network failures?

5. Infrastructure Cost: Any suggestions for managing costs in cloud environments like AWS? I would love this to be accessible to all!

6. Security: Security is crucial with multiple microservices. Excited to learn more about the integrated security measures.

7. Project Release: I understand your approach to release stable code, but opening up early for community contribution could speed up the process and bring in fresh ideas.

 

[Leader]

Hey @Leader,

I'm amazed to see some innovation here on this forum. A step toward distributed architecture could be a real game-changer. A few quick thoughts:

1. Skill Transition: A guideline for developers to smoothly shift their skills from monolithic to microservices would be great. We're all ready to learn and adapt!

2. Testing & Debugging: Could you share any plans for testing and debugging tools? These will be vital in a distributed environment.

3. Inter-service Communication: NATS API for microservice communication sounds interesting. It'd be great to hear how potential network latency and data consistency issues would be managed.

4. Database Management: How does the project plan to handle distributed database management challenges, like transactions and network failures?

5. Infrastructure Cost: Any suggestions for managing costs in cloud environments like AWS? I would love this to be accessible to all!

6. Security: Security is crucial with multiple microservices. Excited to learn more about the integrated security measures.

7. Project Release: I understand your approach to release stable code, but opening up early for community contribution could speed up the process and bring in fresh ideas.

1. Skill Transition: There shouldn't be too much of a difficulty assuming the developers are competent and can follow documentation or understand libraries. Every service I create will expose an internal API with a client lib for other services to use when connecting to it. This allows their interaction to remain type protected and type documented so it's easy for developers to know what they can interact with

2. Testing & Debugging: Tools may be developed as I develop this out. My primary goal for local development is using pm2 for running all apps/services locally and restarting as needed. Long term, I hope to have hot reload capability on the backend services. The frontend is already hot reload capable including library integrations.

3. Inter-service Communication: Assuming the infrastructure is correct in production and all services are on a private network with exposure only where necessary and behind a proxy or gateway - latency should be fairly low and you can setup services to be geolocated in addition. NATS has queue groups allowing for events to be processed once and by a single subscriber. I may also add Kafka for the actual event streams. NATS is primarily for request->response on the internal communication. Locally, I haven't had any issues with latency and it performs similar to monoliths even with 3+ service calls chained

4. Database Management: Every service is directly responsible for it's own data and will use ids for interacting with other services. With this approach, services are simpler and easier to manage. Their databases don't reference external properties aside from a name and key. Feature services will be used for situations where multiple services are chained together and will be responsible for managing the data and ensuring it's success at each step. Databases are setup to be completely independent of one another and have no connections to each other. I use Postgres as well which provides

You must be registered for see links

5. . Infrastructure Cost: Hotels can run this on a single server or hundreds of servers. Hotels would scale their cost to their demand but overall it should use lower resources than a typical setup by being able to fine tune where demand comes from.

6. Security: Services have multiple layers of APIs. Internal APIs should never be exposed on a port and are expected to be ran behind an internal network only. There is no inherit security outside the internal network, although I may add authentication keys for internal APIs in the future. External APIs have full authentication and authorization protections and can be scoped to the feature level. Currently, I use JWT bearer tokens for external APIs and have additional checks by the session service.

7. Project Release: If people could contribute to this project, they would have made their own version. The earliest time at which the source would be shared is when I believe it's capable of reproducing the core retro features.

I haven't worked on this much in the past few weeks - but the core of the project has been built out including everything I described above outside the retro implementation.

 

[TheGeneral]

This is Habbo not facebook.

 

[Leader]

This is Habbo not facebook.

You must be registered for see images attach

 

[TheGeneral]

You must be registered for see images attach

I actually removed it

 

[Leader]

I actually removed it

Thought it was too hard and gave up?

 

[TheGeneral]

Thought it was too hard and gave up?

No, I just never used it in the past 2 years.

 

[Leader]

I got busy with other projects but recently started working on this again, it's going pretty well and I have a reliable architecture and foundation to safely build features on top of.

Working on it with the intent to use it on my own roleplay, but not sure how tangible that will be in the near future as I'm just now starting on handshake

 

[Leader]

This project is coming back and will be developed using Go. I do not anticipate a quick release nor am I in a rush