I’ve recently been wondering if Lemmy should switch out NGINX for Caddy, while I hadn’t had experience with Caddy it looks like a great & fast alternative, What do you all think?
EDIT: I meant beehaw not Lemmy as a whole
I’ve recently been wondering if Lemmy should switch out NGINX for Caddy, while I hadn’t had experience with Caddy it looks like a great & fast alternative, What do you all think?
EDIT: I meant beehaw not Lemmy as a whole
Hm, that’s an interesting take. To be quite honest I saw issues with diesel-rs in production on another website I was contributing too, maybe it’s the issue?
I doubt it is anything that level. The problem is the data itself, in the datababase.
A reddit-like website is like email, every load from the database has unique content. You really have to be very careful when designing for scalability when almost all the data is unique. Especially in modern times where users block other users, and even 2 people loading the same posting do not get the same comments. It’s anti-cache, and you have to really work hard to design that to run efficiently on small servers.
As opposed to a website like Amazon where the listing for a toothbrush is not unique on every page load. There aren’t new comments and new votes altering the toothbrush listing every time a user refreshes the page. And people aren’t switching brands of toothbrush every 24 hours like the front page of Reddit abandons old data and starts with fresh data.
Would a good solution be to just deffer changes to data with something like Apache Kafka? Or changing to something that can be scaled, like cockroach db or neondb? I also heard ScyllaDB could be a great alternative, mostly from reading the discord technical blog.
Not that I see. A database like PostgreSQL can work, but you have to be really careful how new data flows into the database. As writing to the database involves record locking and invalidates the cache for output.
Taking the bulk data, comments and postings, outside PostgreSQL would help. Especially since what most people are reading on a Reddit-like website is content form the last 48 hours… and your caching potential dies way down as people move on to the newer content.
The comments alone are the primary problem, there are lot of them on each posting and they are bulky data. Also comments are unique data.
hmmm a good approach would be to maybe split comments into some kind of database regions and just load as they’re needed instead of loading them all at once
It’s not the tech here. Postgres can scale both vertically and horizontally (yes there are others that can scale easier or in different factors of CAP).
The problem is how the data is being stored and accessed. Lemmy is doing some really inefficient data access and it’s causing bottlenecks under load.
Lemmy (unfortunately) just wasn’t ready for this level of primetime yet… It has a number of issues that are going to be quite tricky to fix now that it’s seen such wide adoption (database migrations are tricky on their own, doing so on a production site even harder, doing so on 8k+ independent production sites… Sounds like a nightmare)
Can you elaborate on what Lemmy is doing that’s inefficient? I’m working on a database application myself, so the more I know about optimizing database queries, the better.
Sorry, I assumed it was just an issue with the tech not scaling well, really shows how little I know about architecture haha.