Ask HN: Columnar Db for Leaderboard Stats?

1 point by JoeOfTexas 10 months ago | 3 comments
Hey all, I'm trying to figure out the best way to build thousands of leaderboards for thousands of different statistics scoped to different time windows. I need to be able to calculate a players rank in the leaderboard as well as the surrounding players.

I know redis can do this using zset and zrevrange using a new key for each stat/timeframe, but it doesn't easily allow referencing the player's full metadata like name, country, and other default stats like wins/loss/score. I would have to store those separately and retrieve them each time I pull the top X players in the leaderboard.

If I pull top 100 players in a leaderboard, that is 100 player metadata retrievals using some identifier like player id, and it doesn't scale well.

I'm looking for something that will allow me to pull the top 100 player id/score AND the player metadata in one step, without having to store the metadata within the leaderboard itself.

Any ideas?

  • davidw 10 months ago
    1000's sounds like it's still Postgres territory?
    • JoeOfTexas 10 months ago
      I don't think relational databases can handle sorted sets optimally. Most of online tutorials for leaderboards lean towards using redis.
      • stop50 10 months ago
        yes that is still the territory of an database. Someone who knows SQL can do some stuff that reduces the strain on the database a lot.