For context, Larian Studio founder Swen Vincke predicted that the game could reach 100,000 peak concurrent users during its debut period, and that was a fairly optimistic prediction. I work in IT, and really feel for those folks. I hope they designed their infrastructure to scale!
And that’s only Steam, not including GOG, or the influx of PS5 users next month. Let’s take it to 1 million!
The first thing that comes to mind is the sheer number of crash reports they were getting early on. I know I submitted probably two dozen myself those first few days. Game is rock solid for me now, though.
The second thing that comes to mind is that they are clearly sucking up a whole lot of data around gameplay. The stats they published on Friday about how people played opening weekend was eye opening to me regarding the amount of data they are ingesting: https://steamcommunity.com/games/1086940/announcements/detail/6199820457241938860
I feel attacked.
For what it’s worth - I work in the AAA game industry. Every AAA game collects those kinds of metrics, even “singleplayer only” ones.
They answer a few questions:
Are things too strong and the player is dying too much? Too weak and the majority of players breeze through difficult sections?
Are players getting lost/stuck? Are there softlocks or progression blockers?
Where do players stop playing the game? Can we work out why those areas are more likely to stop playing?
Are players going to certain areas? Are important areas being skipped? Are areas without a lot of content seeing more traffic than expected?
What kinds of builds are players trying? Why are they playing those builds? Do they change builds midway through? Why?
What NPCs are the players talking to? How do they interact with those NPCs?
What do gamers do over the course of the lifetime of a game? What do hardcore passionate fans like? What do casual players like?
These sorts of data points are really critical for things like future patches or DLC. They help point out the places in the game that need love and adjustment going forward. (For example, how many people romance Karlach vs. Shadowheart? How many people go for the Underdark vs. the Mountain Pass? How many people investigate the Creche vs. skip Gith stuff entirely?)
For example - in a game I worked on, we saw a spike in players quitting the game at the section where they needed to crouch for the first time. This meant either crouch wasn’t being taught well, crouch controls weren’t intuitive, or something broke. A subsequent patch changed the crouch tutorial and the spike of players quitting went away.
Every single moment that is spent working on the game has a cost. The job of production is to ensure that development resources are spent in places that give the biggest bang for the buck. Places like the tutorial or early missions are super duper hyper important because 100% of players will see them. As things diverge, fewer segments of players see them - some players quit entirely, while others commit to one section of content over the other. You want to figure out where “most” players go and focus there, only moving down once those areas are fixed.
For example, 1/3 of players choose evil. So 1/3 of the time should be spent on the evil path, because why invest 75% of resources into something that 66% of players won’t ever see? You still need something there, but when you need to make decisions, production (and publishers!) only drive decisions based on data.
Community feedback is biased. The loudest minority are the ones who choose to give feedback online. These are players who are highly invested in the game, much more than a “typical” player. But they represent a relatively small chunk of the player base - 5-10%, on the high end - and not a representative sample at that. While community feedback is important (and does drive many decisions, if only to make sure the Internet doesn’t get mad), it isn’t “good enough” for dev teams to focus on it exclusively. The numbers guys work in numbers, not sentiment.
So the data basically is used to say “Look: players are romancing XYZ more than we expected. We should invest more resources into making sure that their romance is rewarding” and if that’s true it’s likely that production will greenlight some dev time to work on that.
It can also be used for “Hey, players are seeing framerate drops in these areas, consistently. Can we have a patch that improves framerate here?” and production will balance that against everything else (how many more sales will be made by raising the quality bar and fixing the framerate in this spot?), create tickets, and triage them out for a future patch.
Then if Larian works on Baldur’s Gate 4, data from Baldur’s Gate 3 can help drive decisions. Should the dev team invest resources in a better character creator? What areas are really resonating with players - and how can a sequel replicate that? What areas were more disappointing than the dev team expected, and how can they be fixed? (For example, the devs seem surprised that the community has rejected Giths as much as they have - this isn’t too surprising from the outside looking in, but it’s easy to be blinded and lose sight of things like this when you work on a project for 4-5 years with dev goggles on.)
This is all super duper common data to collect, on any game. I wouldn’t read too much into it; Larian is just being more open than most by sharing selections of that data with the community.
Thank you for the insights. While I think I knew a lot of this on some level, it always helps to have it pointed out. I guess what I find enchanting is Larian’s willingness to share some of that data with us. It’s not something I have spent much time thinking about, and is much more Black Mirror than I assumed. A little scary, really. I thought it was just me by myself playing this game.
For what it’s worth - you are one of millions of entries in a server somewhere. It is extraordinarily unlikely that anyone would look at you in particular.
There’s a list of user IDs that devs can map to various tables. These tables can be device information (OS, GPU, CPU, RAM), graphics settings, framerate data, player character details, whatever. They would basically have to know what your player character’s name is and use that to find your unique ID, which would in turn be able to be used as a lookup for the other tables. (It’s possible that they also log your Steam ID or something but I’ve never seen that happen. Certainly possible though.)
More likely, unless you did something that made you stand out (like you are an obvious cheater, or you were the only player in the whole world to do something, which happens) your data just gets put in a bunch of graphs and averaged out. Then some data guy gives a presentation in a boring meeting where the graphs are presented and analyzed. Conclusions are drawn and then producers chase you down to fix the things based on those conclusions. Rinse and repeat.
There are times where I would find someone doing something that should be impossible, and then I poke at the data to figure out why. Usually that gives me an idea on where a bug could be. But you’re one entry among millions; it’s statistically extremely unlikely that you are the standout that gets analyzed - and if you are, it’s not necessarily the case that they dig deep into everything you’ve ever done.
You can pretty much assume this about every game these days, by the way. Singleplayer, multiplayer… if a AAA publisher is involved, they’ll all do it. Starfield will certainly do it too.
You can probably use Wireshark to peek at the data being sent, if you’re really curious. Multiplayer games I worked on did this stuff server-side when possible to avoid players using Wireshark to look at everything, but singleplayer games don’t have that luxury. I doubt they encrypt the data (but it’s possible).
“you are one of millions of entries in a server somewhere” … that makes me feel better. /s
Have you watched DEVS? This feels a lot like that.
That was a great read, thank you for posting this. As a kid I always wanted to get into game dev but never managed to (I’m still a dev, but not in the gaming industry since it hasn’t thrived where I live). So it was very interesting to see the inner workings of taking decisions in a game dev sense.
Larian also shared an infographic detailing that data collection (e.g. they posted how many players have pet a dog in total), so it makes sense they have a ton of other data collection.