I’ve been writing about the early development of Warcraft, but a recent blog post I read prompted me to start scribbling furiously, and the result is this three-part, twenty-plus page article about the development of StarCraft, along with my thoughts about writing more reliable game code. I’ll be posting the latter parts over the next several days.
- This post: Why StarCraft crashed frequently during development
- Part 2: How we could have fixed the most common causes
- Part 3: Explaining the implementation details of the fix
The beginnings of StarCraft
During the development of StarCraft, a two and a half year slog with over a year of crunch time prior to launch, the game was as buggy as a termite nest. While its predecessors (Warcraft I and II) were far more reliable games than their industry peers, StarCraft crashed frequently enough that play-testing was difficult right up until release, and the game continued to require ongoing patching efforts post-launch.
Why? There were sooooo many reasons.
Orcs in space
StarCraft was originally envisioned as a game with modest goals that could fit into a one-year development cycle so that it could be released for Christmas, 1996.
The project leadership was comprised of the same folks who had started Shattered Nations (video), a turn-based strategy game along the lines of X-COM that Blizzard announced in May 1995 but canceled some months later.
The team members were regrouped to build something that could reach market quickly so Blizzard wouldn’t have a long gap between game launches.
- Q4 1994 – Warcraft
- Q4 1995 – Warcraft II
- Q4 1996 – planned ship date for StarCraft
- Q2 1998 – actual ship date for StarCraft
The decision to rush the game’s development seems ludicrous in retrospect, but Allen Adham, the company’s president, was under pressure to grow revenue. While Blizzard’s early games had been far more successful than expected, that just raised expectations for future growth.
Given a short timeframe and limited staff, the StarCraft team’s goal was to implement a modest game — something that could best be described as “Orcs in space”. A picture from around the time of the E3 game show in Q2 1996 shows the path the game team originally chose:
StarCraft as it appeared in June 1996 at the Electronic Entertainment Expo. Yeah — I wouldn’t play it either.
But a higher priority project overshadowed StarCraft and stole its developers one by one. Diablo, a role-playing game being developed by Condor Studios in Redwood City California, was in need of additional help. Condor, a company formed by Dave Brevik along with Max Schaefer and his brother Erich Schaefer, was given a budget of only $1.2 million — ridiculously small even in those days.
The Condor team had no hope of making the game they aspired to build, but they did such ground-breaking work in developing something fun that it made sense for Blizzard to acquire Condor, rename it Blizzard North, and start pouring in the money and staff the game really deserved.
Initially Collin Murray, a programmer on StarCraft, and I flew to Redwood City to help, while other developers at Blizzard “HQ” in Irvine California worked on network “providers” for battle.net, modem and LAN games as well as the user-interface screens (known as “glue screens” at Blizzard) that performed character creation, game joining, and other meta-game functions.
As Diablo grew in scope eventually everyone at Blizzard HQ — artists, programmers, designers, sound engineers, testers — worked on the game until StarCraft had no one left working on the project. Even the project lead was co-opted to finish the game installer that I had half-written but was too busy to complete.
After the launch of Diablo at the end of 1996, StarCraft development was restarted, and everyone got a chance to see where the game was headed, and it wasn’t pretty. The game was dated, and not even remotely impressive, particularly compared to projects like Dominion Storm, which looked great in demos at E3 six months before.
The massive success of Diablo reset expectations about what Blizzard should strive for: StarCraft became the game that defined Blizzard’s strategy of not releasing games until they were ready. But a lot of pain had to occur along the way to prove out this strategy.
Something to prove
With everyone looking critically at StarCraft, it was clear that the project needed to be vastly more ambitious than our previous ground-breaking efforts in defining the future of the real-time strategy (RTS) genre with the first two Warcraft games.
At the time of the StarCraft reboot, according to Johnny Wilson, then Editor in Chief of Computer Gaming World, the largest-distribution gaming magazine of that time, there were over eighty (80!!) RTS games in development. With so many competitors on our heels, including Westwood Studios, the company that originated the modern RTS play-style, we needed to make something that kicked ass.
And we were no longer an underdog; with the successes of Warcraft and Diablo continuing to fill the news we sure wouldn’t be getting any slack from players or the gaming press. In the gaming world you’re only ever as good as your last game. We needed to go far beyond what we’d done previously, and that required taking risks.
New faces
Warcraft II had only six core programmers and two support programmers; that was too few for the larger scope of StarCraft, so the dev team grew to include a cadre of new and untested game programmers who needed to learn how to write game code without much mentoring.
Our programming leadership was weak: we hadn’t yet learned how essential it is to provide guidance to less experienced developers early in the project so they learn much-needed lessons before the game launches, so it was very much a sink-or-swim proposition for new Padawans. A big part of the problem was just how thin we were on the ground — every programmer was coding like mad to meet goals, with no time for reviews, code-audits, or training.
And not only were there inexperienced junior members on the team, the leader of the StarCraft programming effort had never architected a shipping game engine. Bob Fitch had been programming games for several years with great results but his previous efforts were game ports, where he worked within an existing engine, and feature programming for Warcraft I and II, which didn’t require large-scale engine design. And while he had experience as the tech lead for Shattered Nations, that project was canceled, therefore no validation of its architectural decisions was possible.
The team was incredibly invested in the project, and put in unheard of efforts to complete the project while sacrificing personal health and family life. I’ve never been on a project where every member worked so fiercely. But several key coding decisions in the project, which I’ll detail presently, would haunt the programming team for the remainder of the project.
Some things have changed
After spending months working to launch Diablo, and further months of cleanup effort and patching afterwards, I returned to help with the reboot of StarCraft. I wasn’t looking forward to diving into another bug-fest, but that’s exactly what happened.
I thought it would be easy to jump back into the project because I knew the Warcraft code so well — I’d literally worked on every component. I was instead terrified to discover that many components of the engine had been thrown away and partially rewritten.
The game’s unit classes were in the process of being rewritten from scratch, and the unit dispatcher had been thrown out. The dispatcher is the mechanism I created to ensure that each game unit gets time to plan what it wants to do. Each unit periodically asks: “what should I do now that I finished my current behavior?”, “should I re-evaluate the path to get where I’m going?”, “is there a better unit to attack instead of the one that I’m targeting now?”, “did the user give me a new command?”, “I’m dead, how do I clean up after myself?”, and so forth.
There are good reasons code needs to be rewritten, but excising old code comes with risks as well. Joel Spolsky said it most eloquently in Things You Should Never Do, Part I:
It’s important to remember that when you start from scratch there is absolutely no reason to believe that you are going to do a better job than you did the first time. First of all, you probably don’t even have the same programming team that worked on version one, so you don’t actually have “more experience”. You’re just going to make most of the old mistakes again, and introduce some new problems that weren’t in the original version.
The Warcraft engine had taken months of programming effort to get right, and while it needed rework for new gameplay features, a fresh programming team was now going to spend a great deal of time relearning lessons about how and why the engine was architected the way it was in the first place.
Game engine architecture
I wrote the original Warcraft engine for Microsoft DOS in C using the Watcom Compiler. With the switch to releasing on Microsoft Windows, Bob chose to use the Visual Studio compiler and re-architected the game engine in C++. Both were reasonable choices but for the fact that — at that point — few developers on the team had experience with the language and more especially with its many pitfalls.
Though C++ has strengths it is easy to misuse. As Bjarne Stroustrup, the language’s creator, so famously said: “C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off.”
History tells us that programmers feel compelled to try every feature of their new language during the first project, and so it was with class inheritance in StarCraft. Experienced programmers will shudder when seeing the inheritance chain that was designed for the game’s units:
CUnit < CDoodad < CFlingy < CThingy
CThingy objects were sprites that could appear anywhere on the game map, but didn’t move or have behaviors, while CFlingys were used for creating particles; when an explosion occurred several of them would spin off in random directions. CDoodad — after 14 years I think this is the class name — was an uninstantiated class that nevertheless had important behaviors required for proper functioning of derived classes. And CUnit was layered on top of that. The behavior of units was scattered all throughout these various modules, and it required an understanding of each class to be able to accomplish anything.
And beyond the horror of the class hierarchy, the CUnit class itself was an unholy mess defined across multiple header files:
class CUnit ... {
#include "header_1.h"
#include "header_2.h"
#include "header_3.h"
#include "header_4.h"
};
Each of those headers was several hundred lines, leading to an overall class definition that could at best be called amusing.
It wasn’t until many years later that the mantra “favor composition over inheritance” gained credence among programmer-kind, but those who worked on StarCraft learned the hard way much earlier.
We’re only two months from launch
With its troubled early history, after the reboot the development team was pressured to finish up, and so schedules were bandied about that showed the game could be launched in two months.
Given the number of game units and behaviors that needed to be added, the changes necessary to switch from top-down to isometric artwork, a completely new map editor, and the addition of Internet play over battle.net, it was inconceivable that the game actually could ship in that time, even assuming that the art team, designers, sound engineers, game-balancers and testers could finish their end of the bargain. But the programming team continually worked towards shipping in only two months for the next fourteen months!
The entire team worked long hours, with Bob working stretches of 40 hours, 42 hours, even 48 hours programming. As I recall no one else attempted these sorts of masochistic endeavors, though everyone was putting in massive, ridiculous hours.
My experiences developing Warcraft, with frequent all-nighters coding, and later Diablo, where I coded fourteen-plus hour days seven days a week for weeks at a time, suffered me to learn that there wasn’t any point in all-nighters. Any code submissions [ha! what an appropriate word] written after a certain point in the evening would only be regretted and rewritten in the clear light of following days.
Working these long hours made people groggy, and that’s bad when trying to accomplish knowledge-based tasks requiring an excess of creativity, so there should have been no surprises about the number of mistakes, misfeatures and outright bugs.
Incidentally, these sorts of crazy hours weren’t mandated — it was just the kind of stuff we did because we wanted to make great games. In retrospect it was foolish — we could have done better work with more reasonable efforts.
One of my proudest accomplishments was to ship four Guild Wars campaigns in a two-year window without leading the development team down that dark path.
The most common cause of StarCraft game crashes
While I implemented some important features in StarCraft, including fog-of-war, line-of-sight, flying unit pathing-repulsion, voice-chat, AI reinforcement points, and others, my primary job gravitated to fixing bugs.
Wait: voice-chat! In 1998?!? Yeah: I had it all working in December 1997. I used a 3rd-party voice-to-phoneme compressor, and wrote the code to send the phonemes across the network, decompress them, and then play them back on the other seven players’ computers.
But every single sound-card in our offices required a driver upgrade to make it work, if the sound card was even capable of full-duplex sound (simultaneous recording and playback of sounds), so I regretfully made the recommendation to scrap the idea. The tech-support burden would have been so high that we would have spent more money on game support than we would have made selling the game.
So anyway I fixed lots of bugs. Some of my own, sure, but mostly the elusive bugs written by other tired programmers. One of the best compliments I’ve received came just a few months ago, when Brian Fitzgerald, one of two best programmers I’ve had occasion to work with, mentioned a code-review of StarCraft; they were blown away by how many changes and fixes I had made over the entire code-base. At least I got some credit for the effort, if only well after the fact!
Given all the issues working against the team, you might think it was hard to identify a single large source of bugs, but based on my experiences the biggest problems in StarCraft related to the use of doubly-linked linked lists.
Linked lists were used extensively in the engine to track units with shared behavior. With twice the number of units of its predecessor — StarCraft had a maximum of 1600, up from 800 in Warcraft 2 — it became essential to optimize the search for units of specific types by keeping them linked together in lists.
Recalling from distant memory, there were lists for each player’s units and buildings, lists for each player’s “power-generating” buildings, a list for each Carrier’s fighter drones, and many many others.
All of these lists were doubly-linked to make it possible to add and remove elements from the list in constant time — O(1) — without the necessity to traverse the list looking for the element to remove — O(N).
Unfortunately, each list was “hand-maintained” — there were no shared functions to link and unlink elements from these lists; programmers just manually inlined the link and unlink behavior anywhere it was required. And hand-rolled code is far more error-prone than simply using a routine that’s already been debugged.
Some of the link fields were shared among several lists, so it was necessary to know exactly which list an object was linked into in order to safely unlink. And some link fields were even stored in C unions with other data types to keep memory utilization to a minimum.
So the game would blow up all the time. All the time.
But why did you do it that way?
Tragically, there was no need for these linked-list problems to exist. Mike O’Brien, who, along with Jeff Strain, cofounded ArenaNet with me, wrote a library called Storm.DLL, which shipped with Diablo. Among its many features, storm contained an excellent implementation of doubly-linked lists using templates in C++.
During the initial development of StarCraft, that library was used. But early in the development the team ripped out the code and hand-rolled the linked-lists, specifically to make writing save-game files easier.
Let me talk about save games for a second to make this all clearer.
Save games
Many games that I played before developing Warcraft had crappy save-game functionality. Gamers who played any game created by Origin will remember how looooooong it took to write save-game files. I mean sure, they were written by slow microprocessors onto hard-drives that — by today’s standards — are as different as tricycles and race cars. But there was no reason for them to suck, and I was determined that Warcraft wouldn’t have those problems.
So Warcraft did some tricks to enable it to write large memory blocks to disk in one chunk instead of meandering through memory writing a bit here and there. The entire unit array (600 units times a few hundred bytes per unit) could be written to disk in one chunk. And all non-pointer-based global variables could similarly be written in one chunk, as could each of the game-terrain and fog-of-war maps.
But oddly enough, this ability to write the units to disk in one chunk wasn’t essential to the speed of writing save game files, though it did drastically simplify the code. But it worked primarily because Warcraft units didn’t contain “pointer” data.
StarCraft units, which as mentioned previously contained scads of pointers in the fields for linked lists, was an entirely different beast. It was necessary to fixup all the link pointers (taking special care of unioned pointer fields) so that all 1600 units could be written at once. And then unfixup the link pointers to keep playing. Yuck.
Change back!
So after fixing many, many linked list bugs, I argued vehemently that we should switch back to using Storm’s linked lists, even if that made the save-game code more complicated. When I say “argued vehemently”, I should mention that was more or less the only way we knew how to argue at Blizzard — with our youthful brashness and arrogant hubris, there was no argument that wasn’t vehement unless it was what was for lunch that day, which no one much wanted to decide.
I didn’t win that argument. Since we were only “two months” from shipping, making changes to the engine for the better was regularly passed over for band-aiding existing but sub-optimal solutions, which led to many months of suffering, so much that it affected my approach to coding (for the better) ever since, which is what I’ll discuss in part two of this article.
More Band-Aids: path-finding in StarCraft
I wanted to mention one more example of patching over bugs instead of fixing the underlying problem: when StarCraft switched from top-down artwork to isometric artwork, the background tile-graphics rendering engine, which dated back to code I had written in 1993/4, was left unchanged.
Rendering isometric-looking tiles using a square tile engine isn’t hard, though there are difficulties in getting things like map-editors to work properly because laying down one map tile on another requires many “edge fixups” since the map editor is trying to place diagonally-shaped images drawn in square tiles.
While rendering isn’t so bad, isometric path-finding on square tiles was very difficult. Instead of large (32×32 pixel) diagonal tiles that were either passable or impassable, the map had to be broken into tiny 8×8 pixel tiles — multiplying the amount of path-searching by a factor of 16 as well as creating difficulties for larger units that couldn’t squeeze down a narrow path.
Had Brian Fitzgerald not been a stellar programmer, the path-finding problem would have prevented the game from launching indefinitely. As it was pathing was one of the problems that was only finalized at the end of the project. I plan to write more about path-finding in StarCraft because there are lots interesting technical and design bits.
End of part 1
So you’ve heard me whine a bit about how difficult it was to make StarCraft, largely through poor choices made at every level of the company about the game’s direction, technology and design.
We were fortunate to be a foolhardy but valiant crew, and our perspicacity carried the day. In the end we buckled down and stopped adding features long enough to ship the game, and players never saw the horror show underneath. Perhaps that’s another benefit of compiled languages over scripted ones like JavaScript — end users never see the train wreck!
In part two of this article I’m going to get even more technical and talk about why most programmers get linked lists wrong, then propose an alternative solution that was used successfully for Diablo, battle.net and Guild Wars.
And even if you don’t use linked-lists, the same solutions carry over to more complex data structures like hash tables, B-trees and priority queues. Moreover, I believe the underlying ideas generalize well to all programming. But let’s not get ahead of ourselves; that’s another article.
Thanks for reading this far, and sorry I haven’t yet discovered how to write concisely.
About Patrick Wyatt
Gotcha: this is where you tell the world how wrong the hopelessly inexperienced decisions made by someone else–a decade and a half ago–were while you were off doing other things. You even name him, so everyone knows exactly who it was you were annoyed by. Classy, dude.
Thanks for your comment; you make a good point.
You would be right to assume that I was annoyed with others on the development team at times. It would be inhuman not to be annoyed with them when working long hours in close proximity, under highly stressful conditions, for years at a time. I’m sure they’re annoyed with me for some of the things I did too — damn was I an obnoxious pup back then. But that annoyance doesn’t characterize the entirety of our relationships either.
But I’m at a loss as to how to write history without referring to the participants involved, the decisions they made, and a modicum of analysis regarding the results. Books are filled with analyses of our political leaders’ judgments; should bloggers then not provide critical commentary? Or must I divorce the decisions from the participants so everything is washed out in such a way that there’s no responsibility or ownership?
The agony of the development of Starcraft, following right on the heels of the unbelievable difficulty of launching and supporting Diablo, nearly broke me and some of the others on the team. The best thing I can say about the process is that we made some games that players have enjoyed for years afterwards. But how we got there is still pretty raw even after all these years.
“Gamers who played any game created by Origin will remember how looooooong it took to write save-game files.”
Oh yes. I vividly remember playing Utima 7: Black Gate, and being blown away by the size of each save file: around 700k! My hard drive was 40 mb at the time.
I’m sure Bob doesn’t mind =)
It was his first real project afterall, and it was wildly successful! These stories are a great way of spreading wisdom to new generations of programmers.
Actually, “Rock & Roll Racing” in 1993 was Bob’s first published
game title, and he did awesome work on it. While it was a sequel to RPM Racing (also by Blizzard in its early years), R&R was a far superior game in every respect thanks to Bob’s engineering efforts.
I worked on a very minor portion of the project — porting the “track
compiler”, which took a 2d top-down description of the map and turned it into an isometric view for display on the Super Nintendo’s planar, cell-based tile hardware, and Interplay wrote the sound engine, but Bob coded everything else.
R&R, along with “The Lost Vikings” (also released in ’93) won our company Nintendo Developer of the Year, but tragically that was also the year that the bottom dropped out of the 16-bit console market so both games sold poorly. But Blizzard made lemons into lemonade — we switched to PC development, built Warcraft, and did pretty well thereafter!
So I think Bob gets credit for doing other “real projects”.
I don’t think its an issue,
He is telling a personal account of what happened, of course it will have is thoughts embedded in the writing.
I read no venom in his words, and like you said it was a decade and a half ago, people make mistakes, deal with it, like the blizzard team did.
Uh, I read the whole article and didn’t detect any animosity towards Bob Fitch. The author was just telling things how they were.
Jives with some of my work on SC in QA at the time. Thanks Pat!
What a great article, very insightful. I can’t wait to hear about linked lists. :)
Great article, looking forward to Parts II and III. And it was good to meet you at the Hacker News meetup a few months ago!
Great Read! Thanks for this.
As a Developer from a different industry I find so much of your examples analogous to development teams in general.
That is Difficulty integrating the inexperienced members, balancing the amount of re-factoring/re-writing and band-aid patches vs time consuming architectural changes.
It also goes to show that even with such a large successful games like starcraft was the technical/engineering side had a lot of issues to deal with.
Never imagined the development was so harsh, especially given the quality of the games.
Starcraft and Diablo are the two games that made me want to become a game programmer back in my teens.
I even used the MPQ format to teach myself about binary files, hash maps, cryptography and compression! Heck, my very first experience with “code” was making up trigger maps in the Starcraft editor, which I’m pretty sure was made using wxWidgets!
My guess of the list’s problem is that it had data races. Of course not having it abstracted away in a library is equally bad! One job offer I read on the Blizzard website asked for experience with the boost libraries, I guess that was the solution ;)
I believe the map editor was written using MFC (Microsoft Foundation Classes), but I can’t remember for sure. Ask Jeff Strain over at Undead Labs (http://undeadlabs.com), who is getting ready to release State of Decay, a zombie survival game. He’s the one who wrote StarEdit.
“Thanks for reading this far, and sorry I haven’t yet discovered how to write concisely.”
And I hope you never do! These posts are a great read =D
that was a fantastic read. ive always wanted to read about internal development process for the game i played and loved for many years, including specific design choices from the coding perspective. waiting for parts 2 and 3..
I was maybe 16 when Starcraft came out. It will probably always be my favorite game (Although SC2 is pretty good too).
I’m now a software developer myself, so hearing these stories is just about the most awesome thing I could ever imagine. I remember when SC crashed (it was actually a rare occurance in the final version) I would see the source filename in some kind of custom backtrace that was built into the engine. I still wish I could see the code.
Fascinating article. This combines two of my favorite things: Starcraft and software development.
I can’t wait to read your next article. Also, as far as I’m concerned, the more technical the better.
Amazing read… I can’t properly explain how much these articles mean to me. Thank you for posting them.
Can’t wait for part II
As if he needed a confirmation, as someone who made hacks for Starcraft, I can definitely confirm there were doubly linked lists everywhere. :-)
Did you have any experience with dealing with the nasty loopholes reverse-engineers (like me) uncovered?
Also, have you ever looked at the reverse-engineered unit memory struct the modding&hacking communities developed to see how close we came to figuring out how our favorite game worked?
I’ve known of the StarCraft reverse engineering efforsts as well as the AI competitions using BWAPI, but never looked into them in detail. Given the complexity of the unit structures, I expect it was a hard slog! My guess is that you must be doing pretty well if you can use BWAPI to write AI for the game :)
http://farty1billion.dyndns.org/euddb/?pg=ref&a=unitnode
Awesome, Kaias, thank you. It’s clearly possible to see cases where linked-list pointers are stored in unions with non-pointer data, like for the rally point (wRallyX/wRallyY).
Actually, I’ve since reverse engineered a better unit structure ( http://code.google.com/p/bwapi/source/browse/branches/bwapi4/bwapi/BWAPI/Source/BW/Unit.h ) (Minus the storm template implementation for the prev/next header, and possible base classes). I’m one of the owners of BWAPI and I’m big on getting the “most correct” structures and names. With at least 75% of the game’s memory and functions mapped out over the years of working on BWAPI, the biggest headaches are still the AI and unit pathing. One issue we’ve had is fixing a memory leak that was in Broodwar itself (the game currently allocates a bit of memory for the HUD but never frees it, and does so for every game played), which was found when some of our users ran their bots thousands of games a day non-stop using BWAPI’s speed hacks. If you want to join the project or toss us some hints I would gladly accept. >:)
I’m one of the guys participating to the annual SC AI contests. The BWAPI team does indeed an amazing work, giving an efficient API that us, AI researchers, can exploit to apply new AI techniques on a robust and commercial game. Some AIs are very well documented (commented source + scientific articles) and remains accessible for non-specialist-but-curious people. Have a first look here: http://webdocs.cs.ualberta.ca/~cdavid/starcraftaicomp/ and there: http://code.google.com/p/bwapi/
Great article, please keep coming more of these!
On a scale from 1 to oh god my eyes, how bad was the code for dealing with iscript.bin?
Writing iscript.bin code was actually pretty easy. The code looked roughly like that shown here: http://blizzforums.com/showthread.php?22347-I-m-stumped…-and-need-help. I say roughly because that post shows code that was reverse-engineered; the creator of the reverse engineering tool needed to decide what the opcodes should be named, so they’re different from the ones I chose.
The iscript.bin file dates all the way back to Warcraft 1. It is actually a header-less COM file built with Turbo Assembler (tasm.exe) from Borland. Each instruction was an assembler macro. The macros would validate their arguments and emit the right byte-codes.
Easy pie, but mind-numbingly tiresome work. If we had more time I would have designed a tool so the artists could have rigged the animations, but as it was the artist and programmer had to sit at the same desk and get the timing right.
The code that ran the iscript.bin file was a byte-code interpreter like you’d see in a scripting language virtual machine, but instead of operations like “add” and “subtract” it had operations like “move 3 pixels”, “wait 2 frames”, “shoot bullet”, and “play sound”.
Interesting. I hope your post on pathing can shed some light on why Dragoons were by far the worst at pathing in the game?! :]
Because Dragoons were larger than other units they needed
to find wider paths that weren’t obstructed by terrain or other game
units.
Biggest Knowledge bomb I’ve read in a long time (learned so much). Please continue on the path of talking about the history of the games development in the same way you did with this first part.
Wow thanks for another great write up Patrick.
Growing up playing these games, and dare I say inspired by these games to head into game development myself, its fascinating to read about the history of these games.
I had no idea that any of this stuff was going on, cant wait for the other parts mate =D
Very interesting article, thanks! :)
As someone who has experience with Westwood’s C&C series internals (I disassembled their engine to fix and extend it for fun), it’s fun to see some familiar things, even if they’re negative.
1) “Experienced programmers will shudder when seeing the inheritance chain that was designed for the game’s units:” You think 4 layers of inheritance are bad, Westwood’s stuff is worse: the unit chain is 7 classes long: Unit < Foot < Techno < Radio < Mission < Object < Abstract,
the ingame UI chain is even longer: http://qdb.renegadeprojects.com/612 .
2) Savegames. Westwood code also saves the entire C++ object into the file as well, and uses a "swizzle" mechanism to reconnect pointers on load. The disturbing part is, when this mechanism fails, it intentionally triggers a 1/0 to crash the game. Because meaningful error messages are for sissies?
On the other hand, Westwood's core engine survived the transition from Tiberian Dawn (1995) to Red Alert 2 YR (2001) with just extensions and polishing, no complete rewrite. Which I think was a mistake just as much as your story's full rewrite was.
1. I had heard rumors that Westwood’s engine had a complicated inheritance hierarchy, and that it slowed down their dev efforts quite a bit. I’m reminded of a comment Jeff Strain made in an email way back before we started ArenaNet, something along the lines of “The arrow class is derived from CFlyingObject and CWoodenThingy — hey, why does the game run so slowly?!?”
2. Pointer swizzling: yup, that’s exactly what StarCraft did too. Don’t know how well we handled invalid data though, so I won’t throw stones.
Thanks for the post!
Hi Patrick. Any chance of Starcraft’s source code being released? It could blow new wind and help immortalise the project like the diverse modding scene that Quake has had since source release. Pretty please :3 Thanks for reading ^^
I can’t see Blizzard doing that, seeing as they own the copyright to it.
They have the copyright, therefor, they wont release the source? Can’t see how that follows…
Cuz: Bli$$ard.
Nuff said
Blizzard’s also stated that they have no plans (/intentions) of keeping Brood War running on future OS’s. An OpenSource modding scene could keep the game running, and perhaps enhance it for modern HD interface & capabilities.
I’d play BW any day over SC2, so it’ll be a shame to have to emulate an older version of Windows just to show the youngin’s how the truly great games used to be.
> The disturbing part is, when this mechanism fails, it intentionally triggers a 1/0 to crash the game.
I laughed out loud at this one!
I’m just hoping the actual line of code looked like:
if (!reconnectPointer(&data->ptr)) { int a = 1 / 0; /* fuck you */ }
Oh, this brings back memories. I had the honor of being one of the legion of inexperienced. I did not know what I was doing; it’s a miracle that game ever shipped!
Howdy Dan, good to hear from you! It was something of a miracle, wasn’t it. We all had lots to learn from that experience.
Write as much as you’d like, sir! We’ll read it. =) I’m curious if you reused the fog of war updater to do unit range checking in Brood War. A decade of UMS games taught me a lot of similarities between how these operate (and how to manipulate them), just on different timescales.
Thanks for taking the time to write about your development experiences. Even 15 years later, it’s still relevant and insightful. Despite today’s vastly improved technology people still fall into the same kind of traps.
My god, what an awful design, I can barely read this. 3 words per line?
It’s a responsive layout. If you’re only seeing 3 words per line then it suggests your screen is tiny. Or your browser is ie7.
How about sending a proper bug report instead of flaming? I’m using an off-the-shelf WordPress theme, but if you tell me something about the problem it’s more likely to get fixed.
I would suggest Ctrl + – in browser. It could helps :)
Uh, works fine for me.
Very interesting article, looking forward to read about the pathing because the glitches in Starcraft’s unit pathing is one of the major factor that made this game so competitive afterwards. Controlling units in this game is an art.
Now I know why SC2 took so long …
I was a serious Starcraft “Mapper” hobbyist for much of my youth, (and still sometimes still dabble in it for fun), so I know a lot of bugs, quirks and nuances in the Starcraft engine (I created the first version of this
http://www.staredit.net/starcraft/Quirks_and_nuances). The map-making scene is more active than you might think, despite being 14 years old, and new stuff is still discovered. Case in point:
http://www.staredit.net/topic/15356/, which is only a month old. So, needless to say this article is deeply interesting to me, as the next parts will be I’m sure.
P.S. If I had a wish list, it would be more locations, or the ability to place a location at specified coordinates. I developed ridiculous grid systems to deal with the serious location limitations. I think you’d be surprised at the ridiculous feats I/we’ve been able to accomplish in mapping (if you’re ever curious I could point you at some good examples).
this serie of articles is extremely interesting :) i loved those games back in the day, it’s almost moving to read about the “behind the scene” :) Thank you!
So that’s why Dragoons are so dumb, too small pixels!! :D Nice article!
I didn’t read this entierly, but I will soon.
I am happy that they didn’t rush it tho if they did I wouldn’t be playing it on multiplayer on ICCUP, right now!
It’s fun to read this after all these years. I’ve always assumed StarCraft was programmed pretty damn well, because the game ran so incredibly smoothly on my modest Pentium 100Mhz at the time. I never would’ve guessed it was such a monster under the hood.
I don’t understand how doubly linked lists made it faster. Traversing a doubly linked list forward or backward is just as fast as traversing a singly linked list forward, is it not? Can you clarify this for me please?
Imagine that you have a pointer to an object in a linked-list, and you want to remove that object from the list. With a doubly-linked-list, you can do that in O(1) time; simply use the object’s “previous-node” and “next-node” pointers to find the adjacent nodes, and update those nodes’ pointers to point to each other. With a singly-linked-list, OTOH, the only way to find the previous node is to traverse the entire linked list starting at the head of the list, which will be quite inefficient if the list is long.
Makes perfect sense. Thanks for that.
TL;DR: Doubly Linked Lists were chosen to make adding/removing items (units, building queues, movement queues) fast, which is different to traversal.
Each type of data structure (singly linked list [SLL], doubly linked list [DLL], vector [V], etc.) has different performance characteristics for the different operations performed on them. Choosing the most performant data structure depends on the operations you want to do most frequently.
Traversal — for things like movement — is pretty much the same across SLL, DLL and V data structures.
Insertion and deletion — adding/removing a unit in a queue, units being killed, etc — varies. For SLL, you need to traverse the list so is O(N): it takes at most the number of items in the list to complete. For V, you need to copy the elements after the one being removed back one, or copy the whole vector if adding a new item goes over the vector’s capacity. For DLL, the item has direct access to the previous and next items so there is no need to traverse the list and each item is allocated independently so there is no need to copy items around in memory. DLL insertion/deletion is O(1): it happens in constant time, irrespective of the number of items the list has.
I disagree that traversal is pretty much the same when using a vector or a list (in real life conditions).
With vectors you have a much better locality, all elements of the vector are in consecutive positions in memory. This is not the case with lists, every time you access the next or previous cell you have to jump around in memory, causing more memory-cache faults.
dope, I love that you guys pulled through
I have been thought Starcraft is one of the masterpiece game. This post is so interesting. Thanks for sharing your thinking and experience!! I’m looking forward Part2!!!
I have to say that you and Blizzard did a very great job debugging the whole game. I played StarCraft for more than 5 years, only playing campaign and random matches against AI. I haven’t Internet at that time nor Games Magazines, so I was playing during the whole period with the 1.0 version of the game! And you know what: I’ve never experienced any bug, any crash, any speed reduction, any corruption of anything in that game. Even I was playing the last mission in the protoss campaign for more than 12 hours (I was a kid) and everything works fine: units, resources, save games, etc.
Five years playing with flawless 1.0 version of a game. It sounds like a great achievement to me.
Amazing read. You guys are heroes and it’s magical to get to know your experience with the game that defines my leisure time. Thank you!
If you’re ever in Berlin, I’ll be forced to buy you a beer. Maybe two.
I look forward to it!
If you wanna come to Beijing, I’d like to offer you3!
3 parts 20 pages? I would gladly buy a book the size of LotR if you wrote one about developing these games.
Perhaps you should pick up a copy of Michael Abrash on the development of Doom and id Software.
What is that book called?
Masters of doom
One of the best books I have read.
Also take a look at Jordan Mechner’s “The Making of Prince of Persia” which a wonderful read (so far, I’m about 40 pages in).
Love it, please keep writing anything StarCraft + software construction.
> It was necessary to fixup all the link pointers
Ouch.
>> Taking special care of unioned pointer fields.
Double ouch.
At this very moment I’m working on a system for my company allowing automated saving/loading for arbitrary sets of C structure-based objects with pointers and unions and I can feel the pain of identifying/tracking/saving/loading the type that a pointer references or union contains. A great shame this repetetive technical stuff distracted so many people from writing the game logic itself. High-level languages nowadays must have all sorts of tricks to tackle the saving/loading issue in a uniform way.
Thanks for writing this Patrick!!
I think it’s safe to say StarCraft has helped shape all of us, and to finally get a chance to dive into it’s development process is a bit surreal. Definitely looking forward to the other 2 parts :)
As someone who was very heavily involved in the StarCraft modding scene, I’ve got to say this is an absolutely fascinating article. It definitely explains shall we say, some of the “unique quirks” we encountered in reverse engineering stuff.
Great read!
But it makes me wonder.
“Blizzard” didn’t learn a single lesson?…
I’m thinking of Battle.net and Diablo3. Releasing D3 which was not only buggy but obviously must have had some serious game mechanics development problems, an auction house bugged to kingdom come and battle.net which would let hackers brute force passwords which are not even case sensitive (with the addition of Battle.net support blaming the user).
I guess you were not following the launch of Diablo 2 if you think Diablo 3 was buggy. Or maybe you chose to forget? :)
Strange how all the bad code I’ve had to maintain was written by programmers with a sense of humour.
I will excuse excellent code written by a comedian however.
The main lesson I keep hitting over and over again is that invariants are generally only violated temporarily inside of function calls, and a header file full of functions should generally define legal calling orders (ie: an FSM or grammar of functions) so that invariants that span sequences of calls aren’t violated. These are issues that go far beyond C++ versus C versus $GARBAGECOLLECTEDLANGUAGE debates. I boggle at the OO focus on inheritance, rather than focusing on things like using co-routines or things in the type system to help document function calling order (ie: Protocols). When the invariants are truly tight, you should be able to invoke random sequences of the APIs without crashes and drop lots of other runtime checks that you would otherwise want to have.
More please.
Excellent post! May Adun be with you!!
Thank you, this was a great read, and a great game!
Awesome read although not being a progammer, certain parts go over my head :)
I wonder, are you familiar with “move-shot micro” such as the attack-retreat micro using the patrol command with the vulture? And if so was this intentional design or incidental unit behaviour due to how they were programmed? (Hold command works also for a bunch units including dragoons, un-sieged tanks, mutalisks, etc.)
Because there are so many crazy things that keeps SC awesome even 14 years later.
There was a lot of emergent behavior (like vulture micro) that players discovered over the course of years; most of it was because — in the rush to finish the game — many edge cases weren’t discovered by the design, programming and test teams.
My favorite discovery by players was the “Korean eraser”, where Terrans would irradiate a flying building and use it to wipe Zerg units off the map; it wasn’t something we had anticipated.
Another boo-boo was that, with careful timing, it was possible for a Terran player to land a Command Center much closer to resources than should have been possible, enabling faster harvesting, which made the Terrans unstoppable in the early game until patched.
None of the many “features” that were eventually discovered were intentional.
Interesting how programming bugs become part of the high-level competitive game and become “features”. It seems like this is common only in competitive games and not other genres. For example Quake has the strafe jump that was originally a bug but was ported to subsequent games.
I actually remember these! lol in the original release you could have CCs next to minerals and you could also somehow overlap bunkers and CCs leading to a makeshift Planetry fortress lol
Even when I was an inexperienced developer I couldn’t ever imagine handling linked lists like that. When I was first taught about linked lists and various other data structures I was taught to have wrapper classes / functions for add, deleting, etc. Why wasn’t that caught early on? It almost sounds like you had an entire team of new devs with no real leadership. If I missed the explanation of that forgive me.
We *had* those awesome wrapper classes in the form of C++ templates in Storm.dll, but the team chose not to use them. I also made a huge mistake in fixing the bugs one at a time as they were discovered instead of “doing it the right way”.
I am reading this thinking, wow, they *must have* been inexperienced… It’s not hard to write a simple, consistent set of routines to manipulate the list (even without C++ – witness how Linux kernel manages linked lists in C as an example), or create a C++ template that ends up with a memory layout identical to whatever code you were using. I really doubt making the save-game hack work while still maintaining a sane set of interfaces was as intractable as the team thought.
Great and wildly successful game though. I knew already that this game is something that would be hard for any team to replicate (in both gameplay and success), but now we see more of how true that is.
Awesome read, I can’t wait for more!
I’ve always loved tinkering around with StarCraft … be it mapping or modding (or “breaking” it with ridiculous values …). It’s interesting to see how and, in some cases, why the things are the way they are (such as the mess that is CUnit — which I always just attributed to incomplete documentation on our end).
And as someone who has previously tried working to figure out how to make better modding tools, I’m just curious about what kind of in-house tools have existed? I’m aware of the GameTemplate.exe thing that got leaked (or whatever the story was on that…), but was everything just sort of cobbled together similarly to iscript.bin? (Gotta love that tasm.exe!)
With the small number of programmers we had for StarCraft, building tools for designers and artists wasn’t a huge priority.
With the exception of the map editors for Warcraft 1 (CED – never released), Warcraft 2 (MapEdit) and StarCraft (StarEdit), virtually all of the other tools we had were “format conversion” or “data packing” tools designed to input image files (DPaint LBM, DPaint ANI, etc.) and convert them to game-format data that could be rapidly blitted onto the screen. I wrote most of the art-conversion tools, but every programmer wrote some: James Anhalt (file compressors, DPaint ANI format reader), Mike Morhaim (LBM format reader), Bob Fitch (FPack), Tim Thein (MapEdit, Mopaq), Mike O’Brien (various), Jeff Strain (StarEdit), and others that I now forget.
One cool set of tools I remember was one that Mike O’Brien wrote, which read every single image file in the entire StarCraft project, built the best possible color-weighted 256-color palette, then reprocessed every single image to perform color remapping to that new palette. It would take a couple of hours to run, but meant that the artists could draw each piece of artwork without using a single “locked down” color palette. Oh, I am so glad we use 24-bit color now!
For Guild Wars we spent years building tools to make content generation easier for artists, designers and localizers, though there were always areas that could have used more work.
Love these articles man, can’t wait for the next one!
Best story on game dev i have seen! Please share more!
Warcraft 1 and 2, and Starcraft were a huge part of my life at the time.. I am loving your posts as much as I did the games..thank you!!
Really liked the article, and look forward to reading the rest when you get to it!
I am eternally thankful That the game was supported for so well for so long. I started out on a 200Mhz MacOS8.5 system, Later upgraded to an iMac, then to OSX on that iMac. I was so amazed how much better the game ran under osx, It ran as well as the Windows Version. the MacClassic version did not. I am loading up StarCraft now. Still I wish the game was open sourced to support 16 players, and more simultaneous units.
Patrick, thank you so much for this!
I’m new to gamedev and read from your experience with an awesome game like SC, that I used to play in those days, it’s inspiring!
Looking forward to part 2!
One of the things I’m terrified at when handling a new project is using new and untested tools, like that switch from C to Visual Studio/C++ Blizz did with Starcraft. I had a terrible experience while working with a company that decided to switch to .NET 1.0 Release Candidate so that we would have “an edge”. Lack of documentation, bugs and just a very different way to do things ended up derailing that project. I wonder how you guys managed to pull it off so successfully.
Great post.
I’d like to bring to light something even such excellent game developers as yourself may not fully appreciate or understand (which SC2 blatantly proved), namely the fact that objectively better code. Take SC2 pathing, for example. Even though it’s one of the best pathing AI’s coding wise, it has caused many gameplay problems due to the units clumping up really tight:
– chokes on the map were no longer a significant obstacle, which resulted in diminished importance of proper army positioning,
– splash/AoE spells had to be nerfed significantly because they were too powerful; as a result they were made so weak that they were no longer game changing or spectacular,
– it led to the unanimously hated “deathball” type of gameplay, where large “balls” of units would collide in an unspectacular fashion – because that would be the most efficient way to position your units, most of the time.
Another example of better code worsening the gameplay is the removal of overkill of Siege Tanks. In Brood War, a Protoss player would drop Zealots on sieged Tanks to cause friendly fire, with all nearby Tanks firing, killing their fellow. In SC2, not all Tanks would fire – only enough to kill the Zealot, rendering this tactic useless.
Another thing is that bugs are not always a bad thing. Sure, they were not intended, they may crash the game or break the balance, but sometimes they actually enhance the experience. Can you imagine future Quake game without starfe jumping or rocket jumping? Or new Counter Strike without bunny hopping? I’ve read one of your comments where you acknowledged this, but, nontheless, I’d like to point that out – maybe someone at Blizzard will read this and make them change their philosophy, so that when the next “Void Ray fazing” (http://www.teamliquid.net/forum/viewmessage.php?topic_id=134124#1) gets discovered, they won’t outright remove it, and instead evaluate whether it makes the game worse or actually better.
Great post and analysis! It can take a long time for players to figure out some of these techniques — some SC techniques weren’t discovered for years!
So for example it’s hard for the dev team to appreciate all the ways that a better path-finding algorithm impacts the game right away – the first instinct would be that “packing” units would be good, right?
The key for game developers to take advantage of these types of unintended features is a longer play-balance and testing cycle, which ultimately comes down to having a large enough budget to be able to hold onto the game for a while before releasing — something that’s incredibly difficult to do given the economics of getting a game to market. Few companies have both the willpower and deep pockets necessary.
I don’t agree. StarCraft (and Quake for that matter) should be a game about strategy and not about wrestling the unit AI or the user interface. While I hate the death ball as much as the next guy there are other ways to prevent this than making the units stupid. I also think there are other and more important reasons the game tends towards death balls than the unit AI.
FIrst of all, Brood War is a game of strategy moreso than StarCraft 2. The only match-up that’s deeper strategy wise in SC2 is ZvZ. Mechanics are just as important in an RTS game as strategy, they are what makes the games spectacular for non-advanced players, be it MarineKing’s Marine splits vs. Banelings or Jaedong’s Muta micro in BW.
Where did I ever say anything about making units stupid? You can have perfectly fine pathing AI without units clumping up (and thus creating all sorts of gameplay issues). Case in point: WarCraft 3, Company of Heroes, Armies of Exigo, and many more. Not to mention the fact that SC2 fans usually blow BW pathing’s deficiencies out of proportion. There’s a handful of buggy units, but other than that, there are hardly any issues with pathing. You can fix SC2’s pathing without making it “stupid”. There have been a couple of solutions proposed, one of them being “dynamic movement”:
http://www.teamliquid.net/forum/viewmessage.php?topic_id=223889
If you were refering to the overkill mechanic, then that’s a design decision, not making units “stupid”. Players have to be able to maximize their units’ potential. Otherwise we end up with glorified rock-paper-scissors, which unfortunately SC2 largely feels like. If you get rid of the overkill mechanics, striving to make units “smarter”, what stops you from going down the slippery slope and make the units even smarter? You could very well automate Marine splitting, make air units never get hit by units with lower range, etc. (just look up Automaton for SC2 or BW AI competitions for BW).
Another reason for deathball gameplay in SC2 is the lack of defender’s advantage caused by the severely nerfed high ground advantage, which in SC2 is nonexistent past early game, as well as the lack of positional units for Zerg and Protoss. In vs. P match-up it’s coupled by the warp-in reinforcements. Even Siege Tanks were nerfed and lost their map control role outside TvT. They were nerfed for two reasons: horrible Blizzard maps early on, and unit clumping (the same reason why all AoE/splash spells/attacks were nerfed).
It seems that we mostly agree except for the first point. I want the influence of mechanics to be reduced as much as possible. I find nothing spectacular in marine splits. On the other hand I find it spectacular when NesTea sacs a natual hatch he could save just to give himself time to morph banelings and wipe the opponent with a counter attack or the subtle movements on the ramp where after 1 minute dance not a single shot is fired. This is what I find beautiful and cool about StarCraft and I never admired players with mechanics and I even had some disrespect for them especially if they lacked strategically (like MK was in his first year). I don’t like football for the same reasons.
I just discovered your blog, and I must say it’s brilliant. Your replies in the comments are sometimes as complete as a blog post! I’ll definitely put this on my “must read” list.
Have you thought about launching a Kickstarter project to write a book? I would definitely support it.
Thanks for the vote of confidence :) I think I’ll see how blogging goes for a while before attempting something quite that ambitious!
Thank you for making StarCraft 1 :) I have played it since release and followed the proscene carefully. I don’t know if you have any insight on that side of the fence, but it’s definitely interesting to read about its development. To think it was going to be a “modest” release! So many lives would be very different today.
Thanks for this series. It is essential to improving all of software engineering.
Great read as an aspiring programmer. Helps me to see the kinds of things I should be avoiding. Thank you so much for the hard work you put into one of my all time favorite games. I really appreciate it!
I love the insight into the design of these old games. A while back I read Jordan Mechner’s book about the creation of Prince of Persia with equal interest. I look forward to the following parts!
As someone who has spent years reverse engineering games of all sorts (including bearing witness to the huge amount of effort that went into reverse engineering things like the MPQ file format and the storm.dll library) its always fascinating to see how much effort went into developing these games.
And its interesting to reverse engineer different data file formats, especially the really challenging ones (especially fun to reverse engineer things when the developers would rather you didn’t as seems to be the case for the .mpq files with the encryption and stuff)
Good read. I must say the similarity in our names was almost a WTF after i got done reading. =)
By the way, can you imagine that StarCraft is still being sold in stores in Poland, almost 15 years after release? Congratulations on making such an amazing game. ^_____^
Looking forward to part 3 :)
Great article, thanks for sharing some of your ‘in the trenches’ stories from the creation of Starcraft.
I work on the (foss) Spring RTS engine, which was and is engineered quite well: no crazy inheritance trees, modularized code with clean interfaces (eg. the pathfinder can be hotswapped for a different implementation), MVC as the predominant pattern… all in all it probably has healthier internals than many rushed commercial titles ever did.
Yet when I apply for a job in the industry and present this background I get rejected for being “inexperienced”.
The mind boggles, especially after reading your (interesting :)) article. /bitter
Wow — I hadn’t seen that engine before; it looks amazing, and from what you’re saying it sounds like it was better engineered than many commercial games!
It can be difficult to get into the game industry, but which can seem surprising given how hard it is to find developers. But if you think about how little training developers of all stripes actually get it makes more sense, because none of them are getting trained in how to hire either. Let’s chat via email (mine is above).
You’ve managed to entertain and educate me at the same time.
Thank you so much!
I for one would like to hear more about RTS path finding.
Excellent article – i always enjoy hearing about how a true classic had such a troubled birth! I doff my hat to you.
Thanks for taking the time to post this. As someone who has played and loved the game (not to mention having written a few lines of code), this is as good as having the curtain drawn upon the great and powerful Oz to find that the myth behind the magic is as substantial as one had feared.
just chiming in to say that this is a truly fascinating insight into the process, and a great article all round
Hi I’m wondering if you’re aware of the true extent of what Starcraft Brood war has become, what people have made it to be, and how they learned the game did amazing things with both the game design and the “bugs”
there’s a 4 page thread discussing your blog on Teamliquid btw
http://www.teamliquid.net/forum/viewmessage.php?topic_id=367126
We all miss having people like you
..around in blizzard
Inspiring for young programmers like myself, even though I’m nowhere near gaming industry, except for casual play.
Heading over to gladly read part II.
Thanks for the ariticle, I was chuckling the whole time (good chuckles, not derisive ;-)
I’ve been programming since God made dirt to hold down the trees and I’m aways amazed at how similar the release cycle story is across all environments.
Developing robust, usable software is hard. It’s more communication and art than science, which seems to be hard for some to understand.
Starcraft is one of my all time favorite PC games. All Time. Favorite. I would be proud to have been any part of the team that built it.
Well Done!
I decided to become a programmer when I was very young and started playing serious PC games.
My inspiration was Warcraft II and my wishes only grew with the arrival of Diablo and Starcraft. Blizzard was the company I hoped to work for someday.
So, I started programming when I was about 14 years old (now I am 29) and went professional during college.
The only thing is that Brazil was never strong at game development so I ended up being an enterprise programmer and never got to fulfill the dreams of my youth.
What you wrote is inspiring and I had no Idea how rough it could have been when the final product was so shiny. Your quote on compiled languages fits pretty well here.
Congrats and keep posting, this way I can check what I missed :)
Thanks for sharing. Brings a little bit of comfort when sometimes we’re trying to learn this and confidence is low.
“Since we were only “two months” from shipping, making changes to the engine for the better was regularly passed over for band-aiding existing but sub-optimal solutions, which led to many months of suffering, so much that it affected my approach to coding (for the better) ever since, which is what I’ll discuss in part two of this article.”
This brings back painful memories. When a company keeps moving dates like this and you think ‘if I’d known I had 3 more months I could have fixed this in a *much* better way.’
That’s such a brilliant, amazing read. Thank you SO MUCH for such great insights
I must react to this post. I think that Starcraft is not the only one to suffer from such crashes. But what if we could avoid those crashes while developing videogames, what if we could know which gameplay is the best while reducing the gamers’ dissatisfaction? Today there is a solution for doing that by using eye tracking for gaming. With this technology you can know what gamers are looking at on the screen that could help a lot for the development in gaming. I know the efficiency because I have been having pro gamers testing an eye tracker and their feedback are very good.
Why was intentional latency created when playing on Battle.net but not when playing over LAN or singleplayer?
Good question. The speed of sending messages from one game client to another is dependent upon the transport mechanism. Messages on a LAN can arrive in ~1 millisecond, whereas messages sent over the Internet can take on the order of 50-300 milliseconds (or more), depending upon the quality of each user’s individual Internet connection.
Aside: I remember the good old days when I was an LPB (low ping bastard) and played CounterStrike in beta against folks with 250 ms latency, which was like shooting frogs in a barrel!
The second factor which affects packet transport is the channel capacity. If the game attempts to send too many messages per unit of time, it’s possible to saturate a player’s bandwidth, causing dropped packets, which necessitates sending missing packet requests, which causes packet re-sends, which consumes even more bandwidth.
StarCraft is tuned to be able to play 8 player battle.net games on a 33.6K modem connected to an Internet provider. Remember modems? Remember AOHell? These speeds are far slower than even the worst LAN, where users were connected to 10Mbit Ethernet.
So the parameters for StarCraft games played on battle.net are tuned with the expectation that messages take longer to arrive than they would over a LAN, and that there is far less bandwidth to utilize.
In single-player games it’s possible to queue messages so that they’re able to be processed during the next game frame — there’s no reason to make the game feel more sluggish holding packets for non-existent players.
You’ll find more relevant details in another article I wrote here: http://www.codeofhonor.com/blog/reducing-perceived-latency
Well as I said, I have a lot of questions but I’ll fire one at a time :p
Why wasn’t the ability to change the game ports officially added into the game? Starcraft recognizes changes in the registry which allows ports 6112-6119 to be used however when I spoke to Blizzard’s tech support, they said that they don’t support it because it isn’t officially added into the game itself.
Short answer: it adds tech support costs without adding significantly to the prevention of each gamer’s Internet connectivity problems. It’s easier just to ask everyone to punch a hole for port 6112.
Is there a reason why you’d want to use a different port?
Fun fact: 6112 was chosen by Mike O’Brien because it looks similar to the letters bliz.
Routers are widespread nowadays and Starcraft can’t handle multiple players playing behind 1 router unless the ports for each player are changed.
I don’t really believe you(sorry) when you say it’s easier to ask everyone to port forward one port when Blizzard’s support site says Diablo 1 and Warcraft 2 Bnet Edition uses ports 6112-6119, which were created before Starcraft.
Source: https://us.battle.net/support/en/article/firewall-configuration-for-blizzard-games#4
And back on why intentional latency was created, if you took a moment to think about it during the development proccess, internet gets faster over time, not slower so it’d be logical to make an Extra Low Latency option in the options menu.
Nowadays, people use routers to share internet connections but Starcraft obviously has problems with multiple people playing behind one router so changing ports is necessary.
Also, how come Diablo 1 and Warcraft 2 uses ports 6112-6119 and Starcraft only uses 6112 then? At that time, those three games were the only ones that uses the Battle.net service so wouldn’t it be logical to have universal ports for Blizzard games?
I didn’t write the battle.snp transport-layer code; it was written by Mike O’Brien, as was battle.net. However, I ended up taking over both projects so I know something about it. But it was a looong time ago so I’m going to have to make some guesses.
My recollection is that battle.net is supposed to work with Network Address Translation (NAT), so it ought to work for multiple computers behind a router sharing a single public IP address.
The way the games (Diablo, StarCraft, War2BNet) work is that they connect to battle.net via TCP. Since only the server side of the connection requires a port-binding to TCP 6112 and the client side is unbound (that is, choose any ephemeral port), any number of computers can connect to battle.net from a single IP address.
Then (and this is where it gets hazy), when a game client behind the router wants to create a game it requests (via TCP) that battle.net reply with a game identification token. Then the game client connects to battle.net on UDP port 6112 and provides the authentication token and other information to validate its identity. By doing this, battle.net now knows which UDP port the game client is using, and can advertise that information (IP address + UDP port) to other game clients who want to join that game.
This of course only works if the router is willing to accept UDP messages from arbitrary IP addresses, which is dependent upon the firewall policies of the router in question.
Now I could have this all wrong, but that is why it should not be necessary for players to choose their own ports to connect, which would be a horrible UI experience for users.
If I was going to guess, I would imagine that the reason the registry setting is there is for cases where port 6112 was blocked at the firewall, say because it’s used for some other purpose.
So you’re saying all this techical business done by StarCraft doesn’t work, and I believe you. I guess the question is “why not?”, and that’s something that the programmers at Blizzard would have to investigate.
I don’t entirely understand your reply but I think I forgot to tell you two problems regarding routers and ports and stuff. Let’s say 2 computers behind 1 router, you can only port forward a port to only one computer at a time, leaving the other computer unable to host. The next problem is that if both of the players are in the same game, Starcraft sends data to the router and it’s up to the router to decide which computer to send the data and if it guesses incorrectly, then the game will horribly lag.Anyways, did anyone on the development team thought about map protection so that maps won’t get stolen?
The short answer is: StarCraft is supposed to work even if you have multiple computers sharing the same public IP (Internet) address. The job of your router is to “know” where the packets are supposed to go, even if you have several computers using port 6112. The router is supposed to perform port-remapping. For a more detailed explanation, see http://computer.howstuffworks.com/nat.htm/printable.
Since StarCraft *isn’t* working, you should contact Blizzard Support.
In regards to protecting maps against theft, it is not possible in peer-to-peer based game like StarCraft. The rule in security is that anything on the client computer cannot be protected.
Of course there will be a way around stuff like this but anything simple like a password protection that has to be entered into the map editor would of done it.
Did you ever help release any of the patches that was made for Starcraft?
Stumbled on this through Reddit… thanks for the reminisce :) Good times!
You can be sure I’ll be coming back on your blog very often. I am a young game programmer and that was a fantastic read for me. It’s even better knowing that it’s about my absolute favorite game ever. The unit design in StarCraft: Brood War is a work of genius. It’s still the game I play the most even to this day. Professional leagues in Korea helped getting me into digging the game design and to fully appreciate the beauty that this game is. I can’t praise StarCraft: Brood War enough.
I can honestly say that you are my idol.
Awesome article.. keep it up plz! :)
A note about the culture and habit of coding for long hours:
A brain and neurology scientist was commenting about EU leaders sitting all night long discussing financial problems. She said that sleep deprivation works like alcohol; it shuts down the higher cognitive levels of the brain. And if you know how the human brain works, you would know that what is left is the “repitile” part of the brain. Pulled to an extreme, one could say that some of the most important decisions were made by reptiles. Think about this every time you decide to work “all night long” and over time. You get stupid!
Still waiting on Part 2? Did I miss it somewhere?
doesn’t doubly-linked list still have to traverse through all the elements?
I’m just curious about why you didn’t use std::list as your linked list.
This is a great article. Thanks for writing it!
I do wish Blizzard would release the alpha/prototype Starcraft.
I imagine the only folks who would have that are the people who downloaded it from the Internet or have bootleg CDROMs. We didn’t save a copy ourselves at the time, and in fact made a point of destroying out-of-date disks to avoid accidentally using (or worse, releasing) old builds!
I’m an android developer and I’ve published an APP to stream #starcraft replays, if anyone want try it the link is goo.gl/js4Lm
Patrick, why doesn’t Blizzard ever release the source code for their games? You seem to indicate that one reason might be that Blizzard is ashamed of their code. Or is it that every penny is sacred to Blizzard? The 90’s Blizzard games are still marketable today, especially on mobile devices, but it would be a very charitable thing of Blizzard to release the source code to these games. Blizzard seems to be doing plenty well, money-wise, when you look at, especially, all the WoW players. Diablo and StarCraft are coming up on 20 years old. Blizzard games are too good to keep their source locked down forever. One of the first things I would want to do is make it so that the AI in StarCraft doesn’t give up. (As long as they have resources available to them, they should be able to keep restarting their base, over and over, after every beat down.) Do you think the public will ever see Blizzard source code?
Oh, there are lots of reasons for Blizzard not to release, don’t read too much into the notion that there is shame involved. There are almost certainly licensing issues with 3rd-party code/libraries/tools that would need to be cleared up or the code rewritten before it could be released. Given that, what’s the incentive vs. having the developers work on a new game?
If you’re interested in working on an RTS engine, check out Stratagus (http://stratagus.com/), I’m sure they’d love your help.
Thanks for the response, Patrick.
I would think most of the licensed code would be in less important areas, like sound. Blizzard could at least release the code they own all rights to. If it’s as simple as the game missing sound, the community could fix something like that, without significantly diminishing the authenticity of the game. Alternatively, Blizzard could try to get permission from the licensed 3rd parties to release their code. That licensed code is at least as old as the game is, after all. If the 3rd party company no longer exists, I don’t how that would work, but Blizzard could ask their lawyers. I agree that Blizzard shouldn’t probably spend a lot of time and resources on something like this. Rewriting large sections of code would be silly, and doing so would diminish the authenticity of the game, potentially, depending on what part of the game we are talking about. So, I would not advocate for Blizzard to do that, unless they really wanted to release the source code, and that was their only option.
The primary incentive for Blizzard to release old game code would be philanthropy — providing a gift to people. There is a lot for people to learn by seeing how things have been done successfully in the past. Tinkerers, that are old fans of the games, would also enjoy modifying the games in ways that they’ve always dreamed would be fun to do. Players would enjoy playing these modifications — different flavors of the game. The longevity and broad appeal that modding gives to a game is often incredible — almost astronomical, in some cases. But the real incentive here, if Blizzard needs more than “we did something nice *hearts*”, is that people would hold Blizzard in even higher regard than they already do, by viewing the company as a giving company. And when people like you, they tend to buy your products more. Although, companies like EA seem to have shown that this isn’t always the case… So, maybe I’m wrong on that..
I won’t pretend that there is nothing for Blizzard to lose by releasing the source code. To be fair, it would probably be impossible for them to release, for example, a modified StarCraft, for mobile devices, if the community has already ported StarCraft to mobile devices. There also might be algorithmic trade secrets in StarCraft, but I would think that there are none that aren’t by now known to the world, and possibly already obsolete, many of them. Someone can always reverse engineer it, if they want something from the game that badly, though. Also, considering how similar StarCraft 2 is to StarCraft 1, I wonder if Blizzard’s aversion to releasing source code has been simply to keep people from making their own StarCraft 2. The community could have easily added some of the new features of SC2, to SC1, that were ‘desperately’ needed and could have easily been done in SC1, but Blizzard needed as much incentive for people to buy SC2 as they could get, since SC2, in my opinion, is mostly just a graphical update with some needed improvements that could have easily been done to SC1. So, releasing source code forces Blizzard to be more innovative, and that is bad for them, but good for the consumers. So, game longevity can both help and hurt a game company. They kind of need their old games to die, if their new games aren’t going to blow the old games completely out of the water. I’m sure there are several more reasons for Blizzard to not release the source code. Anyway, thanks for reading. I hope this wasn’t too ramble-y.
Aww.. Did my response get hidden/deleted?
I guess I’ll just have to forever wonder why my comments are just vanishing. Is it a computer glitch, or intentional? If it’s intentional, is it because I offended someone? Did someone think I was saying something stupid or untrue? Did someone think I was saying something true that they didn’t want publicly said? There are all sorts of reasons comments can disappear without explanation. There is nothing to learn from a big question mark, unfortunately. If anyone would like to send me a short explanation (or any length you like), you can send me one at [email protected], since this site, and Disqus, do not seem to have any sort of private messaging system.
Test one last time.