Testing Video Games Can't Possibly Be Harder Than an Afternoon With Xbox, Right?

If the main qualification for this job is a love of games, there will never be a shortage of readily available workers.

"Hard lock!"

Twenty-five-year-old Adam Theuret hears the call and comes running. Another Xbox 360 has just crashed.

A new update to Microsoft's 11-million-selling video game system is about to go live, and it's being tried out first by a roomful of $8.25-an-hour minions. Outfitted at each desk with a flat-screen TV and three Xbox 360 consoles, the testers are checking that Xbox 360's latest boredom-eradicating features—which enable you to fast-forward through movies before they've finished downloading, and chat with your friends via MSN instant messenger while you download free game samples—can be installed and used without the system freezing or crashing. So far, it's touch and go.

Days from now, every Xbox 360 user will be prompted to install the new upgrade when they boot up their machines. But the testers have to do it first, downloading the software, then performing the new tasks, over and over, on several units of each variation of the console: Those sold in North America, the European Union, Japan, and "Rest of Asia" all differ.

Each row of testers has a designated "lead," who manages the team and copies down the data: IP addresses, software version, serial numbers. The star of the leads is Theuret, a man of unerring precision and efficiency, clad in a black T-shirt with dates of the VMC WORLD TOUR listed on the back. VMC isn't a band, though; it's our employer, the company Microsoft has hired to check the Xbox 360 upgrade for bugs.

The process is both grueling and, from the evidence, ineffectual; last week, Microsoft announced that the "failure rate" of its Xbox 360—which some Web sites have pegged as high as 30 percent—was "unacceptable." The company said it was extending its warranty on the machine from one to three years and will take a billion-dollar hit to cover the mess.

Countless cords are plugged and unplugged. Every few minutes, someone else yells "Hard lock!" as a Microsoft executive and a few leads come running over to assess the situation.

In each row, the nine others—myself included, each of us working since 7:30 a.m.—have no choice but to sit and silently assess the chaos. A guy sitting next to me fiddles with his controller, feeling, as we all do, a mildly suicidal mixture of boredom and panic, and then sheepishly asks: "Are we going to get to play today?"

Not today, my friend.

The "dream job" of being a video game tester may sound like a way to get paid for doing exactly what you'd choose to do in the middle of the afternoon on your own living-room sofa, but the reality is very different. To find out how different, I spent a couple of weeks at Volt, a Redmond company that is the country's largest independent video game tester. Hundreds of testers work at Nintendo and Microsoft during crunch times. More than 50 smaller Seattle-area video game developers—like Surreal, Valve, and Zipper—employ anywhere from five to 20 testers each. But when it's time to contract out some of the most grunt-worthy testing tasks, companies call Volt.

After responding to a help-wanted ad on the Web, I received a call within an hour.

"It says that you're a passionate female gamer," my recruiter began.

My interview consisted of a few questions: What kinds of games do I play? On what kind of system? Can I prove that I'm legally allowed to work in the U.S.? Can I get there on time? The recruiter then started giving me directions.

"Do I have an interview?" I ask.

"That was the interview."

Volt's contract employees—a term meaning, roughly, "no guarantees, no benefits"—pick up their shifts by calling an employment hot line at 2 p.m. If you're selected for a shift, you get an automated confirmation call back that night. Which I did.

I show up at 7:30 a.m. the next day. Volt sits across the street from a golf course on Willows Road in a desolate area in Redmond. Volt's parent company was founded in New York in 1950 as Volt Technical Services, publishing technical manuals during the Korean War. Known today as Volt Information Sciences, it has more than 300 offices around the world and is in the business of temporary staffing, yellow-pages publishing, and information systems. (Its Redmond-based tech consulting unit is called VMC; hence the T-shirt.) Volt's stock has taken a beating this year as its profits have fallen.

In the testers' area of the parking lot, one car has a door attached with duct tape and willpower alone. A few guys in their early 20s, wearing black, are smoking near the entrance. On a counter in the cafeteria sits a row of sign-up sheets, each for a different lab, or workroom, upstairs. I enter and look for a sign-up sheet with my name on it, without success.

"Sorry," chirps one of the receptionists, after telling me that, despite confirming my shift the night before, I'm not working after all. "Maybe get here a little earlier next time?"

My shift has already been taken by someone who wasn't on the schedule, but who came in and signed his name on the "Bullpen" sheet, a kind of day-laborer list.

"It's literally like the 'shape-up' that they used to do at the dock," says Marcus Courtney, president of the Washington Alliance of Technology Workers (WashTech). He's referring to an early-20th-century way of hiring longshore workers and deckhands. "You go there, and you kind of hang around, and they say, 'All right, we need five guys to help unload this ship—you, you, you, you, and you. The rest of you come back tomorrow.' It's not any different." Courtney, a former Microsoft employee, has been working on organizing groups like video game testers for almost a decade.

"A lot of people just don't show up for their shifts," admits Volt's Theuret, who says that this shortage of consistent workers is why the Bullpen, designed to fill empty shifts on the fly, exists at all. It helps identify those who really want to get ahead—like Theuret, who says he signed up on the Bullpen sheet almost daily for a year before being promoted to a full-time position as a lead.

I leave at 7:35 a.m. The bus ride home takes two and a half hours.

If the main qualification for this job is a love of games, there will never be a shortage of readily available workers. "It's every stoner's dream in the Northwest to play video games for a living," jokes Sean Day, 36, who's worked as a tester for Volt and Nintendo, and currently works as an apartment manager. "And I should know—I've tried it three times."

Other industries have scads of workers making minimum wage, but video games inspire a special sort of madness. Addiction has been linked to deaths, like the 2005 rash in which 10 South Koreans died from blood clots caused by prolonged playing sessions. Last month, delegates at the annual meeting of the American Medical Association considered including video game addiction in the upcoming revision of the Diagnostic and Statistical Manual of Mental Disorders, but decided the idea needed further study.

"I think the [video game] industry in general, they want addicts," says Dr. Hilarie Cash, founder of Redmond's Internet/Computer Addiction Services. "It is to their economic benefit to have people really hooked on their games. That there happens to be a pool of addicts out there who will work for peanuts, like methadone treatment, is to their advantage," she says.

"A lot of the guys I've worked with are testers," Cash says of her clients. "If they have any job at all, that's the only job they seek...and can keep."

The thrill of victory is universal to competition, but it's also universal to addiction. Video games hit these sweet spots using psychological principles of intermittent reinforcement, of keeping you on your toes awaiting the payoff.

"Video games, for those people who get hooked, is elevating dopamine in the brain, and then your body adjusts to the overly produced levels of dopamine. That's the same way all addictions work," says Cash. Video games also tap into our desire to adopt a new persona, to be transported elsewhere now, to have a "redo" button for our actions, and to offer the feeling of interaction while ultimately being in control of the situation.

That all-encompassing sense of control, I realize while being turned away at 7:35 a.m., does not happen in real life.

The next day, I show up early and make my shift on time. At 7:20 a.m. the fluorescent-lighted cafeteria, adorned with only the mandatory minimum-wage and OSHA postings, is already full. The crowd numbers well over 50; most are young, white men. The T-shirt on one reads: "Your skill in Reading has increased by one point" (the first of countless references to seeing the world through points and levels). Soft drinks are free, but juice and bottled water cost 25 cents. (Red Bull, which I'd hoped to see served in IV form, is $2.)

One table is occupied with a group playing Magic: The Gathering, a card-based game that requires a 20-sided die and, evidently, the ability to make voices in the high-pitched style of Monty Python. Magic is an RPG, or role-playing game, and the guys in the cafeteria morph into elves and trolls and other sorts of fire-breathing characters. One of the participants is Adam Theuret, whose signature look includes yellow racing shades and paint-splattered black cargo pants.

Theuret doesn't even have to be here. He says he retired three years ago, the day after his 22nd birthday, when he sold a few of his high-end engineering patents (which include a mechanism for motors to store their kinetic heat after being shut off, allowing them to restart immediately in subzero conditions). Not long into his retirement, he was spending nearly $1,000 daily on gadgets just to keep his boredom at bay. After a nearby cousin asked him for a ride to Volt, Theuret—himself a lifelong avid gamer—stuck around to try it out. And he's been here since.

"I played games every day anyway," Theuret says.

About 20 of us are brought into an office, called a "lab," for which we're each given keycards. Each workspace consists of a desk and chair with a small flat-screen TV, an Xbox 360 console, and a controller. Everything—everything—is numbered. I sit down; it's the first time I've played an Xbox 360 in my life. We're barked instructions on what profile to sign up under, how to log on, and what movie to download. Although I'm slower than the other testers, I'm fine once I realize that everyone's eyes are locked to their own screen. We're assigned to test a new feature on the Xbox 360—another part of the ADD-inducing spring update that will allow the console to download movies while you're playing games.

We're given games to play and instructed to download Superman Returns. The leads don't care how well we play or what we do on the game, they just want to know if we can play for an hour—the time it takes for Superman Returns to arrive—without the whole thing crashing. At the end of each hour, we mark on a sheet if the game crashed or if there were any other major errors.

I dance like a banshee in real life, but experience problems while playing Dance Dance Revolution. In the game, you make a virtual Japanese schoolgirl dance to the beat by pressing the buttons in time to the music. It's usually played with a sensory pad you actually dance on, but here it's done with a handheld controller incapable of grasping my flawless rhythm. I circle FAIL before passing the sheet in.

Theuret comes over shortly thereafter and asks me to explain myself.

"I was awesome, but it didn't give me any points. The controller seemed off."

"But it played?"

"No matter how much I changed—"

"Didn't crash?" It didn't, and he circles PASS.

The end of my first full day of testing is also the end of the first time I've played video games for eight straight hours, by far my longest-ever stretch. The last two hours, I feel nauseous. Severely so. When I get up to leave, I nearly fall over from dizziness and a massive headache. Was it the recycled air? The Mountain Dew fumes? I get up and use the women's rest room, which is empty and spotless. Later that night, I diagnose my condition as simulator sickness—the clear sign of a rookie.

"Nintendo and Microsoft are very good places to start off and get a feel for what testing means. They're always looking for basically anyone who can come in off the street and sit down for eight hours a day, looking at their titles. They're looking for general feedback. [It trains you] how to look for bugs, how to get into the mind-set, how to get the specific information that a developer's going to need to fix those problems," says Mike Zadorojny, a quality assurance (or QA) manager.

Zadorojny, 26, spent eight years working summers in QA, including four years at Nintendo and a few at Microsoft. He's now working for a smaller Seattle video game developer, ArenaNet, which develops MMOs, or massively multiplayer online games. (ArenaNet's best-known title is the medieval-era GuildWars.) "The dev houses are more fast-paced than the other studios. It's not just sitting down and playing a game over and over again, it's a battle of wits against the developers," says Zadorojny.

During my time at Volt, I get to work in the "hardware" and "compatibility" sides of testing. But the third, and most interesting, is "functionality," which looks deep into the game itself.

"You might track shape or missions or the game-play experience," says Mark Shoemaker, a 32-year-old lead tester at Surreal Software, a local game developer. "And then you have this special kind of tester, who looks at code. They're often seen as a junior developer. They write code to break things."

But that's all at the upper levels of QA, not at the lower levels of drudge that I see.

"People think it's going to be this thrill ride and you'll have to play this fun video game all day long," says Darci Morales, a lively, 32-year-old redhead who started in Sony's QA department in Mira Mesa, Calif., 10 years ago. She's now a producer at Surreal, overseeing teams including programmers and QA. "It's really hard telling people, nine times out of 10, you're going to hate the fucking game you're working on because it's going to be busted 90 percent of the time."

Another thing that's hard telling people? Sometimes you won't be playing a game at all; you'll just be—brace yourself—turning a machine off and on.

One day at Volt, I arrive at one of the labs and discover a stopwatch and packet of spreadsheets at each of the 30 workspaces in the room. We're to turn the console on six different ways—by the main power button, by the remote control, by the controller (wireless and wired), and by pressing the eject button on the remote and the console—10 times each. In each case we start the stopwatch and record two measurements: how long it takes the console's green lights to go on, and how long for the TV screen to be fully booted up and ready for play. This takes more than an hour; small talk is allowed but minimal, given that we have an intermittent 20 seconds of downtime each minute.

After we finish and think we're done, we get another packet.

For this part, we repeat the first task—pressing the console's main power button and taking the same measurements—hundreds of times in a row. A few people finish far enough ahead to have obviously fudged their numbers, but the entire process is so far beyond mind-numbing that even the leads don't seem to care or notice. One 17-year-old is really getting into it, content to demonstrate his savvy to others.

One of the leads stomps over to the guy next to me, holding the first packet.

"Kozell, I can hardly read your name!" she screams, offering him a lecture on penmanship and explaining that she has to enter all of our numbers—written to the hundredths of a second—into a spreadsheet herself.

I now understand the allure of working for a game developer like Surreal over a place like Volt, Microsoft, or Nintendo: no hardware testing.

But it's a serious concern for Microsoft. Well before last week's admission from the company, Xbox 360s had achieved legendary status amongst users for their propensity to die, or "brick." In November, 360 owner Kevin Ray filed a lawsuit, which is aiming for "class action" status, against the company, claiming its October "update" had wrecked the machine. While testing, I see countless hard locks, but experience only one of my own—plus a few glitches that would have hurt, had I cared about saving my actual score.

"I never realized how important hardware is," says Tommy Brosman, one of my cohorts at Volt. Brosman, who describes himself as "19 and a half" years old, is a student at DigiPen, the Redmond video game college on the Nintendo campus. He was awarded a full scholarship to study "real-time interactive simulation" after designing a game in less than three months. ("I'm not a lazy bastard like most people," he says.) He has silky skin and pale hair, a large black T-shirt, and rimless eyeglasses; he rolls his eyes habitually.

Despite the rote tasks before us, Brosman says he's there because he "wants to have an edge when I get out by having that variety. A lot of testers, surprisingly, really don't care about game design or programming at all, they just want to play games." Playing games for hours on end is "a good challenge for me to try to improve my attention span."

"That's funny," I say, "because a lot of people think gamers—"

"You know," he cuts me off, index finger pointing toward the ceiling, "funny fact. You actually learn more raw analytical skills playing video games than you do from the American school system." I ask for a source, but he doesn't want to get anyone—specifically, any of his DigiPen instructors—in trouble. "But I believe it," he continues, "because it makes sense. You can probably Google it."

One day at lunch, Brosman starts drooling when fellow DigiPen student Garrett Woodley unwraps a sandwich from Subway.

"That looks awesome—but you coulda gotten double meat," says Brosman.

"I won't get double meat because of Dead Rising," says Woodley of the zombie-video game, which he says carries a strong political message about current levels of meat consumption.

The cool part of the job—the ability to test and see things before anyone else does—is the one thing you're not supposed to talk about. We were all required to sign lengthy nondisclosure agreements from both Volt and Microsoft, barring us from revealing product secrets. One of the leads fans a rumor about someone going to prison for taking a photo of an Xbox 360 console before its release, but I find no evidence of this having happened.

Security at Volt is intense, with cameras recording our moves in the cafeteria, hallways, and laboratories. Our keycards register which room we're in. We're not allowed to bring anything into the labs; a group of lockers at the end of the hallway holds boys' miscellany (cell phones and unnecessary hoodies) and my purse. One day, when the job consists of downloading and then playing free versions of some cheap, nostalgic, lo-fi games like Teenage Mutant Ninja Turtles and Bejeweled, a cell phone rings. The tester takes the call and steps outside. An uneasy silence descends upon the room. "That's probably the last time he'll ever work here," says one of the leads.

Soon, someone collects his hoodie and takes it outside; he never comes back.

"People were there because they thought it was going to be this great job," says Sean Day, referring to his time as a tester, "but you were always threatened with the prospect of losing your job at any minute. There was a guy, we were testing and sitting there, it was 10:30 in the morning, and just out of nowhere, this security guard comes up, and a supervisor comes up after him and says, 'Grab your things.' I've seen that several times. At all three jobs," says Day, who counts these inexplicable firings among the more unsavory aspects of his time in QA.

Another? Twenty-four-hour shifts. Day recalls working as a video game tester three years ago at Nintendo, where he was paid $10 an hour. After months of working on a video game, when it was on the verge of shipping, the game had to undergo "submission testing."

"We have to play it for hours without any critical stops." says Day. Then, as the job neared an end, the lead made an announcement. "He's like, 'You will be here tomorrow. And you will work for 24 hours, so come prepared.' Everyone's like, 'What the fuck? How do you do that?'

"Apparently, it's not unusual at Nintendo, because they had a whole system for it that was ready to go. They had all the breaks planned out, they had all the food ready for 2 a.m., the Krispy Kreme doughnuts ready for 5 a.m. Within the first three hours, everyone is so freaked out, drinking tons of Red Bull. Everyone completely overdoes it. People have, like, grocery bags full of candy. By hour four, you have people puking in the bathroom. But it doesn't matter what your strategy is....This is the end of the project cycle! So what does this mean? You do the 24-hour shift, and you're fired."

My own hours dry up after that frenzied day of hard locks on the precipice of the Xbox 360 spring update. Having spent two weeks on the job, I don't get a call on Friday afternoon to come in over the weekend, and a few days pass before Volt returns a call I've made, expressing interest in another shift.

Many people I meet at Volt are intelligent enough to get jobs that have more security and pay more than $8.25 an hour. "Keeping the good ones there," admits Volt's Theuret, is "one of the greatest challenges. Turnover is huge."

When I go back to Volt for lunch a week after my last shift, I'm dismayed to see that the most interesting people I met are gone. A recent UW graduate who's biding his time until his "real job with Microsoft" starts, and is taking an actuary exam, just for fun. A funny guitarist who just quit his job as a server at Johnny Rockets. A 50-something guy fully immersed in Joseph Campbell's The Masks of God, Vol. 3: Occidental Mythology. But given how difficult it is to stay on board or move up (and the number of students and stopgap seekers I met who weren't fully invested), I'm also not surprised. The ones who end up making it over the long haul are flukes of some sort who have figured out a way to afford working there.

Even those with steady jobs in QA know that their pay isn't fully commensurate with that of their fellows. "By and large, you're looking at a $10,000–$20,000 difference between doing the same job in games versus another version of software," says Shoemaker. He did contract work for Microsoft and Nintendo, collecting unemployment on and off for years, before landing his first full-time gig as a tester at Microsoft. He's now working full time at Surreal.

"This is really an industry of the haves versus the have-nots. And the testers are the have-nots," says WashTech's Courtney. The ones who want to stick it out find their way around the feast-or-famine lifestyle.

"I've got a friend who works at Microsoft for nine months, then goes to Thailand for three," says Shoemaker. "50 cents is a meal in Thailand. Now he has a little property there, a condo in Bangkok."

Some, like ArenaNet's Zadorojny, seem to view the ability to work around the schedule as a kind of intelligence in itself. "There is a hot-and-cold cycle in various seasons, at least for companies like Microsoft and Nintendo. But there's usually always work at one of the studios for contractors. We're in a very rich environment with two publishers and tons of [video game development] houses basically sitting in our backyard; there's always work around for those who really want it."

But wanting work and being able to afford the downtime between jobs are two different things.

How do some testers get chosen to move up the ranks while others fall behind? Having ovaries certainly helps. Right now the industry's workforce roughly matches the demographics of the players—almost 90 percent male—which severely limits sales growth.

"I'd really like to see the gender breakdown go to 50-50," Sony Online's John Smedley recently told The New York Times of his efforts to recruit women. "I just can't explain to a 30-year-old single male why 10-year-old girls like horses."

What that means is women who can tolerate the plethora of Monty Python jokes and tangibly uncomfortable silences will be ushered in with open arms. While many testers are unemployed for months every year, Darci Morales says she only saw four months of downtime during her first six years at Sony.

And despite how patently obvious it is at Volt that I have no idea what I'm doing—I lose track of how many times I die while playing Tom Clancy's Rainbow Six: Vegas for one hour, never coming close to leaving the opening scene of the game—I'm treated markedly well. "You're still here!" a lead exclaims during my second week, passing me a bowl of candy.

Women have the skills to be in a management-style position, says Shoemaker. "Microsoft has some of the most brilliant, socially retarded people I've ever met in my life....You need [the testers] to be articulate. And adding the articulate part really reduces the number of people who are eligible." (The more accurately a tester can pinpoint a game's problem area, the more quickly the programmers can fix the bug.)

"Most of the girls who come through have enough intelligence, so that they're the ones who get other jobs quickly or move up," Theuret says. "Stuck in the tester position are the other guys. It's just the difference between the gals and the guys."

Being stuck in the low end of QA also means you remain much more subject to the product boom-and-bust cycle. After a game ships, people lose their jobs. That's just how it is.

One night, Morales gives me a tour of the Surreal office. A bottle of Maker's Mark sits atop her desk. Shoemaker is there, too, with his QA team. It's 11 p.m. He's tired, but radiating a diffuse glow. Another staffer is on a scooter, breezing around the office. None of them are complaining about getting home.

"Because I've been in this industry so long, I've seen so many people come and go. It is very cold, and it is very hard," says Morales. "But at the end of the day, you end up having this executive who's got all of these purse strings in his pocket, saying, 'I secured this amount of money for this project. And with this amount of money, I'm going to give this much to all of the different departments.' And what really sucks is that QA is always the first thing to go. Always. It is hard, and it is sad...at the end of the day, all you can say is, 'You're getting laid off. Just look at this like it's a little vacation.'"


comments powered by Disqus

Friends to Follow