Hi Everyone,

My name is Peter Hofstee, I am one of the architects of Cell, primarily responsible for the synergistic processor. I have given quite a few technical talks on Cell, but in this blog I’d like to talk a bit about some potentially further reaching implications of the next generation of gaming technology.

For me the most significant aspect of this next generation of technology is the transition from consoles and games designed primarily for stand-alone use, to systems and games defined by broadband connectivity and interaction in real-time within a virtual 3-D environment.

The ability for large groups of people who are only virtually co-located to interact in real time I think is profound and is likely to have an impact far beyond gaming. It is likely to change education with virtual classrooms that can become better than real ones, for example by allowing everyone in the class to get an instant good view of someone asking a question. It is likely to change research and collaboration by significantly lowering the barriers to organizing an international conference or increase the value of a smaller meeting. It is likely to change the way sports are watched by allowing personalized points of view and audience participation. It will likely change business, with new opportunities for advertising and new ways for people to respond to advertisements and new ways for businesses and consumers to interact.

A key enabling technology that needs further development is the creation in real time of 3-D models of the real world (just like cameras construct a 2-D model). This brings world-data into the format that games and CG movies use. Today the process is in its infancy, for example requiring actors to put on black suits with white markers so the segmented motion can be tracked. In the case of medical data, x-ray or MRI data need a considerable amount of off-line processing to create a 3-D image that isn’t even based on objects. Still imagine for a moment that true 3-D recording technology to did exist and classrooms or sportsgames could be captured this way. Once the data is captured in this format, it can be manipulated, for example by adding other participants to the modeled classroom or conference. Also, just as in games, it allows an infinite choice of camera locations for final display as in the sportsgame example. Applications are endless; a surgeon can see an overlay of 3-D data from an MRI scan to what is visible to the eye, and may prefer this type of virtual view even when not operating remotely. The technology will improve virtually enhanced displays, not just for fighter pilots and airplane mechanics, but for car drivers and mechanics and plumbers, and electricians etc. etc. etc. I think this type of technology would have a lot more impact than conventional 3-D displays that offer only very limited freedom in the choice of a point of view, though of course 3-D displays and 3-D modeling and distribution are mostly orthogonal technologies and can be used in combination.

While the goal of full real-time 3-D recording will take some time to achieve, in many cases data that has been collected off-line addressing the static aspects of the scene that is being modeled can make the problem tractable. Examples include merely recording the positions and orientations of pre-modeled racecars on a pre-modeled race track to allow gamers to participate in the race, or recognizing the players and their body configurations in a soccer game to apply it to detailed 3-D models of the players that were previously built to allow viewers to control how they watch the game. Another example is the camera recording your body position and motion and applying it to a virtual character in a game. An example where the data is (almost) static is the collection of 3-D models of the world itself, in particular cities, to allow real-time exploration in 3-D. Perhaps an ultimate example is world modeling with enough detail to make a virtual hike worthwhile. Several of these exist already, what I have sketched is far from an all-or-nothing proposition. Most likely it will follow the paradox of recent technological progress (think MMOGs); viewed up close the rate of adoptions seems to be very slow, and yet, all of a sudden it seems to be there in a big way.

One last comment. For a field to truly drive the development of new technology, there have to be significant problems that cannot yet be solved, and drive the value people find in new generations of technology. This is now case with gaming and microprocessors, and the one class of problems I have described here a little bit is only one of many examples.

We have conducted experiments in our lab where Cell servers were used to feed wireless handheld visual devices (PDAs). We found that we could software render and compress hundreds of frames per second using only a single Cell processor. The limiting factor became how many compressed frames we could push across the 802.11b wireless link. The prototype handheld system encoded the user’s inputs (GPS, Digital compass, Joystick, etc.) and shipped them to the Cell server where the software renderer rendered the correct 3D view of the world, compressed the resulting 2D image and delivered it back to the handheld client. With simple JPEG like compression and 802.11b wireless we were able to deliver 15 frames/sec to the handheld device. Given this result I believe that with Cell SMP servers and more aggressive compression, like H.264, persistent world 3D online games could be played with very low power handheld clients. These handheld clients would not need power hungry 3D GPUs or large amounts of memory, instead they would only need to decompress and display streams of 2D images. Why send megabytes of 3D geometry to handheld gaming devices for storage and processing and then constantly update it every frame when the server can compute and send the finished 5KB frame?

Hi,
I’m curious as to what people thing about the importance of realistic physical simulation in next-gen gameplay. Sony was very interested in improving gameplay with better physics in the next gen console. We conducted an assessment of rigid body dynamics on the Cell and were very pleased with the results, but I think rigid body is only the tip of the iceberg. Simulation of a variety of material such as cloth will be enabled by Cells compute power and memory subsystem. I think this may provided added dimensions to gameplay e.g. grabbing of uniforms when playing sports games. The recent post in Gamasutra seems to suggest that Sony believes phyiscs is important enough to include int he PS3 SDK.

Sony Announces Unreal Engine 3, Havok, NovodeX For PS3 SDK

Officials from Sony Computer Entertainment, Inc. have announced at a Japanese press conference that the company has entered into strategic licensing agreements with Epic Games, Havok, and Ageia to include an evaluation version of Unreal Engine 3, the Havok physics and animation engines, and the Ageia PhysX SDK (also known as NovodeX) as part of the PlayStation 3 software development kit (SDK).

In each case, Sony has sublicensed the product to enable the company to provide access to this complex and necessary next-generation middleware in an easier fashion, and each of the above-mentioned pieces of middleware will now become available for the PlayStation 3 developer community together with the Software Development Kit for PlayStation 3.

Frontline technical support for the products will be provided by SCEI, particularly important for many Japanese developers because the Western-developed middleware products do not necessarily have a major Japanese technical support presence, and a large and respected corporation like Sony sublicensing the product will imply a certain stability and reliability for those Japanese developers who might want to go on to license the software, in a territory where the growth of middleware has been relatively sluggish compared to the West.

“We’re very excited to be able to work closely with SCEI to provide our complete end-to-end solution to PS3 developers,” said Tim Sweeney, CEO, Epic Games. “Now, every PS3 developer will be able to try out Unreal Engine 3 and be productive on their very first day of PS3 development.”

“This agreement between SCEI and Ageia marks a significant landmark in the development of advanced physics-enabled games,” said Manju Hegde, founder and CEO of Ageia Technologies, Inc. “The capabilities of the Ageia PhysX SDK combined with the power of the Cell architecture will give developers the tools necessary to introduce dynamic physical properties within games that will leapfrog game interactivity as we know it today.”

“We are excited and proud to partner with SCEI to provide our physics and animation technology with the PS3 development kit,” said David O’Meara, CEO of Havok. “Havok’s technology has been used by major developers all over the world in more than a hundred games. Our agreement with SCEI opens the door for all PS3 developers to use Havok to create spectacular game worlds with incredibly realistic special effects and interactive game characters.”

First let me introduce myself. My name is Barry Minor and I have been on the Cell processor project since the fall of 2000. Before Cell I developed 3D graphics processors for IBM and Diamond under the FireGL brand.

Cell has been a great project and from the beginning we have focused the architecture around graphics and video processing. Once we had the architecture locked down I started writing a real-time ray-caster for Cell optimized around height-maps. As the design of the renderer progressed it became very apparent that Cell was not just good but stellar at such tasks. We found that we could ray-cast 720P images (1280×720) of complex scenes at frame rates greater than 30 frames/sec with a single Cell processors (50x a G5 VMX processor) and double that rate with a two way SMP configuration. Cell has the potential to move a new class of previously off-line rendering algorithms to real-time speeds thereby pushing us beyond polygon rasterization.

I think you will initially see hybrid approaches where backgrounds are rendered with ray-casting and foregrounds are rendered with GPU rasterized polygons but with the focus of people like Philipp Slusallek full blown real-time ray-tracing on Cell will be a reality.

INTERESTING NEWS FROM THE WIRES:

We discussed games as an eBusiness portal to the web and to eCommerce transactions. We discussed product placement just like movies. We had always talked about how this would all be possible. Well someone finally plans to put video and audio adverts into games. Has anyone seen anyone else announcing this type of insertion in games? Are they the first to actually do this? How will gamers react to appropriate or even inappropriate adverts in games? “This next challenge in Metroid is brought to you commercial free by CitiBank — the bank for ……”. “Sword of Cimarron: 4 talens, Hyper-Cavitating Death Ray: 9 talens, Slaughtering your friends: Priceless”.

Seems that they only want to make the games more realistic.
Now if they could make the games cheaper because of sponsorship, well ……
Thoughts? Comments? Rants?

From Dean Takahashi, San Jose Mercury News,

Gaming gets boost in realism, revenue from video, audio plugs:

As video games become more realistic, so are the advertisements that run
within them.

Massive, a New York game ad technology firm, plans to announce today that it
can run video and sound ads in video games.

Aimed at enhancing realism, those ads will be built into the scenery of the
game. For instance, if a game character passes by a TV set in the setting of
the game, the TV could broadcast a 15-second video. The same goes for other
objects in the game such as electronic billboards, stereos or radios.

Read the entire article here and on Dean Takahashi’sblog

EXCERPT OF NEWS ON THE WIRES (sources are The New York Times and Reuters)

The New York Times
July 25, 2005
Microsoft to Launch Xbox 360 in Japan, Europe in ‘05
By REUTERS

Microsoft’s new white-and-silver concave console will feature three
PowerPC-based microprocessors joined onto one chip to deliver powerful
computing and advanced graphics, as well as a detachable 20-gigabyte hard
drive and a wireless controller.
================================ EXCERPT ENDS ========

This news may well surprise many in the Industry. Looks like Microsoft is going to launch almost simultaneously on all 3 fronts. They currently have poor marketshare in Japan but continue to make good progress in Europe and the USA.

They are launching ahead of their console rivals in all 3 market areas — what do you think of this? Is this a master stroke that will vault them to the top of all 3 geographies for the next gneration? Are they being overly ambitious? Will gamers delay their buying decisions until they see what Microsoft’s competition comes up with? Will they use a Rolling Stones song at the launch? If so, which one?

Let’s have your comments, thoughts and wild *wild* guesses. Thanks. Mike.

** self-imposed edit

There was an interesting article in last month’s IEEE Spectrum on 3D displays. The article discussed three basic types of 3D displays. Stereoscopic, swept volume, and dithered slice (my words for the types). Relatively speaking the computational requirements to support the different types are quite different. The stereoscopic type requires 2X the graphics performance as the scene is displayed from two slightly different viewpoints. The swept volume type could take 1000X of the graphics performance as the third dimension is displayed. Lastly the dithered slice display which might have 16 layers in the display with computation needing to be done for each slice.

The primary issue will probably be the end user cost of the display technology. It seems to me that only the stereoscopic and dithered slice display approaches might be possible for the new generation of game machines although 16x or so graphics performance might put a strain on these machines. It’s more likely that there would be at least the opportunity to support stereoscopic 3D assuming a low cost hardware solution was available.

I suspect that it may be a few more years before the dynamics of cost effective displays, compelling 3D content (although most game format might support stereoscopic and dithered slice display technologies), and game and graphics performance align.

This week Forbes published a report titled “Do Game Publishers Ignore Piracy?” ( http://www.forbes.com/digitalentertainment/2005/07/18/videogame-piracy-worldwide-cx_ld_0718piracy.html?partner=yahootix ).

The article highlights the similarities between the music and video business and the games industry, and points out some very relavent concerns games publishers need to address and be very concerned.

Piracy has been a huge issue for Music and Movies, whether it is via physical distribution or online/ file sharing. Piracy is definitely a growing concern for games publishers, as the price of technology continues to drop piracy becomes easier to achieve by less skilled individuals.

One thing that I think is grossly overlooked in the article ;however , there is one HUGE difference between games and other media. Games are written with the intent of being run in an interactive environment on a ‘intelligent’ device (console, pc, etc). Since the expectation is the user will be interacting with the game, security can be approached at a multitier approach; the hardware tier(console, pc, etc), the software tier, and the user tier .

The multiple tiers allow for many more options to address piracy than other media forms, from basic product key to more creative approaches such as “Fade” , developed by CodeMasters and MacroVision - http://radio.weblogs.com/0105910/2003/10/10.html

I wasn’t born yesterday so I fully understand and expect hackers and pirates to become more sophisticated along with the protections, however the economics of piracy diminishes greatly the longer the game is on the market. So if new technologies can simply delay the ineveitable piracy by 60-90days perhaps that is enough to discourage all but the most driven pirates.

Let me be the first to welcome you to GameTomorrow, a new collaborative weblog focusing on the future of gaming, and how it will impact our lives beyond the confines of the console or the PC. Our contributors are IBM leaders who are involved with games and game-related technologies — whether it’s inside the consoles, within massively multiplayer online environments or the business of gaming. They’ve come together on this blog to spark more discussion about the long-term implications of gaming and third-generation user interfaces.

Our starting line up:

• Ron Eisses is a business development executive for IBM’s Digital Media group.
• David Laux is a global executive for games and interactive entertainment on IBM’s emerging markets team.
• Michael Ash is a business consultant in IBM’s Media & Entertainment group.
• Bruce D’Amora works on visual and geometric computing at IBM’s Watson Research Lab.
• Jim Kahle is an IBM Fellow and leads the Cell processor development team.
• Kathy Papermaster is the director of IBM’s STI design center.
• Ted Maeurer manages the Cell software development center.
• Barry Minor designs 3-D graphic software in IBM’s Austin development lab.
• Mike West is a multimedia architect for IBM’s pervasive computing team.
• Ilan Spillinger is an IBM Distinguished Engineer and is director of game and low-end processors.
• Jeff Brown is a Distinguished Engineer in IBM’s Engineering & Technology Services group.
• Albert Wong is an avid gamer and an IT architect with IBM’s Linux and Grid services team.
• James Snell is a longtime gamer and works on IBM’s software standards team.

As an avid gamer and specialist in emerging software technologies, my personal contributions on this blog will be an exploration of gaming experiences and how such important trends as social networking software will impact the way we communicate and play online. Other contributors will focus on the technologies, like the IBM Cell processor, that power the gaming experience. Still others will focus on the business of gaming. Given the broad range of experiences and points-of-view of the contributors, the discussions on this weblog are expected to be diverse and lively. However, it’s not a true discussion unless our readers get involved. We want to hear from you: about your experiences; about where you think gaming technology is going; about how these important technologies are impacting lives.

This is going to be fun.

- James Snell, XBox Live Gamertag “Diamond Lane”