Wave Issue 9610 10/11/96 Article 6-01

***Gemini Announces First Benchmarking Results
by John Latta

The 3D industry has been looking for 3D benchmarking results with both
anticipation and trepidation - and the first results have arrived
courtesy of Gemini. Posted on their home page are a number of surprises
with more to come. First we describe the tests and then summarize the
results based on the posted results and an interview with John Archdeacon
of Gemini. WAVE readers are encouraged to visit the home page for more
details.

There are two tests: gvr and gvf. Each of these tests determine the real-
time frame rate performance of two types of simulations. gvr is a race-
car environment with medium pixel depth and scene complexity (depth
complexity is greater than 2 but less than 4 on average and peak up to 8)
while gvr is flight simulation-oriented which emphasizes a range of
transformation loads (the depth complexity is greater than 1 but less
than 3). All test results are referenced to SGI Onyx Reality2 system
which Gemini calls an re2stone (TM) of 1.0. Performance is based on frame
rate and most tests are in single frame buffer mode. Thus, for systems
which have a higher average frame rate the reference platform will rank
proportionally faster relative to the reference platform. A common output
format was chosen of 640 X 480 VGA mode. In general, those features which
provide the best image quality were turned on that was supported by the
system under test. If anti-aliasing was available it was used and the
best filtering was used (in all cases MIP Mapping except one which used
bilinear). The benchmark binaries are available from the Gemini web site
for Windows 95 and Windows NT OpenGL, Windows 95 3Dfx Glide, Windows NT
Real 3D Pro and SGI IRIX OpenGL and DEC UNIX and Alpha OpenGL.

Graphics System gvf re2stone rating

SGI Onyx iReality 3.21
3Dfx 2220-SLI 1.03
Real 3D Pro 1000 1.00
SGI Onyx RE2 1.00
SGI Maximum Impact 0.92
SGI High Impact 0.63
3Dfx 2220 0.5
3Dfx 2200 0.47
Intergraph TDZ/GLZ5 0.23
E&S Freedom 3000 0.05

Graphics System gvr re2stone rating

SGI Onyx iReality 2.52
Real 3D Pro 1000 1.82
3Dfx 2220-SLI 1.51
SGI Maximum Impact 1.30
SGI Onyx RE2 1.00
3Dfx 2220 0.67
3Dfx 2200 0.57
SGI High Impact 0.56
Intergraph TDZ/GLZ5 0.47
E&S Freedom 3000 0.11

The 3Dfx systems stand out. The 2220-SLI is actually two boards which
uses 2 PCI slots. Each board does every other scan line and the effective
pixel fill rate is 90Mp/s. The 2220 has two texture memories of 2MB each
which allows this version to do MIP-mapping. The 2200 has only a single
board with 2MB of texture and can only do bilinear filtering. An
important caveat in the 3Dfx tests is that the host CPU is used for the
geometry calculations because the chip does not have this capability. In
the case of the 2220-SLI the host is a Pentium Pro 200 and it was nearly
100% utilized when running the tests.

It has been Gemini's observation that the primary bottlenecks in the
image generation pipeline are pixel fill rate first and then geometry
transformations. For example, the SGI InfiniteReality gains considerably
over the older RE2 through its use of ASICs for geometry calculation as
opposed to the Intel i860 processors in the RE2. These same
considerations apply to the 3Dfx where this chip is totally dependent on
the host CPU. Although not yet quantified Gemini has seen impacts of the
performance of the OpenGL implementation in the ratings. For example, it
appears that the software implementation of OpenGL on the E&S Freedom
3000 could be partially responsible for the poor performance (note the
host CPU utilization was only 3% in these tests). Gemini cautions that
the version of OpenGL used, including the versions being supported by
Microsoft, may play a significant role in the ratings. In addition it has
been Gemini's observation that 3Dfx's Glide API is quite fast and
efficient. Although Gemini is not currently reporting on the version of
OpenGL used in the tests this is likely to be added in the future.

One of the most difficult parts of the 3D benchmarking is an assessment
of Image Quality (IQ). Gemini does not address this issue other than try
to make the tests between the cards and the features which impact IQ as
close as possible. For example, both the Infinite Reality and RE2 use
what is described by Gemini as full scene anti-aliasing (sub-pixel
processing). In the tests they use 2X2 sub-pixel matrix for each pixel.
Only the Real3D Pro 1000 uses anti-aliasing which is described as
"automatic polygon edges." Gemini claims that the impacts of turning on
and off anti-aliasing would only be seen on the polygon edges, where
there would be a major IQ difference, but on the interior of the polygons
the IQ would be the same. In spite of the limitations of subjective
assessments at WAVE we feel that a subjective evaluation of the IQ would
be valuable.

Another interesting data point which would be valuable to report on would
be the frame rate statistics during the tests such as max and min frame
rates and a correlation with the pixel depth complexity. This would allow
for a better assessment of the overall frame rate performance of the
system. Gemini has observed that there appears to be a similar
correspondence between the frame rate variations during the tests on each
of the systems.

Many chips/cards are not present on the list. Benchmarking will get
interesting when results from the Mitsubishi Pro 3D, Real 3D R3D/100,
3DLabs 500TX, and Intergraph Intense3D are posted.

Benchmarking in 3D will be a high stakes game - with the best results
defining "winners." Many factors go into determining the system
performance and as Gemini has seen that it is not only the host
environment but the API which has a significant impact. And these tests
the implementation of OpenGL apparently played a role in the rating also.
To be addressed is how IQ can be factored into the test evaluation.

www.gemtech.com/rwb