Saturday, June 03, 2006

MacBook & G5 iMac benchmark Part 2: Photoshop and Exposé

Here is our second set of benchmarking tests on the iMac G5 2.0 vs. the MacBook Core Duo 2.0. The first set of benchmarks were detailed in this article.

We tested Photoshop CS2 using the Photoshop benchmark, which consists of a 3.42 MB file run through a battery of scripted Photoshop actions.

Not surprisingly, the iMac easily bests the MacBook, since Photoshop is running natively on the single-core G5 and under Rosetta on the dual-core MacBook. The difference was about 25%.

What was interesting to note though was the fact that although the MacBook had 2.0 GB of RAM, it paged out to the hard drive much more often than the 1.0 GB G4 Power Mac Cube and the iMac. In fact, that 1.5 GB iMac G5 never paged to disk at all. However, further checking revealed the iMac had only 50% of it's 1.5 GB (735 MB) allocated to Photoshop, whereas the MacBook had 70% of its 2.0 GB (1374 MB) allocated. I repeated the test on the MacBook with it also set to 50% (981 MB) and curiously, all the pageouts disappeared, but the time was almost exactly the same, this time taking 3 seconds longer. I'm not sure exactly why I got the results I got, but suffice it to say that Photoshop on the MacBook under Rosetta isn't particularly fast, with the test times with both settings both noticeably longer than the iMac's.

The second benchmark I did was simply to test Exposé performance with a lot of windows open. Both the iMac and the MacBook were connected up via VGA to a Dell W2606C LCD TV with a 1360x768 screen resolution. All other windows were closed, but multiple Safari windows measuring 1013x772 in size were opened, until Exposé began to lag.
Exposé stutter threshold (Higher is better)
iMac G5: 31 Safari windows
MacBook: 15 Safari windows
The fact that the iMac could open around twice as many windows before lagging sounds about right considering that the iMac has 128 MB video RAM with its Radeon 9600, twice the 64 MB on the MacBook. On Apple's MacBook page, it hints that GMA 950 may be able to use more than 64 MB in OS X. It says "Memory available to Mac OS X may vary depending on graphics needs. Minimum graphics memory usage is 80MB", but in tests the Apple System Profiler continued to state a 64 MB video RAM allocation, and the Exposé lag persisted after repeated Exposé activation. Note though that with just the MacBook screen (without the additional external screen), Exposé could handle many more open windows before it began to stutter. Basically what this test means is that ideally for a dual-monitor 2D setup, 128 MB is preferred if you wish to maintain smooth-as-butter GUI eye candy. For a single 1280x800 screen however, 64 MB is fine. This result concurs with the Exposé results from the AnandTech Power Mac G5 review from a couple of years ago. In that test they felt that the 64 MB Radeon 9600 was insufficient for good dual-screen Exposé, but a video card with more memory worked great.

2007 may prove to be an interesting year for the MacBook. Photoshop will go Intel native in 2007. Also, by then the MacBook will most likely have an updated GPU, and it's possible that Apple could change the OS X GPU shared memory setting to include a 128 MB (or more) option, at least in OS X 10.5 Leopard. It's true that for most practical non-3D work, the MacBook is very responsive already, but the new features could make the MacBook an even more complete all-in-one small laptop.

Wednesday, May 31, 2006

Pro Tools 7 now supports Intel Macs

Pro Tools, one of the flagship music applications on the Mac platform, has finally gotten support for Intel Macs. Intel Mac support was announced previously, but the downloads for Pro Tools LE 7.1.1 & Pro Tools M-Powered 7.1.1 were finally posted at Digidesign's website yesterday.

Interestingly, it is not a universal binary. The 7.1.1 update adds Intel Mac support, but only works on Intel Macs. Those with PowerPC Macs will continue to use version 7.1, which has identical functionality.

This is an important milestone for Apple, since previous versions of Pro Tools did not work on Intel Macs at all, even under Rosetta. Other major hurdles for Apple to overcome in this Intel x86 Mac transition are the universal binaries of Adobe Creative Suite 3 (which is not due until 2007), and QuarkXPress 7 (which is due in the second half of this year). Fortunately, these at least work in Rosetta, albeit slowly. Microsoft Office is also a flagship Mac program which is still running under Rosetta, but fortunately, speed in Office is not a critical issue for most people. Office performance on Intel Macs is slow, but it remains quite usable for most.

Sunday, May 28, 2006

MacBook & G5 iMac benchmarked: Handbrake, Aperture, and much more

Now that I've had a few days with my MacBook with new RAM installed, I've been able to properly compare the performance in various applications on my my MacBook and my iMac G5. My MacBook has 2 GB of PC2-5300 RAM, an 80 GB 5400 rpm 2.5" drive, Intel GMA 950 graphics (using 64 MB system RAM), and a Core Duo 2.0 CPU. The iMac has 1.5 GB of PC-3200 RAM, a 500 GB 7200 rpm 3.5" drive, Radeon 9600 graphics (with 128 MB dedicated video RAM), and a G5 970FX 2.0 (single-core).

I was shocked and overjoyed to see just how well the MacBook performs in one of my favourite applications, Handbrake 0.71, which I use to convert DVD clips to H.264 for archival purposes. I tried encoding a 6:03 minute 4:3 640x480 clip of my friends and me jumping out of a plane (seriously), at 1000 Kbps main profile H.264 with video deinterlacing, and 128 Kbps 44100 Hz AAC audio.

The above graph shows what Handbrake recorded as the average frame rate for the first pass of a two-pass encode. The iMac took just a shade over 25 minutes, whereas the MacBook took only 7 minutes and 15 seconds! In other words the MacBook has 350% the performance of the iMac, despite having only 200% of the CPU GHz.

Similarly, Aperture 1.1.1 shone on the MacBook compared to the iMac, despite the fact that Aperture is fully supported on the iMac and not on the MacBook.

The Aperture test consisted of a web page export of a project I shot a few months ago at the local market. The pictures were mildly edited Canon 20D 8 Megapixel RAW files, exported to an HTML webpage from within Aperture. Despite the fact that the GMA 950 in the MacBook is terrible at 3D work, I suspect most of the export time (including RAW conversion) is CPU based, and of course the MacBook has CPU performance in spades. However, one should realize that my Aperture test included mainly only lightly edited pictures, which may not tax the GPU as much as heavily edited pictures. If one plans to use a laptop as a primary photo editing machine with Aperture, a MacBook Pro is thus recommended because GPU performance is still very important for optimal overall performance. Furthermore, a larger screen is easier to work with when using Aperture.

In Cinebench 9.5, the MacBook again flexed its CPU muscles.

The MacBook was more than twice as fast at CPU rendering. However, things changed when 3D GPU performance became important:

The MacBook did fine in the software OpenGL lighting test, but that's a CPU-based benchmark. When it came to the hardware OpenGL lighting test however, the MacBook fell far behind the iMac G5 with its Radeon 9600. The reason for this is that the MacBook's GMA 950 does not have hardware support for this function, and must therefore emulate it on the CPU. It actually did a little bit worse in its faked hardware score than its software score.

All of the above benchmarks are with applications that are native on both Intel and PowerPC, and which are able to leverage dual CPU cores. What about applications under Rosetta (which translates PowerPC instructions to x86 on-the-fly), and Rosetta on applications that are not well dual-optimized? Not surprisingly, the MacBook does extremely poorly in these types of applications.

One of the flagship applications on the Mac platform is Microsoft Office. I use PowerPoint 2004 a fair bit, and in this test I exported a 30 MB PowerPoint presentation (with lots of JPEG images) to a web page. In this particular test, the iMac is almost 3 times as fast as the MacBook. It took almost 2 minutes to do something that took only 41 seconds on the iMac. One can feel the difference in daily usage too, since application and file loading in Office on the MacBook as well as various other functions feel somewhat sluggish. Office is still quite usable on the MacBook though.

Finally, just for kicks, I ran Skidmarks GT 4.0.1, a PowerPC application found in the developer tools and used for quick and dirty PowerPC benchmarking:

These scores seem to parallel the PowerPoint scores, with the iMac being roughly 2X-4X as fast as the MacBook in the non-vector benchmarks. Luckily, applications like those in MS Office aren't heavily dependent on Altivec for their functionality. While Rosetta does seem to support Altivec, it is very, very slow at it as would be expected under emulation.

Overall, the MacBook is an excellent machine for its pricepoint. It's a budget machine but with pro-level CPU power. The low end GPU can be problematic for some, but for many others it's not an issue at all. Rosetta is still a concern, but it's a concern not just with the MacBook, but with all Intel Macs. Fortunately, most of the big applications should be Intel native by next year, rendering the Rosetta issue moot. In the meantime, I'll be encoding quite a bit of H.264 video on my new laptop with Intel Inside™.