Although Apple has included DDR3 1067 MHz memory with the new iMacs, but the built-in DDR3 memory controller officially supports 1333 MHz memory. Someone has now tried 1333 in his new Core i7 iMac and yes 1333 MHz memory works, with some caveats. The biggest issue is that the memory must have a cache latency of 7 - CL7 (like the 1067 MHz memory Apple provides) to work properly. However, a lot of 1333 MHz memory out there is the slower CL9 speed. CL9 modules can cause the iMacs to become unstable. Furthermore, it seems the speed boost from 1333 MHz RAM is at best small, even in synthetic memory benchmarks. Presumably, the lack of a third memory channel (as in the Mac Pros) limits overall memory performance.
So, save your money and get 1067 MHz RAM, to go along with the 1067 MHz RAM Apple provides. 4 GB may be fine for basic usage, but 8 GB is the sweet spot for most more advanced users from a price perspective, as the additional 4 GB is only US$85 for name brand RAM at some large retailers.