This one is for the “techies” among you who would like to know about low-level performance specs of the iOS devices. Maybe you’ve heard of Mike Ash before? He’s a Mac programmer who writes about his profession on a weekly basis and Mike’s development blog is well worth browsing for highly interesting material. Back in March 2008 he published one of the first iPhone performance comparison for common operations.

Stuart Carnie then took this performance comparison and updated it with results for iPhone 4 and iPad, and published them in his Micro-Benchmarking blog post along with the source code. If you’re interested to learn how long it takes to perform a floating point division or a 1 MB memcopy across several iOS devices, these are the tests you were looking for.

When TechnoKinetics mentioned on Twitter he was annoyed with cocos2d, I wanted to know why. It turns out it was the change away from NSMutableArray to CCArray in cocos2d which required him to update his codebase after he upgraded to the latest cocos2d version. He called it a “seemingly insignificant” change. At the time CCArray was introduced I didn’t think much about it but now I wondered, how significant or insignificant is it?

As far as I looked, there were only vague results published and the cocos2d forum thread ends with Pearapps asking for a performance test, so I believe there are no CCArray performance test results available. The conversation also lead me to this NSArray vs. C-Array performance comparison. To my dismay, it is comparing apples with oranges, as the NSArray had to wrap and unwrap the floating point values into NSNumber objects, so the results are terribly skewed in favor of the C-Array. Since this article was referred to in the CCArray cocos2d forum thread, it made me skeptical about the initial performance tests putting ccArray (the C implementation which is wrapped by CCArray, mind the case) at 3.7 times faster, or just plain text “very very faster”. I’m an engineer and not a believer, such “much faster” statements always make me very skeptical (how much is “very”?). I thought I should put in some time to generate some actual numbers.

Test Setup

I decided to take parts of Mehmet Akten‘s array test code (eg. measuring the time), then use all available array types in a way that is most common to cocos2d: storing pointers. I wanted to do a little more real-world-ish test. Internally cocos2d uses CCArray to store the children of a node, which are pointers. Without the boxing & unboxing of NSNumber we can better compare the results of how the individual array types perform. So I derived my own testing project to figure out how fast read access is between a C-Array, CCArray, NSMutableArray and NSArray by looping over each element in the array, retrieving the element (a CCLabel) and changing its tag property, just to give the loop something to do. I’m wary of the compiler possibly over-optimizing the loop if it doesn’t do anything. I’m also wary of any caches (whichever the iOS devices may or may not have) affecting the test, so I made sure that the item count (50,000) was large enough to not fit into any caches.

Code that is measured for for loops:

Using fast enumeration:

Using CCARRAY_FOREACH fast enumeration:

Sequential Read Performance

The results: a mere 10% read access performance increase for CCArray compared to NSMutableArray when using a for loop (2nd and 3rd column from the right in the chart below). And a tiny, negligible improvement when using the CCARRAY_FOREACH keyword compared to NS(Mutable)Array’s fast enumerator for(CCNode* node in array) to iterate over the array. Both these results are in the same ballpark with the C-Array, and I was pleasantly surprised to see the CCArray and NS(Mutable)Array all perform basically the same as the C-Array when using fast enumeration, with CCArray just a tiny fraction faster – exactly the same performance as a pure C-Array.

The test did also reveal an interesting effect I hadn’t expected: the NSArray’s read access performance without fast enumeration is noticeably slower than NSMutableArray. It really shouldn’t be slower, but NSArray consistently performed around 70% slower than NSMutableArray when using a for loop. I have no explanation for this anomaly. And with fast enumeration both are exactly the same speed-wise.

The Y axis is in milliseconds:

These results indicate that you do not need to use a C-Array over NSMutableArray or CCArray when the array is fixed in size and only read from. If you’re only concerned about access speed, and you use the fast enumerator respectively CCARRAY_FOREACH, you can use CCArray, NSArray, NSMutableArray and a C-Array interchangeably. The performance difference is indeed insignificant. There’s a lot of other things you can do to improve your game’s performance before you should start considering which type of array to use. But for a fixed-size array, NSArray is definetely the worst choice.

Things change of course if you need to store primitive data types like float, int, double. In that case, the C-Array will win hands down against its competitors because it doesn’t need to wrap primitive data types in a NSNumber object. But in the other cases, when you store actual pointers, by all means use the convenience collections available to you and benefit from bounds checks and a better interface to set, retrieve and iterate over objects. In this read access test, CCArray only has an advantage if you’re using a regular for loop to iterate over the elements (see the last 3 columns in the chart above). But at most it’s just 10% faster than NSMutableArray, that’s still negligible, especially if you consider the time spent running whatever code within the loop.

Add, Insert & Remove Performance

I also wanted to find out how much of an improvement CCArray is over NSMutableArray for adding and removing objects. It causes the array to expand or shrink as the number of objects increase and decrease, which creates an additional and sometimes significant overhead. That’s where CCArray should shine. Here’s the test setup:

Add object to end of array:

Insert object at first position (shifting the remaining part of the array back by 1):

Remove last object:

Remove object at index 0 (first position):

The initial finding was promising, the CCArray is 40% to 50% faster than NSMutableArray for the addObject and removeObjectAtIndex:last operations (meaning: to add or remove and item at the last position in the array) on my iPhone 3G. On the iPad as native application, the difference is even more significant with CCArray being 8 times faster than NSMutableArray for the addObject operation, and 2.2 times faster for the removeAtIndex:last operation. That’s great news!

But then came the shock when I tried insertObject:AtIndex:0 and removeObjectAtIndex:0 … I thought my iPhone had locked up and crashed, but alas CCArray is just dead slow in these cases! And by dead slow I mean: broken. Over 200 times slower, even on the iPad. Unless I made a grave mistake in my test (which I don’t see), it’s very likely a bug in the CCArray implementation. Or a design flaw. In any case, something isn’t right when it takes CCArray 53 seconds to insert or remove an object at the first index when NSMutableArray only needs 0.2 seconds to perform the same operations. If you rely on the removeAtIndex and insertAtIndex methods you should refrain from using CCArray until this problem is fixed. As far as I can tell, cocos2d rarely uses these methods internally so this issue shouldn’t have a measurable impact on cocos2d’s overall performance.

Update: @manucorporat has already fixed this CCArray performance issue (within minutes!!), it should be integrated in a new cocos2d build soon. I’ll update the performance test results soon.
Update #2: Apparently the fix was not all encompassing. It was faster on the iPad but just about 8 times, but still at least 13 times slower than NSMutableArray. On my iPhone 3G the results were worse and went down to a factor of 360. It’s being worked on. Stay tuned.

Note that this diagram is not to scale! If it were, you wouldn’t be able to make out the blue bars for NSMutableArray!

All tests were done on an iPhone 3G running iOS 4.1 with a release build of cocos2d v0.99.5 beta. The results for iPad were only glanced at to check for relative differences, see text above. All results are in milliseconds, averaged over several runs (3-5) with an iteration count of 50,000. The read speed tests were run individually, not in sequence, because running all read speed tests in sequence caused slightly different results depending on the order in which the tests were run. For example, the slower speed of NSArray was less pronounced if the tests ran in sequence.

Feel free to download my ArraySpeedTest.zip project and try to reproduce the results. Only run the tests on a device and in release builds to get comparable numbers. Let me know if you find flaws in the test setup. I’m also interested to hear if the relative speeds vary on other devices.

cocos2d Book, Chapter 7: Side-Scrolling Shooter

On August 6, 2010, in Announcements, book, cocos2d, by Steffen Itterheim

Chapter 7 – Side-Scrolling Shooter

The shooter game will be controlled with a virtual joystick using SneakyInput. The background parallax scrolling will be implemented not with the CCParallaxLayer, as it does not support endless scrolling (as far as I know, please correct me if I’m wrong). The rest will be gameplay code, mostly spawning enemies, moving them and collision tests.

The chapter will be submitted on Friday, August 13th. Yup, Friday the 13th. Scary.

Summary of working on Chapter 6 – Sprites In-Depth

I decided to rename this chapter to Sprites In-Depth as it deals mostly with Sprites, Sprite Batching (formerly known as Sprite Sheets), Texture Atlases and Zwoptex as well as general texture memory management. All the while laying the foundation for the game to be made in Chapter 7.

While working on this chapter I noticed that it’s awfully complex to create a CCAnimation class, especially if you’re not using a Texture Atlas. So I decided to illustrate how to add helper methods by adding them via a Objective-C Category to the CCAnimation class. Now you can create a CCAnimation with just one line of code, instead of around ten.

Once more I created some of my now famous doodle artworks. If anything this should show that even a programmer can do art. Or, well, at least something that vaguely resembles art.

I was a bit surprised by one thing though, and that is how little the use of the CCSpriteBatchNode contributed to the framerate in this particular case. I added all the bullets to a CCSpriteBatchNode and found only a 15% increase in performance, it went up from 45 fps to a little over 50 with all those bullets flying. I sort of expected a bigger impact from previous experiences.

Common cocos2d Performance Questions added to the FAQ

On May 19, 2010, in cocos2d, by Steffen Itterheim

I keep running into posts where users ask how fast cocos2d is, or why their particular code is slow.

I compiled a procedures checklist that you should go through before reporting performance issues.

I also tried to explain why the question of how fast cocos2d is has no real-world relevance.

Then i added a link to All-Seeing Interactive’s cocos2d Performance Tips as this is such a great post! It covers a lot of topics:

  • Profiling with Instruments
  • Testing on different devices
  • Textures and Texture Atlases
  • Speeding up loading times
  • Reducing memory usage
  • Flipping textures
  • Pixel formats
  • PVRTC (Texture Compression)
  • CCSpriteSheet
  • Pre-render programatically generated textures
  • Avoid using atomic accessors
  • Read the best practices documentation

As always i hope you find these additions help- and useful.

Tagged with:  
Page 2 of 212