In Depth iOS & Cocos2D Performance Analysis with Test Project

On November 17, 2011, in idevblogaday, by Steffen Itterheim

I took Mike Ash’s performance measuring code from 2008 with the improvements made by Stuart Carnie in early 2010 and turned that into a performance measuring project for 2012.

I know it’s still 2011, consider this a forward-looking statement. In any case, the test project is available for download, ready to run, includes Cocos2D v1.0.1 and is relatively easy to modify for your own needs. This project is also available on my github repository where I host all of the iDevBlogADay source code.

Since numbers are so dry and hard to assess, you’ll find the rest of this post garnered with charts and conclusions based on the results obtained from iPhone 3G, iPod 4 and iPad.

Continue reading »

Unofficial Cocos2D API Reference

On January 25, 2011, in cocos2d, support, by Steffen Itterheim

I’m now hosting an unofficial Cocos2D (for iOS & Mac OS) API Reference, accessible via the Knowledge Base tab.

What’s so “unofficial” about it?

My version of the API reference includes previously undocumented classes, some are documented but just not in the official documentation, others are completely undocumented but at least now you know they’re there. Currently, there is no additional documentation added other than what’s in the official cocos2d source code files, and as far as I can tell nothing is missing. If there is, please let me know.

Since at least the release of v0.99.5 about a month ago important classes like CCLayer, CCArray and CCDirectorIOS have been missing from the official API reference. Did anyone even take notice, or was it just me? I hope it will be fixed soon and the missing classes added back to the official API reference.

I’d like to call it “unofficial” just to make sure everyone gets the idea that I’m not involved in the development of Cocos2D in any way, and my version of the API reference may contain crucial omissions or mistakes as well.

Angry Birds Tutorial causes slowdown

Current and well-deserved cause of the cocos2d-iphone.org slowdown is the SpaceManager Game Tutorial with source code by the mobile bros. LLC. The tutorial shows you how to make an Angry Birds style of game.

When TechnoKinetics mentioned on Twitter he was annoyed with cocos2d, I wanted to know why. It turns out it was the change away from NSMutableArray to CCArray in cocos2d which required him to update his codebase after he upgraded to the latest cocos2d version. He called it a “seemingly insignificant” change. At the time CCArray was introduced I didn’t think much about it but now I wondered, how significant or insignificant is it?

As far as I looked, there were only vague results published and the cocos2d forum thread ends with Pearapps asking for a performance test, so I believe there are no CCArray performance test results available. The conversation also lead me to this NSArray vs. C-Array performance comparison. To my dismay, it is comparing apples with oranges, as the NSArray had to wrap and unwrap the floating point values into NSNumber objects, so the results are terribly skewed in favor of the C-Array. Since this article was referred to in the CCArray cocos2d forum thread, it made me skeptical about the initial performance tests putting ccArray (the C implementation which is wrapped by CCArray, mind the case) at 3.7 times faster, or just plain text “very very faster”. I’m an engineer and not a believer, such “much faster” statements always make me very skeptical (how much is “very”?). I thought I should put in some time to generate some actual numbers.

Test Setup

I decided to take parts of Mehmet Akten‘s array test code (eg. measuring the time), then use all available array types in a way that is most common to cocos2d: storing pointers. I wanted to do a little more real-world-ish test. Internally cocos2d uses CCArray to store the children of a node, which are pointers. Without the boxing & unboxing of NSNumber we can better compare the results of how the individual array types perform. So I derived my own testing project to figure out how fast read access is between a C-Array, CCArray, NSMutableArray and NSArray by looping over each element in the array, retrieving the element (a CCLabel) and changing its tag property, just to give the loop something to do. I’m wary of the compiler possibly over-optimizing the loop if it doesn’t do anything. I’m also wary of any caches (whichever the iOS devices may or may not have) affecting the test, so I made sure that the item count (50,000) was large enough to not fit into any caches.

Code that is measured for for loops:

Using fast enumeration:

Using CCARRAY_FOREACH fast enumeration:

Sequential Read Performance

The results: a mere 10% read access performance increase for CCArray compared to NSMutableArray when using a for loop (2nd and 3rd column from the right in the chart below). And a tiny, negligible improvement when using the CCARRAY_FOREACH keyword compared to NS(Mutable)Array’s fast enumerator for(CCNode* node in array) to iterate over the array. Both these results are in the same ballpark with the C-Array, and I was pleasantly surprised to see the CCArray and NS(Mutable)Array all perform basically the same as the C-Array when using fast enumeration, with CCArray just a tiny fraction faster – exactly the same performance as a pure C-Array.

The test did also reveal an interesting effect I hadn’t expected: the NSArray’s read access performance without fast enumeration is noticeably slower than NSMutableArray. It really shouldn’t be slower, but NSArray consistently performed around 70% slower than NSMutableArray when using a for loop. I have no explanation for this anomaly. And with fast enumeration both are exactly the same speed-wise.

The Y axis is in milliseconds:

These results indicate that you do not need to use a C-Array over NSMutableArray or CCArray when the array is fixed in size and only read from. If you’re only concerned about access speed, and you use the fast enumerator respectively CCARRAY_FOREACH, you can use CCArray, NSArray, NSMutableArray and a C-Array interchangeably. The performance difference is indeed insignificant. There’s a lot of other things you can do to improve your game’s performance before you should start considering which type of array to use. But for a fixed-size array, NSArray is definetely the worst choice.

Things change of course if you need to store primitive data types like float, int, double. In that case, the C-Array will win hands down against its competitors because it doesn’t need to wrap primitive data types in a NSNumber object. But in the other cases, when you store actual pointers, by all means use the convenience collections available to you and benefit from bounds checks and a better interface to set, retrieve and iterate over objects. In this read access test, CCArray only has an advantage if you’re using a regular for loop to iterate over the elements (see the last 3 columns in the chart above). But at most it’s just 10% faster than NSMutableArray, that’s still negligible, especially if you consider the time spent running whatever code within the loop.

Add, Insert & Remove Performance

I also wanted to find out how much of an improvement CCArray is over NSMutableArray for adding and removing objects. It causes the array to expand or shrink as the number of objects increase and decrease, which creates an additional and sometimes significant overhead. That’s where CCArray should shine. Here’s the test setup:

Add object to end of array:

Insert object at first position (shifting the remaining part of the array back by 1):

Remove last object:

Remove object at index 0 (first position):

The initial finding was promising, the CCArray is 40% to 50% faster than NSMutableArray for the addObject and removeObjectAtIndex:last operations (meaning: to add or remove and item at the last position in the array) on my iPhone 3G. On the iPad as native application, the difference is even more significant with CCArray being 8 times faster than NSMutableArray for the addObject operation, and 2.2 times faster for the removeAtIndex:last operation. That’s great news!

But then came the shock when I tried insertObject:AtIndex:0 and removeObjectAtIndex:0 … I thought my iPhone had locked up and crashed, but alas CCArray is just dead slow in these cases! And by dead slow I mean: broken. Over 200 times slower, even on the iPad. Unless I made a grave mistake in my test (which I don’t see), it’s very likely a bug in the CCArray implementation. Or a design flaw. In any case, something isn’t right when it takes CCArray 53 seconds to insert or remove an object at the first index when NSMutableArray only needs 0.2 seconds to perform the same operations. If you rely on the removeAtIndex and insertAtIndex methods you should refrain from using CCArray until this problem is fixed. As far as I can tell, cocos2d rarely uses these methods internally so this issue shouldn’t have a measurable impact on cocos2d’s overall performance.

Update: @manucorporat has already fixed this CCArray performance issue (within minutes!!), it should be integrated in a new cocos2d build soon. I’ll update the performance test results soon.
Update #2: Apparently the fix was not all encompassing. It was faster on the iPad but just about 8 times, but still at least 13 times slower than NSMutableArray. On my iPhone 3G the results were worse and went down to a factor of 360. It’s being worked on. Stay tuned.

Note that this diagram is not to scale! If it were, you wouldn’t be able to make out the blue bars for NSMutableArray!

All tests were done on an iPhone 3G running iOS 4.1 with a release build of cocos2d v0.99.5 beta. The results for iPad were only glanced at to check for relative differences, see text above. All results are in milliseconds, averaged over several runs (3-5) with an iteration count of 50,000. The read speed tests were run individually, not in sequence, because running all read speed tests in sequence caused slightly different results depending on the order in which the tests were run. For example, the slower speed of NSArray was less pronounced if the tests ran in sequence.

Feel free to download my ArraySpeedTest.zip project and try to reproduce the results. Only run the tests on a device and in release builds to get comparable numbers. Let me know if you find flaws in the test setup. I’m also interested to hear if the relative speeds vary on other devices.

cocos2d Book, Chapter 5: Getting bigger and better

On July 23, 2010, in Announcements, book, by Steffen Itterheim

Chapter 5 – Getting bigger and better

The gist of this chapter will be to discuss the simple game project from the previous chapter. I threw everything into one class, clearly not what you want to do for bigger games. But getting from one-class to real code design is a big step which some hesitate to take. I’ll make that easier and discuss common issues and their solutions, such as what to seperate, what to subclass from and how you can have all the seperated objects communicate with each other and exchange information in various ways.

A big topic will of course be how to take advantage of cocos2d’s scene hierarchy and which pitfalls it may have when moving from a single-layer game to one which has multiple layers and even multiple scenes.

As for the chapter title I’m not so sure if that’ll be it. Maybe along the way while I’m writing I’ll change it. Suggestions welcome!

The chapter will be submitted on Friday, July 30th.

What’s your take on good cocos2d code structure?

Did you ever struggle with cocos2d design concepts? Or the cocos2d scene hierarchy? Or how to layout a scene and divide your game into logical parts? Tell me about it.

I know theses questions are somewhat generic to ask. It’s about the things that don’t feel right but there doesn’t seem to be a better, more obvious way. I think we all know some of those, if you do, be sure to tell me! Leave a comment or write me an email.

Summary of working on Chapter 4 – First simple game

The game I chose to make is called Doodle Drop and features dropping spiders and an accelerometer controlled alien trying to avoid the spiders. All in all it got divided into 8 concrete steps. Lots and lots of code comments, too.

It starts simple enough, adding resources to Xcode and adding sprites. It gets more gameplay-esque when the accelerometer-driven player controls got tweaked to provide acceleration and deceleration of the player object. In contrast, the spiders movements are driven only by actions.

I introduce you to two undocumented features of cocos2d, namely CCArray which is since v0.99.4 used to store all children of a node. The other are the CGPointExtension class which has all the functions normally provided by a physics engine, however not every game should link a physics engine just because one needs those math functions. That’s why CGPointExtension comes in handy.

With the ccpDistance method the collision checks are done. Simple radial collisions, and in debug mode the collision radii are drawn too.

In between the CCLabel for the score got replaced with a CCBitmapFontAtlas, because it killed the framerate. I shortly mentioned Hiero and how to use it in principle but for all the details there was no room. But while I was at it I created the Hiero Tutorial.

At the end of the project I added some polish which isn’t described in the book (too many details) but really adds to the game’s look and feel. The spiders drop, hang in there, then charge before dropping down, all done using actions. I’ve also added the thread they’re hanging from using ccDrawLine. And then there’s a game over label which shows even more action use.

One of the principles I followed is to stay away from fixed coordinates as much as possible. So the project, once finished, did run just fine on an iPad. Although the experience is a different one, there’s more spiders dropping and they drop faster but there’s also more safe space to maneuver to.

Oh and, the game art is all mine. Yes, I know … but Man-Spiders do have just six legs! :)