Lessons Learned in Tilt Controls

Ahh, my first idevblogaday post! For those that don’t know, idevblogaday is a group of indie dev bloggers started by Miguel Á. Friginal of @mysterycoconut games to get developers blogging more regularly. So we’ll see how that goes :]. After finishing the initial release of Tilt to Live I had been wanting to gather up my thoughts on what I had learned about making a game that is heavily dependent on accelerometer controls. Sadly, this was on the back burner for the longest time. I eventually got contacted by Noel Llopis of Snappytouch and through some e-mail conversations with him I was able to get some of my ideas finally in writing. So now, I figured I’d take that and present it to everyone to see, learn, and laugh at my horribly awesome code.

Some Background

I want to give a quick and dirty history of Tilt to Live (TtL). This is not a post-mortem quality account (That will come later), but something so readers have a context of how TtL evolved and where it came from code-wise. TtL was the first app I wrote for an iDevice platform, ever. First time using a mac regularly, first time using XCode, first time using Objective-C. There were a lot of firsts. Typically when I’m learning something I don’t bother with style or best practices, or platform specific conventions. I learn all that as I go. TtL has come a very long way from the first iteration of it.

Then

The first signs of things to come! The green dots are what are considered the ‘nukes’ in today’s TtL.

..and now:

 

While the game is pretty, thanks to Adam’s Awesome Art (AAA quality!!!), the code at times can feel like one giant ugly beast. Usually  when I’m in an environment I know well, I write a prototype, then start over and write the game ‘correctly’ so the code can be maintainable throughout. Not so with TtL, this game evolved straight from prototype hack-a-thon to full on production code. The code base went through lots of rewrites, refactoring, and dubious ‘fixes’ to keep things somewhat on the side of sane. On the bright side, I now have a much clearer idea of what ‘maintainable’ code shouldn’t look like and a lot of design decisions I know to make earlier on in the coding process to avoid some clunkiness in the future. There’s certainly something to be said though for making it ‘just work’ because in the end, no one gives a damn (except the author) how it’s written as long as the game is fun :].

With that said, I’d like to talk about some of the design issues we came across while developing TtL’s tilt controls.

Design issues with tilt controls

Our main area of focus on ‘controls design’ was how to present calibration to the user. Overall, we had a very minimalistic approach to the game’s user experience. We wanted minimal taps to get into the game, and once in the game, all you did was tilt to interact. Naturally, having calibration be automated seems like the ultimate in minimalist user-experience. The user doesn’t even experience it. It’s supposed to “just work”. So how did our calibration work initially? Well, when the game counts down on ‘3,2,1’ and the ‘1’ disappears the game calibrates at that point in time as the neutral position. Also, when the player unpaused the game, it would recalibrate. Sounded great, and it worked great…if you understood how it worked. From playtesting we found:

  • Users would get confused as soon as they change their play position (from say, in lap to on top of a table) without pausing the game. It was a cause of frustration for them.
  • The more savvy players immediately looked for options to calibrate because the terminology was engrained in them from previous iPhone games. Without options to calibrate they felt frustrated that they could not control the way they’d like to play.
  • In more social settings rather than isolated playtesting, the game would be played haphazardly. This caused all sorts of havoc on our auto-calibration scheme. The player would either lay the phone down, swing it down to their side as they’re talking to their friend while the count down is going on and then when they’d swing it back up again to play the calibration is royally screwed up. This didn’t make for a good first impression either. Ouch.

So what happens when we decided to tell users that pausing and unpausing the game would recalibrate the game before they played? We did that as a quick way to emulate ‘custom calibration’.

  • For the group of users who understood the technical need of calibration instantly got it.
  • Everyone else pretty much said “What ees des calibration meen?”. Yes, in bad fake accent and all.

Well, since we wanted to appeal to more than “hardcore” (I use that word very loosely here) iPhone gamers we had some take aways from the testing:

  1. On auto calibration: Without proper feedback, the user can set the calibration very wrong, making them think the game is broken.
  2. On custom calibration: Some users get it, but with a near infinite number of choices to the user, it’s hard to expect a non-gamer to know what the best setup/calibration would be.
  3. On choices in general: Experienced users like it, nay, demand more options. The fewer decisions a user has to make on how to “play” your game, generally the more accessible it is. A powerful concept in the mobile space, especially with non-precise/noisy physical inputs like accelerometers.
  4. In respect to TtL’s design specifically: TtL’s gameplay requires accurate tilting controls or the game falls flat. Other games that require less precision can certainly get away with auto calibration.

So after brainstorming a few ways to present a new calibration screen, Adam had a pretty unique idea compared to most tilt-based games at the time:

While we thought it was a good idea and it playtested well, we didn’t think the preset choices would resonate with users as much as it did.

“The game has some of the best options I have found for customizing the tilt controls.” -Nate Adcock, iphonelife.com

“For those of you worrying about manual calibration, don’t.” -nodpad.com

“Unique to this game is a calibration option that provides three default control schemes that should satisfy most positions that you’d like to hold your device in.” -Ben Briggs, gamesuncovered.com

 

Players and reviewers alike seemed to respond positively to our approach of essentially what is just a common preset list. The regular setting works for the vast majority of play cases and top-down works great for the purists. Sleepy was a rather experimental one that has mixed results. It works for me, but if you ask a group of players to play in a “sleepy” position you’ll most likely get a higher variance of orientations than what “regular” and “top-down” imply. I don’t think our technical implementation of tilt controls was any more accurate than the next game’s, but I feel because the user was able to more accurately set the game up successfully with a reduced number of choices their experience was much more positive.

Technical issues with tilt controls

Design issues aside, the technical side of my approach to tilt controls had a few issues I had to overcome as well. The first and biggest issue was a limitation of how I went about calculating the tilt offset. As the calibration of the iPhone approached a purely vertical orientation, detecting horizontal motion reduced to zero pretty much. To counteract this, I added an additional control scheme for when the device was in such a position and require the player to move the ship by “steering” the iPhone instead of tilting. Not the most ideal situation, and with the advent of the iPhone 4’s gyroscope this could probably be mitigated. There are certainly ways to implement tilt controls on the 3G/3GS where this isn’t an issue, but I haven’t had much time to experiment with them yet. I’ll certainly post an update on my findings as I get to them.

The 2nd issue was noisy accelerometer data. I had experimented with filtering the raw input, as per Apple doc recommendation, but in the end I let the input through unfiltered and just altered the rendering of the ship. Rendering the ship’s orientation based on the raw data caused the ship to shake uncontrollably as if it was hyped up on speed. Also this would make it impossible to use any weapons that required aiming. Solving it was pretty basic as I just added an interpolation function that would ease the ship to the current desired orientation over a couple of frames and adding a dead zone so that players can sit still without spinning wildly. I’ve played a few tilt-based games where the players’ avatar had an orientation and no dead zone was implemented. Be nice to your players, add a dead zone at the very least damnit.

A 3rd but minor issue was with sampling rates. I sample the accelerometer at 30hz and TtL runs at 30FPS. When (not if) the FPS start dipping the accelerometer data seems to get a bit out of sync and you start seeing small jolts of movements from the accelerometer data that came in between some of the frames. TtL doesn’t do much in managing varying frame times (shame on me), but this problem hasn’t brought much pain so I’m not looking at it worth fixing yet.

Well how does it REALLY work?

For those that are still reading this and are curious to see how it’s implemented I’ve attached a sample XCode project with the implementation that TtL uses. I took a GLSprite sample project and mutilated it until it resembled a sufficient “tilt controls sample”. The approach I had in mind was to make the iphone function as a “reverse” joystick. I’m not even going to try to explain WTF that means in text so here’s a poorly drawn illustration:

I  interpreted the accelerometer data in a way that would project a vector that always pointed away from the screen of the device. I would use this to find the difference vector between it and the ‘neutral’ position (blue) and finally project this onto the 2D plane of the screen to give me a velocity. The green arrow is what the player’s velocity was set to. So when the neutral position and inverted accel data are very close to each other, the player’s velocity is clamped to zero (deadzone).

 

You’ll find in the sample project that all the relevant accelerometer logic is implemented inside the TiltPlayer class in the ‘accelerometer:didAccelerate” method. Inside the TiltPlayer’s ‘update’ method it handles the smoothing of the icon’s rotation to reduce jitter. The rest are just supporting classes for rendering and other boilerplate code. So for those that wish to just see the pertinent code without firing up XCode, here it is:

[code]- (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration
{
/* while not ideal, most of the relevant code is stuffed in this method for clarity. A lot can be computed once
and saved in instance variables, preferences, etc instead of re-calculating each frame
*/

// hard coding our 'neutral' orientation. By default this is the orientation of
// having the device tilted slightly towards your face.
// if you wanted strictly a 'top down' orientation then a (0,0,-1) vector would be put here.
// to create a new 'neutral' position you can sample the UIAcceleration parameter for a single
// frame and set that to be the new nx,ny,nz for the remainder of the app (aka calibrating).
const float nx = -0.63f;
const float ny = 0;
const float nz = -0.92f;

// this quaternion represents an orientation (I like to think of it as a 3D vector with 0 rotation in this case)
// that points straight out from the device's back away from the user's face.
Quaternion neutral(0,nx, ny, nz);
// take a straight up vector and rotate it by our 'neutral' orientation
// to give us a vector that points straight out from the device's screen (at the user's face)
Quaternion neutralPosition = neutral * Quaternion(0,0,0,1);

/* now with our 'neutral' quaternion setup we:
1. take the incoming accelerometer data
2. convert it to a 2D velocity projected onto the plane of the device's screen
3. and rotate it by 90 degrees (since we are landscape oriented) and feed it to our player's
velocity directly.
*/

// convert our accel data to a Quaternion
Quaternion accelQuat(0, acceleration.x, acceleration.y, acceleration.z);

// now rotate our accel data BY the neutral orientation. effectively transforming it
// into our local space.
Vec3 accelVector = (accelQuat * neutralPosition).v; // we only want the 3D vector at this point

// now with our accel vector we wish to transform it into our standard (1,1,1) coordinate space
Vec3 planeXAxis(1,0,0);
Vec3 planeYAxis(0,1,0);
Vec3 normal(0,0,1); // the normal of the plane we wish to transform our data into.

//project this movement onto our X/Y plane by removing
// the accel part that is along our normal
// note: Vec3 * Vec3 = dot product of the 2 vectors.
Vec3 projection = accelVector - normal *(accelVector * normal);

// now decompose that projection along our X and Y axis that represents our 2D plane
Vec2 accel2D(0,0);
accel2D.x = planeXAxis * projection;
accel2D.y = planeYAxis * projection;

const float xSensitivity = 2.8f;
const float ySensitivity = 2.8f; // yay magic numbers!
const float tiltAmplifier = 8; // w0ot more magic numbers

// now apply it to our player's velocity data.
// we also rotate the 2D vector by 90 degrees by switching the components and negating one
// since we are in a landscape orientation.
vx += (-accel2D.y) * tiltAmplifier * xSensitivity;
vy -= accel2D.x * tiltAmplifier * ySensitivity; // we do a (-) here because the accel y axis is inverted.
}
[/code]

if you want to download the XCode project, you can grab it here. [note: updated with Mikko's more concise version below]

That’s it for now. As always, feedback is welcome and if anyone takes that code and improves it, it’d be nice to know :].

*Update*

Thanks to Mikko Mononen for making the code much more concise! Below is the altered method with Mikko’s contribution:
[code]
- (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration
{
// A much more concise version courtesy of Mikko Mononen http://digestingduck.blogspot.com/
Vec2 accel2D(0,0);
Vec3 ax(1, 0, 0);
Vec3 ay(-.63f, 0,-.92f);
Vec3 az(Vec3::Cross(ay,ax).normalize());
ax = Vec3::Cross(az,ay).normalize();

accel2D.x = -Vec3::Dot(Vec3(acceleration.x, acceleration.y, acceleration.z), ax);
accel2D.y = -Vec3::Dot(Vec3(acceleration.x, acceleration.y, acceleration.z), az);

const float xSensitivity = 2.8f;
const float ySensitivity = 2.8f; // yay magic numbers!
const float tiltAmplifier = 8; // w0ot more magic numbers

// since we are in a landscape orientation.
// now apply it to our player's velocity data.
// we also rotate the 2D vector by 90 degrees by switching the components and negating one
vx += -(accel2D.y) * tiltAmplifier * xSensitivity;
vy += accel2D.x * tiltAmplifier * ySensitivity;

}
[/code]

27 Responses

  1. beatwho Says:

    Awesome first post dude, some great insights into the accelerometer and TtL

  2. Owen Goss Says:

    Thanks for the post! I like that you talked both about the design decisions and the technical. There’s lots of very useful information here. I’ve got a few game ideas that involve accelerometer input, but I haven’t played much with the APIs yet. This should definitely help if I start prototyping some of them. :)

  3. Mikko Mononen Says:

    You can probably reduce that long code with quats et all to this? (my head treats up=y, I think in your code it is z)

    Vec3 ax(1,0,0);
    Vec3 ay(0,-0.63f,-0.92f); // down
    Vec3 az(normalize(cross(ay,az)));
    ax = normalize(cross(az,ay)); // no need for this if ay.x = 0

    accel2D.x = dot(acceleration,ax);
    accel2D.y = dot(acceleration,az));

    The worst thing about tilt controls is that the player needs to put all the energy into finding the neutral position. This is usually due too little feedback.

    Often just visual feedback might be enough to give the player more understanding on what happens under the hood and allows to counter act on the situation without too much frustration.

    Related problem is that if the players panics and stops doing anything, the game will not treat that as “no-operation”, but “random” input instead.

    This can be solved by requiring the player to press something to make the tilt input active. That is easy way to “autocalibrate” too… all the movement is relative to the pose where the player pressed the button.

    Although, this second option suffers from the same counterintuitiveness as long scroll with mouse (you need to lift the mouse and relocate it).

    It did quite a lot of prototyping on tilt controls long time ago, before zen bound was even called zen bound. For that game the touch controls were so superior in precision and tactile feel (and calmness) that the choice was unanimous :)

  4. Matt Rix Says:

    The first thing I noticed when playing Tilt To Live was that it had great tilt controls. TTL and Dark Nebula are the only iOS games I’ve played that get touch controls completely right. It’s nice to see the actual code behind the controls, this could be very handy if I ever make an tilt-powered game.

  5. Matt Rix Says:

    Sorry, when I said “touch controls”, I meant “tilt controls”.

    PS: The “about us” page on the One Man Left site is awesome. Just saying.

  6. Alfred Says:

    Coming from a web development background I just learned a bunch of stuff with your code (quaternions, slerp and the general theory behind rotation).

    By the way, bought TTL after studying the code and playing around with the prototype.

    Thanks!
    Karnak

  7. Alex Says:

    Thanks all for the comments!
    @OwenGoss, @MattRix -Glad you find the code useful. It can definitely be used to quickly throw together a proof of concept for a tilt-based game with controls ready to go.

    @Mikko Appreciate the feedback! That is much more concise, thanks! Always for shorter code! I updated the download to the smaller version. I also adjusted your sample to work in the landscape mode.

  8. AndyN Says:

    Great post. Thank you. Very handy for someone like myself just starting out developing on iPad.

    Would love to see something similar on how you put together your graphics pipeline, but realise that you probably don’t want to give away all your secrets!

    Cheers

  9. Leto Says:

    Have to agree with you regarding making code that works. I’m pretty pedantic on details. I have a colleague who hacks together code with reckless speed, and it freaks me out!, but in the end it just works.

  10. Learning Diary #10: Lua, socket server, path finding, quaternions | Karnak Games Says:

    [...] http://www.gamedev.net/reference/articles/article1095.asp, http://www.paradeofrain.com/2010/07/lessons-learned-in-tilt-controls/ No Comments by Alfred R. Baudisch on July 13, 2010 | filed in Learning Diary | tagged debug, [...]

  11. Diego Says:

    Great article! One question: How did you deal in your game for the situation when the z axes switches from negative to positive and the x values reaches a maximum? By this I mean the situation where the player is holding the device in landscape mode with the longest side parallel to the ground.

  12. Alex Says:

    @Diego I’m assuming you mean when the device is tilted completely vertically. In that case, I switch the control scheme to start interpreting the tilt of the device as if it was a steering wheel. When the player rotates it slightly to the left, I start applying a force on the ‘x’ component of the player’s character and so on.

  13. Diego Says:

    Right Alex, my issue is that I wanted to continue using the same control scheme when the device is completelly vertical. My issue is that the maximum values for x switch between devices, going a liitle over 1.0 (1.05 approx).

  14. Stop talking, start doing. Postmorten of how I made 2 games in 10 days. | Karnak Games Says:

    [...] On the same day I had a flying Santa with nice tilt controls, thanks to the amazing source code released by Alex Okafor (the developer of Tilt to Live) here: Lessons Learned in Tilt Controls. [...]

  15. Josh Says:

    Great blog, Im surprised I haven’t come cross it before now.

    Thanks for the code samples, only Im not quire sure how to apply this. Could you please fix the link to the sample xcode project?

  16. Alex Says:

    @Josh Sorry about that, the link should be fixed now.

  17. Russell Holmes Says:

    Ive added you classes to my app to try and implement the tilt but i keep getting the error

    Expected specifier-qualifier-list before ‘Vec3′ in /Users/russellholmes/Desktop/Classes/Vec3.h

    Can someone please help me solve this issue!
    THnkas

  18. Drahrim Says:

    Hi, Alex ! Nice tutorial here. I found myself really confused about how to properly do the calibration, until i found this marvelous tips. But, i found a little problem. If i try a positive value for the Z-Axis, for example : Vec3 ay(-.63f, 0, .7f); then the control for Y-Axis became reversed. It worked fine though if it’s a negative number. Any insight bout this ? Thanks in advance :)

  19. pfg2009 Says:

    Thanks for a great post! Worked like a charm and saved me at least of week of head scratching.

  20. Nat Weiss Says:

    Rad tutorial here and I love Tilt to Live!

    Was wondering though… Why not set up your app to receive device orientation updates and then recalibrate your red inverted accel data vector based on the new orientation?

    We can see if the iDevice is portrait, portrait upside down, landscape left, landscape right, flat face up, or flat face down and use that info to determine the red vector.

    Is it better to just ask the user anyway?

    Or is a completely automated, bulletproof tilt control possible?

  21. Alex Says:

    @Nat Weiss While I’m sure that could work to an extent. The issue would come up with people calibrated for ‘top down’ controls. You would flip between landscape left/right and your controls would invert? We did try to play around with auto-calibration very early in Tilt to Live’s development, and found that being explicit about the calibration process resulted in more predictable ‘controls’ than trying to be clever and hide the process all together.

  22. Chris Says:

    Thanks it works like a charm but i’m having difficulty to modify this code to move sprite along Y axis

    Can you help me out?

    Thanks
    Chris

  23. Stephen Says:

    Hi,

    Great tutorial. How do you recalibrate to the current orientation?

    Regards

    i

  24. Stephen Says:

    Ignore my last comment, I can see the recalibration comment in the code (doh!)..having trouble getting the C++ files included in a cocos 2D project though, any advice appreciated.

    http://www.cocos2d-iphone.org/forum/topic/22423

  25. Here’s a nice little link on handling tilt… « coffee bean studios Says:

    [...] http://www.paradeofrain.com/2010/07/lessons-learned-in-tilt-controls/   [...]

  26. Indian Indie Game Dev Chat Session #4 | Vijay's Blog Says:

    [...] Krishna: prototyping for his game, strong reco to play Tilt To Live in case you’re interested in doing an accelorometer game. Lessons Learned in Tilt Controls [...]

  27. Juan Says:

    Im wondering, I am developing my first game and I wonder if your code can be used (CC, MIT, GNU) or is only for learning/commenting/reference?

Leave a Comment