Ahh, my first idevblogaday post! For those that don’t know, idevblogaday is a group of indie dev bloggers started by Miguel Á. Friginal of @mysterycoconut games to get developers blogging more regularly. So we’ll see how that goes :]. After finishing the initial release of Tilt to Live I had been wanting to gather up my thoughts on what I had learned about making a game that is heavily dependent on accelerometer controls. Sadly, this was on the back burner for the longest time. I eventually got contacted by Noel Llopis of Snappytouch and through some e-mail conversations with him I was able to get some of my ideas finally in writing. So now, I figured I’d take that and present it to everyone to see, learn, and laugh at my horribly awesome code.

Some Background

I want to give a quick and dirty history of Tilt to Live (TtL). This is not a post-mortem quality account (That will come later), but something so readers have a context of how TtL evolved and where it came from code-wise. TtL was the first app I wrote for an iDevice platform, ever. First time using a mac regularly, first time using XCode, first time using Objective-C. There were a lot of firsts. Typically when I’m learning something I don’t bother with style or best practices, or platform specific conventions. I learn all that as I go. TtL has come a very long way from the first iteration of it.

Then

The first signs of things to come! The green dots are what are considered the ‘nukes’ in today’s TtL.

..and now:

 

While the game is pretty, thanks to Adam’s Awesome Art (AAA quality!!!), the code at times can feel like one giant ugly beast. Usually  when I’m in an environment I know well, I write a prototype, then start over and write the game ‘correctly’ so the code can be maintainable throughout. Not so with TtL, this game evolved straight from prototype hack-a-thon to full on production code. The code base went through lots of rewrites, refactoring, and dubious ‘fixes’ to keep things somewhat on the side of sane. On the bright side, I now have a much clearer idea of what ‘maintainable’ code shouldn’t look like and a lot of design decisions I know to make earlier on in the coding process to avoid some clunkiness in the future. There’s certainly something to be said though for making it ‘just work’ because in the end, no one gives a damn (except the author) how it’s written as long as the game is fun :].

With that said, I’d like to talk about some of the design issues we came across while developing TtL’s tilt controls.

Design issues with tilt controls

Our main area of focus on ‘controls design’ was how to present calibration to the user. Overall, we had a very minimalistic approach to the game’s user experience. We wanted minimal taps to get into the game, and once in the game, all you did was tilt to interact. Naturally, having calibration be automated seems like the ultimate in minimalist user-experience. The user doesn’t even experience it. It’s supposed to “just work”. So how did our calibration work initially? Well, when the game counts down on ‘3,2,1’ and the ‘1’ disappears the game calibrates at that point in time as the neutral position. Also, when the player unpaused the game, it would recalibrate. Sounded great, and it worked great…if you understood how it worked. From playtesting we found:

  • Users would get confused as soon as they change their play position (from say, in lap to on top of a table) without pausing the game. It was a cause of frustration for them.
  • The more savvy players immediately looked for options to calibrate because the terminology was engrained in them from previous iPhone games. Without options to calibrate they felt frustrated that they could not control the way they’d like to play.
  • In more social settings rather than isolated playtesting, the game would be played haphazardly. This caused all sorts of havoc on our auto-calibration scheme. The player would either lay the phone down, swing it down to their side as they’re talking to their friend while the count down is going on and then when they’d swing it back up again to play the calibration is royally screwed up. This didn’t make for a good first impression either. Ouch.

So what happens when we decided to tell users that pausing and unpausing the game would recalibrate the game before they played? We did that as a quick way to emulate ‘custom calibration’.

  • For the group of users who understood the technical need of calibration instantly got it.
  • Everyone else pretty much said “What ees des calibration meen?”. Yes, in bad fake accent and all.

Well, since we wanted to appeal to more than “hardcore” (I use that word very loosely here) iPhone gamers we had some take aways from the testing:

  1. On auto calibration: Without proper feedback, the user can set the calibration very wrong, making them think the game is broken.
  2. On custom calibration: Some users get it, but with a near infinite number of choices to the user, it’s hard to expect a non-gamer to know what the best setup/calibration would be.
  3. On choices in general: Experienced users like it, nay, demand more options. The fewer decisions a user has to make on how to “play” your game, generally the more accessible it is. A powerful concept in the mobile space, especially with non-precise/noisy physical inputs like accelerometers.
  4. In respect to TtL’s design specifically: TtL’s gameplay requires accurate tilting controls or the game falls flat. Other games that require less precision can certainly get away with auto calibration.

So after brainstorming a few ways to present a new calibration screen, Adam had a pretty unique idea compared to most tilt-based games at the time:

While we thought it was a good idea and it playtested well, we didn’t think the preset choices would resonate with users as much as it did.

“The game has some of the best options I have found for customizing the tilt controls.” -Nate Adcock, iphonelife.com

“For those of you worrying about manual calibration, don’t.” -nodpad.com

“Unique to this game is a calibration option that provides three default control schemes that should satisfy most positions that you’d like to hold your device in.” -Ben Briggs, gamesuncovered.com

 

Players and reviewers alike seemed to respond positively to our approach of essentially what is just a common preset list. The regular setting works for the vast majority of play cases and top-down works great for the purists. Sleepy was a rather experimental one that has mixed results. It works for me, but if you ask a group of players to play in a “sleepy” position you’ll most likely get a higher variance of orientations than what “regular” and “top-down” imply. I don’t think our technical implementation of tilt controls was any more accurate than the next game’s, but I feel because the user was able to more accurately set the game up successfully with a reduced number of choices their experience was much more positive.

Technical issues with tilt controls

Design issues aside, the technical side of my approach to tilt controls had a few issues I had to overcome as well. The first and biggest issue was a limitation of how I went about calculating the tilt offset. As the calibration of the iPhone approached a purely vertical orientation, detecting horizontal motion reduced to zero pretty much. To counteract this, I added an additional control scheme for when the device was in such a position and require the player to move the ship by “steering” the iPhone instead of tilting. Not the most ideal situation, and with the advent of the iPhone 4’s gyroscope this could probably be mitigated. There are certainly ways to implement tilt controls on the 3G/3GS where this isn’t an issue, but I haven’t had much time to experiment with them yet. I’ll certainly post an update on my findings as I get to them.

The 2nd issue was noisy accelerometer data. I had experimented with filtering the raw input, as per Apple doc recommendation, but in the end I let the input through unfiltered and just altered the rendering of the ship. Rendering the ship’s orientation based on the raw data caused the ship to shake uncontrollably as if it was hyped up on speed. Also this would make it impossible to use any weapons that required aiming. Solving it was pretty basic as I just added an interpolation function that would ease the ship to the current desired orientation over a couple of frames and adding a dead zone so that players can sit still without spinning wildly. I’ve played a few tilt-based games where the players’ avatar had an orientation and no dead zone was implemented. Be nice to your players, add a dead zone at the very least damnit.

A 3rd but minor issue was with sampling rates. I sample the accelerometer at 30hz and TtL runs at 30FPS. When (not if) the FPS start dipping the accelerometer data seems to get a bit out of sync and you start seeing small jolts of movements from the accelerometer data that came in between some of the frames. TtL doesn’t do much in managing varying frame times (shame on me), but this problem hasn’t brought much pain so I’m not looking at it worth fixing yet.

Well how does it REALLY work?

For those that are still reading this and are curious to see how it’s implemented I’ve attached a sample XCode project with the implementation that TtL uses. I took a GLSprite sample project and mutilated it until it resembled a sufficient “tilt controls sample”. The approach I had in mind was to make the iphone function as a “reverse” joystick. I’m not even going to try to explain WTF that means in text so here’s a poorly drawn illustration:

I  interpreted the accelerometer data in a way that would project a vector that always pointed away from the screen of the device. I would use this to find the difference vector between it and the ‘neutral’ position (blue) and finally project this onto the 2D plane of the screen to give me a velocity. The green arrow is what the player’s velocity was set to. So when the neutral position and inverted accel data are very close to each other, the player’s velocity is clamped to zero (deadzone).

 

You’ll find in the sample project that all the relevant accelerometer logic is implemented inside the TiltPlayer class in the ‘accelerometer:didAccelerate” method. Inside the TiltPlayer’s ‘update’ method it handles the smoothing of the icon’s rotation to reduce jitter. The rest are just supporting classes for rendering and other boilerplate code. So for those that wish to just see the pertinent code without firing up XCode, here it is:

[code]- (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration
{
/* while not ideal, most of the relevant code is stuffed in this method for clarity. A lot can be computed once
and saved in instance variables, preferences, etc instead of re-calculating each frame
*/

// hard coding our ‘neutral’ orientation. By default this is the orientation of
// having the device tilted slightly towards your face.
// if you wanted strictly a ‘top down’ orientation then a (0,0,-1) vector would be put here.
// to create a new ‘neutral’ position you can sample the UIAcceleration parameter for a single
// frame and set that to be the new nx,ny,nz for the remainder of the app (aka calibrating).
const float nx = -0.63f;
const float ny = 0;
const float nz = -0.92f;

// this quaternion represents an orientation (I like to think of it as a 3D vector with 0 rotation in this case)
// that points straight out from the device’s back away from the user’s face.
Quaternion neutral(0,nx, ny, nz);
// take a straight up vector and rotate it by our ‘neutral’ orientation
// to give us a vector that points straight out from the device’s screen (at the user’s face)
Quaternion neutralPosition = neutral * Quaternion(0,0,0,1);

/* now with our ‘neutral’ quaternion setup we:
1. take the incoming accelerometer data
2. convert it to a 2D velocity projected onto the plane of the device’s screen
3. and rotate it by 90 degrees (since we are landscape oriented) and feed it to our player’s
velocity directly.
*/

// convert our accel data to a Quaternion
Quaternion accelQuat(0, acceleration.x, acceleration.y, acceleration.z);

// now rotate our accel data BY the neutral orientation. effectively transforming it
// into our local space.
Vec3 accelVector = (accelQuat * neutralPosition).v; // we only want the 3D vector at this point

// now with our accel vector we wish to transform it into our standard (1,1,1) coordinate space
Vec3 planeXAxis(1,0,0);
Vec3 planeYAxis(0,1,0);
Vec3 normal(0,0,1); // the normal of the plane we wish to transform our data into.

//project this movement onto our X/Y plane by removing
// the accel part that is along our normal
// note: Vec3 * Vec3 = dot product of the 2 vectors.
Vec3 projection = accelVector – normal *(accelVector * normal);

// now decompose that projection along our X and Y axis that represents our 2D plane
Vec2 accel2D(0,0);
accel2D.x = planeXAxis * projection;
accel2D.y = planeYAxis * projection;

const float xSensitivity = 2.8f;
const float ySensitivity = 2.8f; // yay magic numbers!
const float tiltAmplifier = 8; // w0ot more magic numbers

// now apply it to our player’s velocity data.
// we also rotate the 2D vector by 90 degrees by switching the components and negating one
// since we are in a landscape orientation.
vx += (-accel2D.y) * tiltAmplifier * xSensitivity;
vy -= accel2D.x * tiltAmplifier * ySensitivity; // we do a (-) here because the accel y axis is inverted.
}
[/code]

if you want to download the XCode project, you can grab it here. [note: updated with Mikko’s more concise version below]

That’s it for now. As always, feedback is welcome and if anyone takes that code and improves it, it’d be nice to know :].

*Update*

Thanks to Mikko Mononen for making the code much more concise! Below is the altered method with Mikko’s contribution:
[code]
– (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration
{
// A much more concise version courtesy of Mikko Mononen http://digestingduck.blogspot.com/
Vec2 accel2D(0,0);
Vec3 ax(1, 0, 0);
Vec3 ay(-.63f, 0,-.92f);
Vec3 az(Vec3::Cross(ay,ax).normalize());
ax = Vec3::Cross(az,ay).normalize();

accel2D.x = -Vec3::Dot(Vec3(acceleration.x, acceleration.y, acceleration.z), ax);
accel2D.y = -Vec3::Dot(Vec3(acceleration.x, acceleration.y, acceleration.z), az);

const float xSensitivity = 2.8f;
const float ySensitivity = 2.8f; // yay magic numbers!
const float tiltAmplifier = 8; // w0ot more magic numbers

// since we are in a landscape orientation.
// now apply it to our player’s velocity data.
// we also rotate the 2D vector by 90 degrees by switching the components and negating one
vx += -(accel2D.y) * tiltAmplifier * xSensitivity;
vy += accel2D.x * tiltAmplifier * ySensitivity;

}
[/code]

I’ve had this issue in past 2D games all the time. The ability to interpolate 2D rotations for basic animations for anyone that doesn’t understand quaternions becomes a rather laborious task. I’ve done messy if-then statements to try to catch all the cases where rotation hits the ‘360 to 0’ boundary and it’s always ugly and seemed like there had to be a more mathematically correct way to interpolate a circular value.

Well for anyone that sticks with it long enough, they undoubtedly run into quaternions. What are quaternions? I’m not about to explain it in detail, but I like to think of them as a way to express a 3D rotations and not having that annoying gimbal lock problem.

So how do quaternions help our 2D rotation needs? Well if you’re in a situation where you need to interpolate between the shortest path between 2 angles then Quaternions can help. You have a few choices to solve this problem:

  1. Use if-then clauses to try to catch all the cases when you hit a rotational boundary if you are rotating from, for example, 300 degrees to 2 degrees. You would want to rotate all the way through 360 instead of going backwards.
  2. Use quaternions, which are more applicable to 3D rotations, to interpolate between to angles using spherical interpolation (slerp for short).
  3. Use spinors, which are a very similar concept to quaternions, except they are more catered to 2D rotations.

You’ll eventually see that using quaternions is somewhat overdoing it as 2 axises are never used in any calculation in 2D. If you eliminate these 2 axises from the quaternion you end up with a spinor! Thanks to Frank, of Chaotic Box, who helped lead me down that direction for my own iPhone game.

To help test out the concept and make sure it’s working I ended up writing a Blitzmax quickie app using what little I know about quaternions. I’m certainly a bit iffy when it comes to dealing with complex numbers, as I don’t fully grok them as I do normal vector math, but I make do. So if anyone reads this post and sees some sort of logical error in my code, by all means leave a comment and I’ll improve the listing.

So to get started I needed a way to represent a 2D rotation as a spinor. So I created a spinor type with some basic math functions (all of them analogous to how quaternions work):

[blitzmax2]
Type Spinor
Field real:Float
Field complex:Float

Function CreateWithAngle:Spinor(angle:Float)
Local s:Spinor = New Spinor
s.real = Cos(angle)
s.complex = Sin(angle)
Return s
End Function

Function Create:Spinor(realPart:Float, complexpart:Float)
Local s:Spinor = New Spinor
s.complex = complexPart
s.real = realPart
Return s
End Function

Method GetScale:Spinor(t:Float)
Return Spinor.Create(real * t, complex * t)
End Method

Method GetInvert:Spinor()
Local s:Spinor = Spinor.Create(real, -complex)
Return s.GetScale(s.GetLengthSquared())
End Method

Method GetAdd:Spinor(other:Spinor)
Return Spinor.Create(real + other.real, complex + other.complex)
End Method

Method GetLength:Float()
Return Sqr(real * real + complex * complex)
End Method

Method GetLengthSquared:Float()
Return (real * real + complex * complex)
End Method

Method GetMultiply:Spinor(other:Spinor)
Return Spinor.Create(real * other.real – complex * other.complex, real * other.complex + complex * other.real)
End Method

Method GetNormalized:Spinor()
Local length:Float = GetLength()
Return Spinor.Create(real / length, complex / length)
End Method

Method GetAngle:Float()
Return ATan2(complex, real) * 2
End Method

Function Lerp:Spinor(startVal:Spinor, endVal:Spinor, t:Float)
Return startVal.GetScale(1 – t).GetAdd(endVal.GetScale(t)).GetNormalized()
End Function

Function Slerp:Spinor(from:Spinor, dest:Spinor, t:Float)
Local tr:Float
Local tc:Float
Local omega:Float, cosom:Float, sinom:Float, scale0:Float, scale1:Float

‘calc cosine
cosom = from.real * dest.real + from.complex * dest.complex

‘adjust signs
If (cosom < 0) Then cosom = -cosom tc = -dest.complex tr = -dest.real Else tc = dest.complex tr = dest.real End If ' coefficients If (1 - cosom) > 0.001 Then ‘threshold, use linear interp if too close
omega = ACos(cosom)
sinom = Sin(omega)
scale0 = Sin((1 – t) * omega) / sinom
scale1 = Sin(t * omega) / sinom
Else
scale0 = 1 – t
scale1 = t
End If

‘ calc final
Local res:Spinor = Spinor.Create(0, 0)
res.complex = scale0 * from.complex + scale1 * tc
res.real = scale0 * from.real + scale1 * tr
Return res
End Function

End Type
[/blitzmax2]

One caveat in the above code is that any angle from 0-360 is mapped to 0-180 in the Spinor. What does this mean? it means if I want to represent the angle 270 I need to divide it by 2, creating the spinor with the angle 135. Now when I call GetAngle() it will return 270. This allows us to smoothly rotate the whole 360 degrees correctly. This is explained here in more detail.

So now I want to spherically interpolate (slerp) between 2 2D angles. Well, if I call Slerp() on the spinor type it’ll do just that, giving me another spinor that sits in between the 2 angles. I’ve read a couple of articles in the past on slerp and how to implement it, but in order to get things done I just used a publically listed code snippet and ported it to blitzmax. That whole article is worth the read and recommended, even if you’re just using 2D.

Now that the concept of spinors is encapsulated in the Spinor type I wanted just a basic convenience function that I can feed 2 angles and get another angle out:

[blitzmax2]
Function Slerp2D:Float(fromAngle:Float, toAngle:Float, t:Float)
Local from:Spinor = Spinor.Create(Cos(fromangle / 2), Sin(fromangle / 2))
Local toSpinor:Spinor = Spinor.Create(Cos(toAngle / 2), Sin(toAngle / 2))
Return Spinor.Slerp(from, toSpinor, t).GetAngle()
End Function
[/blitzmax2]
the ‘t’ variable is the time variable from 0 to 1 that determines how far to interpolate between the 2 angles. Giving it a value of 1 means you will get toAngle out, and a value of 0 is equal to fromAngle and anything in-between is just that…in-between.

Now I can call Slerp2D(300, 10, 0.5) And get the correct angle without worrying if I’m interpolating in the right direction :).

Also, I want to point out that the above code was just used in a quick proof of concept app. It’s somewhat sloppy and to be honest, I’m not sure I named the portions of the spinor correctly (real,complex), but it works. For clarity and to save time I didn’t inline the Slerp2D functions or the Spinor type methods, so it generates a lot of garbage. You would need to optimize this to use it in a tight loop somewhere. Any suggestions are welcomed.

As for references, I did some searching and got most of the ‘missing link’ materials I needed to bridge the gap between quaternions and spinors from:

…Adios…