Playing Around With Flash on iOS

About two weeks ago I mentioned I was looking into using the open source library, gameswf to integrate into our little 2D engine for iOS. I was at the point where I needed a UI solution for Outwitters, and I had a few options:

  1. Keep doing what I was doing with Tilt to Live and create the UI by handing coding everything and procedurally animating it
  2. Create a UI editor to help automate some of the UI features I had created in Tilt to Live
  3. Look into gameswf and see if I could go for the whole she-bang and just use Flash for the UI.

I’m happy to report the gameswf seems to be doing just dandy on iOS. When researching, I was hearing a lot of complaints on performance and compatibility (it’s only compatible with actionscript 2.0), and if you’re using a lot of complicated vector shapes you can bog down the FPS. Well reading the post date on a lot of those articles I found that some were pre-dating the 3GS! So in the end, I just put aside a week to mess around with gameswf on an ipad 2 and try to convert some of the ‘hand coded UI’ in our prototype to using flash.

Exhibit A

Just a disclaimer, all that you see in this video is a work-in-progress and extremely early. It was mostly a mock up to play around with the UI.

So in the video you see the main menu with the Outwitters logo, some buttons, a blue blade, and a background illustration being animated. This is entirely made in flash with pngs and some static text as buttons (just a full screen swf). As I press a map to load a pop up blinks into existance asking if I want to resume a previous game or not. This pop up doesn’t have any external bitmaps or pngs in it, it was made as vector shape rectangle with curved corners and some static text (which actually are bitmaps, but this is handled internally in gameswf).Β And finally, the mock up game board screen you see a flashy banner come across the screen announcing the player’s turn and shimmering, and then pulling away when the player taps the screen.

The animations weren’t anything too special as my flash skills are pretty horrible these days, but hopefully one can see that in the right hands this can be an incredibly useful tool to making games look that much prettier and be more engaging.

The best thing that’ll come out of this, is now the artist, Adam, has a large amount of creative control of how the UI will look and feel. Before it was heavily on the programmer side to make things work, and the fidelity suffered for it.

The Downside

Now for the things that might be problematic going forward:

Memory

Nothing has changed about this as far as I know, but images are required to be sized at powers of 2. When your working with the lower level api’s this isn’t a limitation that is difficult to overcome because you have so much control over what’s being loaded and how it’s organized in memory (for example, texture atlases). With gameswf, images are handled internally and they do 1 of two things:

  1. If the image is a non-power of 2 size, it will be resampled into a power 2 size using a software-based resizing algorithm. This can be a rather expensive performance hit if you don’t take this into account. The side effect as well is images that go down this pathway end up being rather blurry
  2. If the image’s height and width are a power of 2 then it uses the image directly.

My current solution? Just making image sizes a power of 2 before importing them into the Flash library. Yea it’s a lot of wasted space, but given how the UI functions currently, they aren’t long-lived objects so the memory is freed shortly after the user dismisses the screen or moves on. Of course, in the actual gameplay screen any flash-based UI elements have to be optimized. Sprite sheets aren’t exactly feasible in the traditional sense, but you can emulate sprite sheet functionality in flash by creating a movie clip that is masked differently on each of it’s frames. It’s tedious, but I imagine if you’re needing the extra memory savings this could be the way to go. There are a few other memory saving techniques I haven’t explored, like shared libraries that might provide some benefit as well.

In any case, if you’re targeting really low hardware (3G and below), you’ll most likely run into memory issues. The iPad 2 handled it just fine, but I imagine once I get down to the 3GS model and iPad 1 I’ll have to tighten my belt on the memory usage a bit, but it certainly seems doable.

Resolution

I had spent about a week coming up with a small body of photoshop scripts in conjunction with TexturePacker to help streamline our process for creating assets at different native resolutions. The upshot is we have the assets. The downside is now they can’t be used as-is. I don’t have a ‘swf generator’ to generate iphone, retina, and ipad resolution swfs. So right now it’s looking like it’ll be a completely manual process of us creating separate swfs for each platform/resolution. Of course, if our swfs were 100% vector graphics, this wouldn’t be an issue :).

I’ve pondered some possible solutions, but haven’t experimented with any yet. One idea I had after browsing some of the source was using the ‘scaling’ feature of gameswf to scale the rendering of swfs, but inside the actual flash files do some actionscript magic to display the correctly sized graphics in the UI. Not sure how much tedium this would save in the long run without some sort of automated way to do this. On one hand, Adam or I would be laying out 3 individual flash files to be used. On the other, we’d be laying out a single flash file but then I would be scripting the resolution changes in the file itself. I would have to come up with some sort of ‘utility’ style movie clips in the flash library to help with this, but maybe it could work better than managing 3 files for every UI screen?

Scalability

Given the tests I’ve done so far, there’s nothing there that has lead me to believe that doing a commercial quality game on the iphone/ipad with gameswf isn’t possible. In fact, I’m sure it’s probably been done by now and it just hasn’t been talked about? Which is kind of strange, considering how awesome it is. But maybe that’s just me geeking out. This isn’t so much as a concrete downside as it is more of an ‘unknown’. My concern is once we’ve got a lot of in-engine animations/particles running how will it fair with a flash player running the UI. The other caveat is Outwitters lends itself really well to this type of setup as it’s not a very fact paced game. So this solution might only be feasible for particular types of games (in the same way that using UIKit and CoreGraphics instead of OpenGL is feasible for some types of games). I guess only time will tell.

The Upside

Some of the positives of using gameswf particularly:

Lazy loading of images

When a bitmap is being rendered inside gameswf, the texture isn’t sent to openGL until the first time it is drawn. It’s a nice thing to know. We can kind of ‘stuff’ our flash files with image assets and not worry about them taking up tons of memory at runtime as long as we don’t render it all at once. This is useful for HUDs that are themed, in our case, by team colors, shapes, and logos. A point of caution is the the image *isn’t* unloaded from memory (even if you call an unloadMovie() from ActionScript on an externally loaded flash file).

Auto-rotation support seems feasible

Prior to this development I haven’t really considered doing anything too extravagant with support for auto-rotate. If we did support it, it would simply switch to the correct orientation and be done with it. But now it seems pretty trivial to have the UI do a fun but subtle animation as it stretches, squashes, or moves over to accomodate the different screen real-estate. It’ll be a ‘nice touch’ if we get that working.

It isn’t an all-or-nothing setup

Giving the sample project of gameswf for iOS a quick look it seemed at first that running flash was an all-or-nothing deal. It was initializing in the main method and pretty much took over how things were rendered onto the screen. This part was time consuming but after experimenting around with it a few days, I got gameswf to play nice with my already established framework for game screens and UI. Each screen in our game is just a subclass of a ‘Screen’ class that houses ui widgets and any logic particular to that screen. I was able to refactor a few things, and now I have just a subclass called ‘FlashUIScreen’ that takes in a flash file name and a ‘controller’ class as constructor parameters. It now functions and renders identically to anything I previously did. The point being, that if the need arose for something to render in-engine (perhaps using game assets already in memory for some sort HUD) I can still fall back to my previous method of hand coding UI. Hopefully that will be the rare exception than the norm.

Some Things To Know

It’s been a quick 2 weeks since I decided to investigate it and there’s still a few things left to answer, but currently I’m moving forward with the assumption we’ll be using gameswf for this project, and that is a fun thought to entertain :). For anyone else that is curious on getting gameswf up and running here are just some things I ran into while working with it:

If you’re loading separate movie swf files you’ll need a separate gameswf::player instance. I first went with the approach of having a single player and calling load_file() on anything I needed to render because I didn’t completely understand the relationship between a player and a gameswf::movie isntance. But the player tends to house the actionscript VM for a “root movie”, so loading 2 movies from a single player gave me some wonky actionscript behavior.

If you delete all the gameswf::player instances at any point in time, the library goes through a ‘complete clean up’ phase of gameswf. This game me some issues when I would transition from a main menu to an all OpenGL view (no flash UI) so the swf players were delete, and when I went back to a flash-based menu the app would crash with an EXC_BAD_ACCESS error. I got around this by instantiating a single gameswf::player instance that didn’t load any files and is held in my singleton service-based class. It’s only ever deleted when the app shuts down.

Getting the GL state setup correctly was probably the most time consuming part. The library worked great in isolation, but when combined with the framework I was working with, there were all sorts of states I had to track down in order to render a flash file and still be able to render openGL correctly right after. Below is just the snippet of the code I wrapped around a flash movie’s display() method.

Somethings to note, it likes a blank projection matrix (had to reset it to an identity matrix) and you have to disable a few client states in order to render flash properly. Once, you leave the display() method you need reset your gl color, blend functions, and possibly any other states you have may used with the assumption that they are enabled:

[code]
const int w = GetDisplayWidth();
const int h = GetDisplayHeight();

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
_movie->set_display_viewport(0, 0, w, h);
_movie->set_background_alpha(0);

glDisableClientState(GL_TEXTURE_COORD_ARRAY);
glDisableClientState(GL_COLOR_ARRAY);
_movie->display();
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glEnableClientState(GL_COLOR_ARRAY);
glEnable(GL_TEXTURE_2D);
glEnableClientState(GL_VERTEX_ARRAY);
SetGLColor(1, 1, 1, 1);
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);

[/code]

And with that, I’m off to do some more work on Outwitters. Adam and I just played a few games over lunch and I actually WON. Mark this day, for this is the first day I didn’t get completely rolled in my own game.

4 comments

  1. Hi, Alex.

    Your post is very interesting. I liked it a lot.

    I have some doubts:

    1 – Did you download the source from sourceforge and then you’ve ported it to run on iPhone or did you find the code already ported?

    2 – From sourceforge site I saw that the last update of gameswf was almost two years ago. Don’t you consider that use gameswf is a risk? I mean, it is not a very active project.

    Regards.

    1. @Marcio Glad you liked it. In regards to your doubts:

      1. There already was a ported version for iOS that uses an openGLES 1.x renderer. I just dropped the files into my project, modified some includes to fit XCode’s ‘way of doing things’ and it worked as-is.

      2. I believe 2010 was the last activity I saw in regards to igameswf, but it might’ve just been the mailing list. In any case, from what I’ve used so far it seems to be rather usable in it’s current state. But yes, if one is looking for ‘support’ while developing with it you’ll be out of luck.

      I still haven’t 100% committed to using gameswf yet mainly because of performance reasons on the lower-end hardware. On an iPad 2 gameswf runs flawlessly, but on ipad 1, if you’re not careful you can get some serious slow down. My current goal is to see what I can do to mitigate the performance hitches on an ipad 1, and if that fails, it’s off to write some sort of editor or offline tool to process the swfs into a format that is more runtime friendly πŸ™‚

  2. Check out Scaleform’s latest 4.0, which is super optimized for iOS and Android mobiles, as well as PCs and consoles, even 3DS and PSP2.
    http://www.flashmobileblog.com/2011/02/15/scaleform/

    It’s available for free as part of Epic’s UDK and there’s lower cost licensing for smaller projects.

    gameswf is a cool open source project if you want to learn the insides of a Flash player, but for any serious development, make sure to at least check out Scaleform. I think you’ll be blown away by the performance, features, and extra tools, samples, support that comes with the kit.

    — Brendan

    Just to be clear – I work for Scaleform, so I’m obviously biased πŸ˜‰

Leave a Reply

Your email address will not be published. Required fields are marked *