This project is read-only.

Please help with Android images compression and performances

Topics: Android, Content Building and Deploying, Sprites and SpriteBatch
Sep 29, 2014 at 5:03 PM
Hello,
i'm once again asking for your help :)

I have 2 problems, I should probably open 2 threads but they're somehow related.

PROBLEM 1:
On Android, I can work with XNB created with the XNA content processor, but the files size is HUGE! I ended up with a 80 MB APK, and I'm still at 75% of my content.
So I tried adding the PNG files directly to the project, instead of XNB files. Result?

XNB:
Image

PNG:
Image

Sadly, I lost part of the alpha channel (note: it's not gone at all, there's still part of it).

I looked around but it seems there's no solution for Android. So do I really have to distribute an APK 3 times larger than it should be? Users won't be happy (and me either!)

PROBLEM 2:

Trying to solve the size issues, I would like to use spritesheet instead of single images (then I could add DXT compression, if needed).
The problem here is the performances are incredibly bad!
Even in Windows Phone, passing from single images (100+ files) to a few spritesheets (approx 5 spritesheets), the performances drop from a stable 30 FPS to something between 5 to 15 FPS.

How is this possible? Everywhere I've read using spritesheet should improve performances, am I doing something wrong?


Thanks so much as always
Sep 29, 2014 at 5:28 PM
On the sprite performance front, make sure you are using CCSpriteBatch nodes. If you are making single CCSprite instances for every sprite drawn on the screen then you have a maximum capacity of about 100 sprites before it starts to suffer problems. The sprite batch node will make it enormously faster.

Next, android. The strategy is to combine non-alpha and alpha sprites into sheets. The non-alpha sheet can be JPEG encoded and added directly to the project as raw content (output always to output dir). This will save you an enormous amount of space. We did that with Mayan Epic recently and cut our APK from 61MB down to 26MB.

The alpha sprites are a little harder to deal with. You could try cutting them down to PNG8 with alpha. This would cut your pixel cost down by a third (8 bits instead of 24 bits), but you are using an index color space. That could lead to unpleasant artifacts on your edges. There are other PNG pixel formats you could use, such as PNG16 which is a 4x4 pixel layout. That produces a little bit better image quality. Another thing to consider is downloading the content after the game is installed.

JPEG doesn't do transparency, so you can't use it for your alpha channels.

Texture compression only affects the texture in-core. On disk, the size of the image will be the same.

The best sprite batch performance will be realized when you can place all of your on-screen sprites from the same texture. That would take some crafty organization on your part and some likely redundancy in the sprite sheets.

Your game looks very cool!
Sep 29, 2014 at 6:01 PM
Hi, thanks for your prompt answer.
I tried PNG8 using pngquant, but the result was the same (though in the website they say "pngquant works in premultiplied alpha color space").
My idea was a big spritesheet (PNG) compressed with DXT, which I understand works on Android.

I used spritesheets without spritebatch, this cause the performance loss. Is this normal? I thought even without spriteBatch it would be better than single sprites!

I know about spriteBatch but it's very difficulty to implement, as all the textures must be in the same file. Each room of my game is a texture of approx 200300, so a spritesheet of 20002000 could accomodate just a few.

I'm having good performances with single images, I just wonder why it's so bad with spritesheet.. I can't understand..
Sep 29, 2014 at 6:13 PM
Maybe I don't understand well, just a stupid example: how would you make a CCTableView showing a list of icons (50+ items) using the spriteBatch? Is there some sample around?
Sep 29, 2014 at 6:58 PM
I used spritesheets without spritebatch, this cause the performance loss. Is this normal? I thought even without spriteBatch it would be better than single sprites!
Yes, this is normal. Each time you draw a single sprite it has to load the texture. For the inefficient use of a sprite sheet with individual sprites you are loading the sprite sheet texture every time. That's an enormous amount of memory pressure on the command buffer.

Let's talk organization of the sheets.
  1. You have an adventure game with a dungeon that has background sprites for the tiles. those can all be in a sprite sheet. You can probably do some without transparency.

    CCSpriteBatchNode backgroundTileSet = new CCSpriteBatchNode(myBackgroundTileSheet);
    AddChild(backgroundTileSet, 0); // root of all drawing
  2. You have dungeon items that all sit atop the background, so they have transparency.
CCSpriteBatchNode stuffSet1 = new CCSpriteBatchNode(myStuffTileSet);
AddChild(stuffSet1, 1); // first set which the player walks atop
  1. You have player-level objects too, so they would sit at z=2 which would also be your player level
CCSpriteBatchNode stuffSet2 = new CCSpriteBatchNode(myStuffTileSet); // Note the same texture here, which is ok
AddChild(stuffSet2, 2); // first set which the player walks atop
  1. Now you have your player, which probably has a bunch of layered stuff like armor, weapons, and such.
PlayerSprite ps = new PlayerSprite(); // this is a complex sprite, not just a simple batch node.
AddChild(ps.BatchNode, 2); // Let the player sprite give me the batch node for drawing

Now you have a draw sequence that only tags the command buffer 3 times instead of 300 times. The game performance is linearly correlated to the number of times you flush the command buffer. Each set of commands will typically use just one texture, unless you have a beefy video card, and then you can do two, or three, or four. That's why having 300 sprites with different textures ends up leaving your hard up for better hardware. You're thrashing the command buffer.

As for CCTableView with a sprite batch, I don't believe that works. The ScrollView is designed to work with discrete nodes, not batched nodes. A sprite batched table view would be a good addition to the framework.
Sep 29, 2014 at 7:17 PM
Edited Sep 29, 2014 at 7:18 PM
Note that you can do the sprite batch node on a CCSCrollView:
    public CCScrollView(CCSize size, CCNode container)
    {
        InitWithViewSize(size, container);
    }
That means if you were going to use the sprite batcher for your table:

CCSpriteBatchNode myBatchNode = new CCSpriteBatchNode(myTexture);
new CCTableView(CCDirector.SharedDirector.WinSize, myBatchNode);

Now when you add the cells for your table, your cells will use sprites from the myBatchNode. You still add the cells to the table view, but the table view adds them to the myBatchNode instance instead of to itself.
Sep 30, 2014 at 3:25 AM
  1. Unfortunately the whole level is inside a ScrollView, to support scrolling and zooming. So if BatchNode doesn't live well with it, that would be a major issue.
  2. My understanding, using spritesheet withOUT batchnode, is that the texture was cached and re-used, The texture is loaded only once and then the same texture is drawn using the sprite CCRect. Then I don't understand the performances loss...
    If so, you mean I should only use spritesheet with batchnode, otherwise use single images for the rest of the game?
Thanks for your help, as always you're great. I will try more and more to see if I can improve performances.

So far the game runs well on average devices (tested on HTC One S, couple of years old), I'm just worry about older models with low memory (especially windows phone 256MB models).

Kind regards
Sep 30, 2014 at 4:40 AM
Edited Sep 30, 2014 at 4:41 AM
OK,
first try, first problem.

I create the node:
CCSpriteFrameCache.SharedSpriteFrameCache.AddSpriteFramesWithFile("game/sprites.plist");
BatchNode = new CCSpriteBatchNode("game/sprites");

I create the sprite (the image is in the sheet):
Sprite = new CCSprite(ImagePath);

This assert fails when adding the sprite as child of the batch node:
Debug.Assert(pSprite.Texture.Name == m_pobTextureAtlas.Texture.Name, "CCSprite is not using the same texture id");

If I comment the assert, I can see the graphics on screen.

Something wrong?
Sep 30, 2014 at 5:41 AM
OK Final update:
  1. Thanks to the batch node, I reduced the draw calls from 980 to approx 50. In low memory devices I passed from 2 FPS to 18 FPS (great!)
  2. I still have the problem mentioned before, the sprite batch doesn't recognize the texture as the same. I manged to run the game commenting this line, but it's an issue I would like to fix.
  3. Using DXT compression, my 2 spritesheet (1 with alpha, the other without) have become 2-3 MB instead of the 16 MB they were before.
    Now do you suggest I use JPG (sheet without alpha) + XNB (sheet with alpha, no dxt compression), or I keep using XNB with DXT compression? Will it be supported in every Android device?
  4. For the rest of the game (normal screens, dialogs, panels, etc.) should I use a spritesheet or single images?
  5. Negative note: using spritesheet + batchnode, the game takes a lot of time to load (10 seconds vs 2 seconds before). It takes a long time finding the textures inside the sheet using CCRect. This is because I re-use the same texture many times, but it does not cache the texture. Should I do this manually?
Thanks for the support, great job!
Sep 30, 2014 at 6:04 AM
Some useful code:
    private static CCSpriteSheet LoadSpriteSheet(string plist, string textureName)
    {
        return new CCSpriteSheet(GameResources.GetEmbeddedResource(plist), CCTextureCache.SharedTextureCache.AddImage(textureName));
    }
That will get the sprite sheet that you can use to solve your #2 problem.

To improve your load time:
    private static void PreloadTexture(string textureName)
    {
        CCTextureCache.SharedTextureCache.AddImageAsync(textureName, TextureLoaded);
    }

    private static void TextureLoaded(CCTexture2D texture)
    {
        // use this to keep track of progress
    }
Now loading a sprite from the sprite sheet:
        public static CCSprite CreateUISprite(string nameOfTexture)
        {
            CCSpriteFrame sf = GetFrameFromUISheet(nameOfTexture);
            if (sf == null)
            {
                CCLog.Log("Failed to load {0} from sprite sheet, using raw image.", nameOfTexture);
                return (new CCSprite(nameOfTexture));
            }
            return (new CCSprite(sf)); // This gets a sprite using the texture of the sprite sheet.
        }

        public static CCSpriteFrame GetFrameFromUISheet(string key)
        {
            CCSpriteFrame sf = null;
            int idx = key.LastIndexOf('/');
            if (idx > -1)
            {
                key = key.Substring(idx + 1);
            }
            string pngKey = key;
            if (pngKey.LastIndexOf('.') == -1)
            {
                pngKey = key + ".png";
            }
            string jpgKey = key;
            if (jpgKey.LastIndexOf('.') == -1)
            {
                jpgKey = key + ".jpg";
            }
            CCSpriteSheet sheet = null;
            if (_SpriteToTextureMapper.ContainsKey(key) || _SpriteToTextureMapper.ContainsKey(pngKey) || _SpriteToTextureMapper.ContainsKey(jpgKey))
            {
                string spriteSheet = _SpriteToTextureMapper[key];
                switch (spriteSheet)
                {
                    case R.Image.UISet1:
                        if (UISet1 == null)
                        {
                            LoadSheet1();
                        }
                        sheet = UISet1;
                        break;
                    case R.Image.UISet2:
                        if (UISet2 == null)
                        {
                            LoadSheet2();
                        }
                        sheet = UISet2;
                        break;
                    case R.Image.UISet3:
                        if (UISet3 == null)
                        {
                            LoadSheet3();
                        }
                        sheet = UISet3;
                        break;
                    case R.Image.UISet4:
                        if (UISet4 == null)
                        {
                            LoadSheet4();
                        }
                        sheet = UISet4;
                        break;
                    case R.Image.UISet5:
                        if (UISet5 == null)
                        {
                            LoadSheet5();
                        }
                        sheet = UISet5;
                        break;
                }
            }
            return (sf);
        }

        public static void LoadSheet1()
        {
            LoadSpriteSheetMapping();
            UISet1 = LoadSpriteSheet("ui_set1.plist", Image.UISet1);
            ByTextureName[Image.UISet1] = UISet1;
        }

        private static Dictionary<string, string> _SpriteToTextureMapper = new Dictionary<string, string>();

        public static void LoadSpriteSheetMapping()
        {
            if (_SpriteToTextureMapper.Count > 0)
            {
                return;
            }
            Stream s = GameResources.GetEmbeddedResource("ui_set1.plist");
            PlistDocument doc = new PlistDocument(s);
            PlistDictionary frames = doc.Root.AsDictionary["frames"] as PlistDictionary;
            foreach (string key in frames.Keys)
            {
                _SpriteToTextureMapper[key] = Image.UISet1;
                if (key.EndsWith(".png"))
                {
                    _SpriteToTextureMapper[key.Substring(0, key.LastIndexOf(".png"))] = Image.UISet1;
                }
                if (key.EndsWith(".jpg"))
                {
                    _SpriteToTextureMapper[key.Substring(0, key.LastIndexOf(".jpg"))] = Image.UISet1;
                }
            }
        }

    internal static Stream GetEmbeddedResource(string name)
    {
        System.Reflection.Assembly assem = null;

if !WINDOWS_STOREAPP

        assem = System.Reflection.Assembly.GetExecutingAssembly();

endif

        Stream stream = null;
        if (assem != null)
        {
            assem.GetManifestResourceStream(name);
            if (stream == null)
            {
                stream = assem.GetManifestResourceStream("My.Game.Namespace." + name);
            }
        }
        if (stream == null)
        {

if !WINDOWS_STOREAPP

            stream = typeof(GameResources).Assembly.GetManifestResourceStream(name);
            if (stream == null)
            {
                stream = typeof(GameResources).Assembly.GetManifestResourceStream("My.Game.Namespace." + name);
            }

else

           stream = typeof(GameResources).GetTypeInfo().Assembly.GetManifestResourceStream(name);
           if (stream == null)
           {
               stream = typeof(GameResources).GetTypeInfo().Assembly.GetManifestResourceStream("My.Game.Namespace." + name);
           }

endif

        }
        if (stream == null)
        {
            throw (new Exception("Failed to load embedded resource with name=" + name));
        }
        return (stream);
    }
I do suggest you use the JPG format for textures that do not need alpha channels.

Load time will be slow if you load the sprite sheets all in sequence. You should use asynchronous loading of the sprite sheets and stage them so that you can build the UI while the main sprite sheets are loading.

You should try to use sprite sheets and batch node for everything. First, graphics compression is most optimal when all of the graphics are combined into a single image. The more noise you have in an image, the better it will compress.
Sep 30, 2014 at 7:01 AM
Thanks, I will look into the code you provided.
Please note that it's not slow when it's loading the texture, it's slow when creating the sprite. I can see in the output log lots of
"cocos2d: tile001 frame CCRect : (x=260, y=1978, width=51, height=51)". Looks like he's cutting each sprite every time. I will check the source code to see what's going on.

For the spritesheets with alpha channel, can I safely release on Android with XNB/DXT Compressed? Will it be supported by every device? I was reading that it will uncompressed to Color format, but that was a 2013 thread, perhaps things have changed.
Sep 30, 2014 at 7:08 AM
It's using the cached texture, all right, but it hangs for 10 seconds showing in the console hundreds of lines like this:
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
cocos2d: tl_fog frame CCRect : (x=129, y=932, width=50, height=50)
I'm using the fog tile (tl_fog) 900 times in the game level, isn't it possible to avoid the whole "InitWithTexture" method, which seems to take forever?
Sep 30, 2014 at 2:20 PM
You need to create the sprite with the sprite frame, and then add that sprite to a sprite batch.

CCSpriteBatchNode fogNode = new CCSpriteBatchNode(fogTexture);
CCSprite sp = new CCSprite(fogSpriteFrame);

Now that's one way to do this. Another way to do this is to create a fog texture:

CCRenderTarget rt;

rt.Add(sp);
for(I=0; I < 900; I++) {
sp.Visit(); // draws the sprite in the RT
sp.Position = new CCPoint(x,y); // moves the sprite for the next draw
}

Then you just use the created texture for your fog.

DXT compression is available on all of the NEW android devices, probably since late 2012. Any old devices may not be able to use DXT compressed textures. These textures are not decompressed by MonoGame. They are passed to the framebuffer and then decompressed, otherwise DXT would be useless.

Now, on the topic of log output and performance. When in debug mode you may see your game appear to be slow, but that may well be from the debug logging. If you build in release mode and run it you will see proper performance.

Also note that sprite sheets are not cut in memory. When you use the batch node and a texture, a series of polygon commands are sent to the GPU. These polygons are defined in u,v texture space and in real space. That lets the GPU perform the texture splitting in hardware and not in software. Yet, if you do not use sprite batch then you are effectively sending one polygon per texture, which is very inefficient. The GPU will attempt to reuse the sprite sheet texture if no other texture is loaded. This all depends on whether or not the sprite batcher is in immediate mode, or not. When it is in immediate mode, draw commands are flushed immediately. When not in immediate mode, sprite commands are batched according to their textures. That allows you to effectively do a sprite batch node without using a sprite batch node. If you used that technique, then you would need to use VertexZ for your sprites, otherwise they would not draw in their proper z order.
Sep 30, 2014 at 3:07 PM
Thanks for the detailed explanation.
I will try to implement the code you provided. However, I feel like using the BatchNode in the game level was sufficient, I don't think further improvements are required at this time. I will do more testing with low memory devices to see if there are other weak spots.

I will also try to mix jpegs and dxt compressed png on Android. I must keep my app under 40/50 MB.

Just a quick note: I tried the new Pipeline tool of Monogame. I was able to compress most of the textures, and they're actually smaller than native XNA, but wasn't able to import the cocos2d processor. I believe it need to be compiled against MonoGame, am I right?

Thanks again and kind regards