Please help me - DxtCompression and Transparency

Topics: Content Building and Deploying, Performance, Windows Phone 7, Windows Phone 8
Apr 23, 2014 at 8:52 AM
Hi,
I'm not sure this problem is related to cocos2d, but perphaps someone of you can help.
It seems I'm the only one who cannot have DxtCompressed textures to work on PNG with alpha channel.

I'm on a windows phone 7 project, the image I want to DxtCompress is this:
Image

But when I run my game the result is this:
Image

I tried all the possible settings combinations. I also tried to premultiply alpha using PixelFormer, the result doesn't change. Blue background.

Can anyone give some advice? I can't find any help on the web, it seems to be working for everyone.

Kind regards
Coordinator
Apr 24, 2014 at 3:05 AM
We let the content pipeline builders handle the dxt compression. How are you explicitly enacting the Dxt compression?

Much of these nuances are handled by the framework so you don't have to deal with it.
Apr 24, 2014 at 3:33 AM
Hi, thanks for your reply.
I really believe it's not an issue related to cocos2d, but I can't find any help online.

I'm setting the DxtCompression option in visual studio, in the Content Processor of the texture. I have tried all the options without success. Tested on emulator 7.1, 8.0 and real device.
According to this post , the processor should use DXT5 when using PNG files.

I also tried a custom content processor as described here, the result is a red background at runtime.

Perhaps I should set a key color on my texture (e.g. Magenta) and set it in the KeyColor option of the DXTCompression in Visual Studio? Can't understand why there's no help at all about this important subject. My game is constantly crashing on low memory devices :-(

Regards
Apr 24, 2014 at 3:52 AM
Edited Apr 24, 2014 at 3:54 AM
Just tried setting a magenta background on the PNG and KeyColor in the processor options. The result is this:

Image

I've got the feeling it has something to do with BlendState of the graphics device.. but unfortunately I have no experience about it what so ever. Any idea?

BTW Where does this blue/purple back color come from?!?
Coordinator
Apr 24, 2014 at 5:09 AM
Edited Apr 24, 2014 at 5:13 AM
Purple background = standard XNA render target background. It's from the render target being in "discard contents" mode

blending state in the framework is set to "Alpha" by default when appropriate. You can see this in the CCDrawManager class.

How are you getting these images into your game? Are they XNB or raw or what??

I read your post, so you are using a Windows XNA content project on Windows Phone 7. You are setting the compression option to be DxtCompression.
Apr 24, 2014 at 5:17 AM

Yes, they are png files (standard 24 bit pngs) that I add to the visual studio project. Once built, they are converted by the Xna content processor into XNB files (which I believe are 32 bit RGB).
I load the images in game using the standard content manager load methos (CcTexture2D initWithFile).
I'm not loading from stream, it would take long time to update amd re-test my whole game.

Regards

Coordinator
Apr 24, 2014 at 5:31 AM
Could be that the framework is trying to convert to PMA format. I can't build the DtxCompressed option any longer on my Windows 8.1 laptop, so I can't test it out for you.

Go to CCTexture2D.cs, line 656. There you will see:
        if (OptimizeForPremultipliedAlpha && !premultipliedAlpha)
        {
            m_Texture2D = ConvertToPremultiplied(texture, format);

            if (!m_bManaged)
            {
                texture.Dispose();
                m_bManaged = false;
            }
        }
Here you want to bypass this conversion, if this is happening.

Also, if you are using InitWithTexture, are you specifying a SurfaceFormat ?? If you are DtxCompress format, then the surface format should be Dtx1, 2, or 3. Otherwise, if you specify Color (the default) then it decode the Dxt compression and you will likely lose the PMA mode on the texture, hence the loss of alpha.
Coordinator
Apr 24, 2014 at 5:40 AM
the problem appears to be at line 719 of CCtexture2D.cs
        if (texture != null)
        {
            // usually xnb texture prepared as PremultipliedAlpha
            return InitWithTexture(texture, DefaultAlphaPixelFormat, true, true);
        }
We are forcing the texture to SurfaceFormat.Color even when you specify the Dxt compression. This would work OK if we would preserve the alpha properly, but we are not doing that.

the ConvertSurfaceFormat method is the cause of the purple background:
        var renderTarget = new RenderTarget2D(
            CCDrawManager.GraphicsDevice,
            texture.Width, texture.Height, m_bHasMipmaps, format,
            DepthFormat.None, 0, RenderTargetUsage.DiscardContents
            );
Coordinator
Apr 24, 2014 at 6:07 AM
I pushed a change that may work for you:

https://github.com/Cocos2DXNA/cocos2d-xna/commit/c4a963e557ad88930395cee8b82ada078283c227?w=1

Let us know if this works....
Apr 24, 2014 at 8:30 AM

I already debugged it yesterday and checked that it's NOT going inside the IF block.
I also tried playing with SurfaceFormat, but I'm not sure which values I set, I will try to set Dx1 or dx5 as you suggested. Will let you know if it changes anything :-)

Thanks a lot
Regards

Apr 24, 2014 at 8:31 AM

Sorry I just read the second and third posts.
Will let you know if the changes are working, thank you!

Apr 24, 2014 at 10:36 AM
Edited Apr 24, 2014 at 10:37 AM
totallyevil wrote:
I pushed a change that may work for you:

https://github.com/Cocos2DXNA/cocos2d-xna/commit/c4a963e557ad88930395cee8b82ada078283c227?w=1

Let us know if this works....
It worked! Now I can just set DxtCompression option on my textures and they keep the transparency!
A small PNG like that, which was 40KB uncompressed XNB, becomes now 17KB Dxt Compressed, and the memory usage should be infinite less (I wish!).

If you are so kind, I've got just 2 more questions about cocos2d.

No. 1: (non vital, I already applied a patch)
CCDirector, line 1001, see my comment below
m_pNextScene.Visible = true;
s.Reset(t, m_pNextScene);
m_pobScenesStack.Add(s); _ //--> I have to comment this line, otherwise when going back the scene is added again to the stack, creating an infinite loop. NOTE: I'm using transitions between scenes, I don't know if this may be the problem.
m_pNextScene = s;
No. 2:
It happens in my latest game that, no matter how much the available memory in the device is, sometimes at a certain point of the game a texture becomes NULL, crashing the game.
I'm using a CCSprite. The property that becomes NULL is CCSprite.Texture.XNATexture . I believe perhaps it's a weak reference and the ContentManager thinks it can dispose it, but it's actually a vital texture.
So... is there a way to tell the ContentManager to NEVER dispose such texture? Strange thing it happens on devices with 1 or 2 GB of RAM, and the RAM usage at the point of the crash is just 20% of the available memory.


P.S. If you ever play my game let me know, I'll let you have some free bonus :-D
Coordinator
Apr 24, 2014 at 6:15 PM
Edited Apr 24, 2014 at 6:18 PM
on #1, I've not seen this problem but we don't have many test cases with the PopScene that uses a transition. You are probably correct about that one.

on #2, the GraphicsResource class uses weak references in MonoGame to manage all of the textures and such. If the texture goes stale because it has not been used, then it will get collected. When it has gone and the texture has been disposed, the content manager should recreate it if it can. You're on WP7 though, so it must be something on our side. If you have any log output when this happens, that would be nice to see.

Is this texture the result of using RenderTarget2D (CCRenderTarget), or just a CCSprite("foo.png") ??

Gena and I both worked on figuring out the DxtCompression problem yesterday. He gets credit for that fix as well.
Coordinator
Apr 24, 2014 at 6:25 PM
The purpose of the #1 change on our side was to keep the transition available if the game would go into the background.

Try switching out of the game when the transition occurs (tap the windows soft key). wait a few seconds, and then tap the back key. Without the add, I found that the resume would just show the next scene instead of completing the transition. That's what I remember.
Apr 25, 2014 at 1:59 AM

The problem with #1 is that without commenting the line, the user is unable to go back from any scene, I think this is more important than handling the softkey, but again this is not so important as I already fixed it.

#2 is the real issue for me.
The texture I'm using comes from a png file I load using CcSprite(filename).
I draw the texture at every loop, so it can't go idle.
I also put a check in the update method, if XnaTexture is null then re-create the CcSprite, but I believe the ContentManager keeps returning a Null reference.
It may be a memory issue, but it happens also on new device with lot of available memory.

Any suggestion/workaround? Or something I can investigate?
Unfortunately I'm not able to reproduce the issue, but I get 1500 crashes per day caused by this :(

Regards

Apr 25, 2014 at 4:23 AM
I'm for the simple things, I'm not looking for perfection, at least at this stage where the game has already been published.
So the solution I will test is as follows:
  1. Instead of referencing a CCSprite in my CCNode (and use CCSprite.Texture.XNATexture), I will reference the Texture2D object directly. In my case, I don't need a CCSprite, I'm drawing the texture using primitives.
  2. On each update I check that the textures have not been set to to NULL. In case they are, I will reload the Texture2D object, using the following method:
public static Texture2D ForceLoadTexture(string fileName)
        {
            try
            {
                //Try to get the texture from cache
                CCTexture2D ccTexture = CCTextureCache.SharedTextureCache.AddImage(fileName);
                if (ccTexture != null && ccTexture.XNATexture != null)
                {
                    return ccTexture.XNATexture;
                }
            }
            catch
            {
            }

            //If still null, try create the texture manually
            try
            {
                CCTexture2D ccTexture = new CCTexture2D();
                if (ccTexture.InitWithFile(fileName))
                {
                    if (ccTexture.XNATexture != null)
                    {
                        return ccTexture.XNATexture;
                    }
                }
            }
            catch { }

            //If still null, use the Cocosd Content Manager directly, no weak references
            try
            {
                Texture2D texture = CCContentManager.SharedContentManager.TryLoad<Texture2D>(fileName, false);
                if (texture != null)
                {
                    return texture;
                }
            }
            catch { }

            //If still null, use the XNA Content Manager directly, no weak references
            try
            {
                Texture2D texture = CCApplication.SharedApplication.Game.Content.Load<Texture2D>(fileName);
                if (texture != null)
                {
                    return texture;
                }
            }
            catch { }

            //There's nothing I can do, the game will crash
            return null;
        }
If this doesn't work... I don't know what else to do :-(
Coordinator
Apr 25, 2014 at 4:55 AM
What you are doing is exactly what CCTexture2D is also doing when the texture is lost (marked as disposed).

I am not sure why the root texture is null for you, but I am looking into it.

What is the game called?
Coordinator
Apr 25, 2014 at 4:57 AM
in the log, do you see this line being output:
        CCLog.Log("reinit called on texture '{0}' {1}x{2}", Name, m_tContentSize.Width, m_tContentSize.Height);
this is the only place where the texture is set to null and the CCTexture2D instance is not being disposed ...
Coordinator
Apr 25, 2014 at 5:07 AM
Edited Apr 25, 2014 at 5:09 AM
I also put a check in the update method, if XnaTexture is null then re-create the CcSprite, but I believe the ContentManager keeps returning a Null reference.
This tells me that you are running low on GPU memory and not system memory. If you reproduce it please post the log. Are the errors occurring on a particular model of phone? Maybe the HTC Windows Phone 7 device, which I found to be very short on GPU memory....

My guess here is that we are not disposing of textures properly and that's why you are running out of GPU memory.

have you tried to put your textures into a single sprite sheet? This would be immensely useful for low memory environments. 1 sprite sheet texture is far far less of a foot print than 100 textures that would all fit on a single 2048 x 2048 texture.
Apr 25, 2014 at 5:21 AM
totallyevil wrote:
I also put a check in the update method, if XnaTexture is null then re-create the CcSprite, but I believe the ContentManager keeps returning a Null reference.
This tells me that you are running low on GPU memory and not system memory. If you reproduce it please post the log. Are the errors occurring on a particular model of phone? Maybe the HTC Windows Phone 7 device, which I found to be very short on GPU memory....

My guess here is that we are not disposing of textures properly and that's why you are running out of GPU memory.

have you tried to put your textures into a single sprite sheet? This would be immensely useful for low memory environments. 1 sprite sheet texture is far far less of a foot print than 100 textures that would all fit on a single 2048 x 2048 texture.
Hi, allow me to reply to each point.

1. The game is called Tuning Cars Racing Online:
http://www.windowsphone.com/en-us/store/app/tuning-cars-racing-online/cc532295-f661-4731-a9ad-c875cce9f0f6

2. Unfortunately I cannot reproduce this error in any way. I tried fill up the memory, but what I get is an OutOfMemoryException, not textures being nullified. So I cannot check any log, except the crash report I get on google analytics.

3. The first thing I thought was a memory issue. I resized all my textures and sprites to the minimum, I added a low quality setting which disable parallaxes. Crashed have improved but there are still many.
I also added an event report when the texture is NULL, this is what I get (I got like 6.000 of this in one day):

PathTexture.Texture.XNATexture NULL.
ApplCurrentMemoryUsage: 99.250.176 AppPeakMemoryUsage: 101.126.144 AppMemoryUsageLimit: 223.768.576 DevTotalMemory: 381.054.976 AppWorkingSetLimit: 223768576 Model: Lumia 710

OR

Texture.Texture.XNATexture NULL.
ApplCurrentMemoryUsage: 40 808 448 AppPeakMemoryUsage: 49 098 752 AppMemoryUsageLimit: 157 286 400 DevTotalMemory: 414 617 600 AppWorkingSetLimit: 157286400 Model: RM-914_eu_russia_229

It happens on Lumia 710 (512 MB) as well as Lumia 1020 (2 GB of RAM!!) and Lumia 610 or 525. Basically any phone model.
From my reports you see that the app is using just 50-100 MB of memory, while the working set limit is 200 or 150 MB.

4. Why a spritesheet would be so much better? Isn't the size in pixels that matters? I may use a spritesheet combined with DxtCompression, that would definitely reduce the textures size in memory, but it means lot of changes to do.

Thank you
Apr 25, 2014 at 5:24 AM
One more thing, which class should I use to implement the spritesheet thing? CCSpriteFrame, CCSpriteSheet or CCTextureAtlas?
Coordinator
Apr 25, 2014 at 5:55 AM
I use CCSpriteSheet as it is the easiest to use.

Every GPU has a finite number of texture slots that can be loaded at any time during a draw. If you use a sprite sheet then you use only a small number of slots, something on the order of less than 5. In Mayan Epic, our current game, I converted it to use sprite sheets and all of the graphics fit into 5 2048 x 2048 sheets.

The benefit of using sprite sheets is performance. One texture means one hit for decoding and that means the GPU can go bull-crazy with applying the texture to any primitives. Each pull of a texture into the GPU memory space requires quite a bit of cycles. GPUs generally suck at memory bandwidth, but rock on reusing the same stuff a million zillion times each iteration. Reduce the number of texture loads and you increase the draw performance of your game in measurable ways.

organize the sprite sheet in terms of the scenes and game levels so that each visual scene uses the least number of sprite sheets. You want to keep it under three if you can.

you will be surprised at how easy it is to convert to using a sprite sheet. Create a common "get a sprite" method in a resource management class. Have that GetSprite method pull the sprite out of the sprite sheets that you pre-load. Then you replace the use of "new CCSprite(foo)" with SpriteManager.GetSprite(foo). that's easily done with search and replace. I use "Advanced Find and Replace" for this kind of stuff, but the VS "replace in files" works well.

you will need to get a plist for the sprite sheet. The PLIST generator from Texture Packer is 100% compatible. I use it without any doubts.

When I load a plist, I use it as an embedded resource. it's the easiest way that I found to manage plist files.

As for your last point about the pixel count. Try this. Copy 1000 files that are reach 1k. Then copy a single 1 MB file. Which is faster? That's how the GPU works with loading textures, only the GPU memory bandwidth is not 6 gps like your hard drive.
Apr 25, 2014 at 6:08 AM
Hi,
thanks again. I was reading about the spritesheet, I perfectly understand how useful it would be.

I'll give a try to this tool:
http://renderhjs.net/shoebox/
Which is free and claims to be compatible with cocos2d.

I hope this, combined with the code I posted above and DxtCompression will once and for all fix my problem.

My understanding is that you're not using DxtCompressed textures. Is it because they're not supported by iOS or there's another reason? It seems to keep the quality at a decent level (for mobile screens) reducing the size of about 50% (png with alpha).
Coordinator
Apr 25, 2014 at 6:17 AM
I don't use the DxtCompression because we haven't had the need for any of our games. I ran into the texture problem on one of our other games that had a bunch of animations in it and that's when I started to use CCSpriteBatch and sprite sheets. now we don't have the texture memory problem all that often. the memory error that we get from textures is when we try to preload too many (run out of texture slots).

Downloaded your game - plays well - very cool game. Great job! I posted a link to it in the Name your Game forum and also on facebook.

I've never used Shoebox. Many tools that claim to be compatible with cocos2d often do not output the correct format codes in the plist file. I found this to be true of Predator. If your plist doesn't load then the program is not compatible with cocos2d. Our plist parser does not forgive errors in the plist formatting.
Apr 25, 2014 at 6:19 AM
totallyevil wrote:
I don't use the DxtCompression because we haven't had the need for any of our games. I ran into the texture problem on one of our other games that had a bunch of animations in it and that's when I started to use CCSpriteBatch and sprite sheets. now we don't have the texture memory problem all that often. the memory error that we get from textures is when we try to preload too many (run out of texture slots).

Downloaded your game - plays well - very cool game. Great job! I posted a link to it in the Name your Game forum and also on facebook.

I've never used Shoebox. Many tools that claim to be compatible with cocos2d often do not output the correct format codes in the plist file. I found this to be true of Predator. If your plist doesn't load then the program is not compatible with cocos2d. Our plist parser does not forgive errors in the plist formatting.
Thank you very much for your support, you guys did a great job and will never thank you enough :-)
Apr 25, 2014 at 2:18 PM
My first test using a single sprite sheet instead of 5 separate textures for the car.
Not really encouraging:

5 separate textures:
ApplicationCurrentMemoryUsage: 21,729,280
With single sprite sheet:
ApplicationCurrentMemoryUsage: 23,592,960
Single sprite sheet/Dxt Compressed
ApplicationCurrentMemoryUsage: 25,235,456
Obviously I ran the test multiple times to check these values are average.

Any opinion?
Coordinator
Apr 25, 2014 at 5:58 PM
You have to compare the memory footprint of loading all 5 textures of the car versus the single sprite sheet texture.

The amount of actual pixel data used is going to be slightly different between the two depending on how the textures are packed on the sheet. Most GPUs will resize the textures to be power-of-two (that's why XNB files consistently have similar sizes).

What you are gaining is the use of a single texture slot, instead of 5. You have to use the GPU profiler to see the actual memory footprint improvement and performance boost. Application memory doesn't tell you much of anything useful in these kinds of mobile applications.

Are you using a sprite batch object to do the car drawing? Make sure you are doing that too. It will improve performance.
Apr 28, 2014 at 12:50 PM
On Android (using monodroid), I had some problems when using a big sprite sheet for the backgrounds (around 2048x2048 or so), so I ended up using sprite sheets for controls (buttons, icons) and vehicles (each vehicle has its own spritesheet).

I couldn't use SpriteBatchNode, because the order of drawing is: backbody / decals / frontbody. Unfortunately the decals cannot be in the same spritesheet as the vehicle back and front.

I also added an "else" statement to my method, so if the texture is still NULL I manually create a plain color texture using a 4 bytes array (2x2 px). This should defintely end the crashes.

:-)
Coordinator
Apr 28, 2014 at 5:10 PM
The sprite sheet honors the z-ordering of sprites. So you can put all of your sprites in one sprite batch and just given them proper z-order.

backbody : z=0-99
decals : z = 100-199
frontbody : z = 200 - 299

etc. I found similar problems with SpriteBatch so we have been working on improving its function so you can make better use with sprite stacking.

Oh, and I also had problems with large sprite sheets on the Nexus 7 and Galaxy Nexus. Some of the older android devices do not like the larger texture sizes, and they definitely do not like XNA Hi-Def, which allows for 4096 x 4096 textures.
Apr 29, 2014 at 2:02 AM
totallyevil wrote:
The sprite sheet honors the z-ordering of sprites. So you can put all of your sprites in one sprite batch and just given them proper z-order.

backbody : z=0-99
decals : z = 100-199
frontbody : z = 200 - 299

etc. I found similar problems with SpriteBatch so we have been working on improving its function so you can make better use with sprite stacking.
Sorry, do you mean I can have one SpriteBatchNode with sprites coming from different sheets?
I took a look at the source code and I saw the constructor of SpriteBatchNode accept only one texture, and in the AddChild method there's a check that the sprite you are trying to add matches the same texture you loaded in the constructor.
So I stopped as I believed I need one sprite sheet for everything.

This doesn't fit in my case, I need to keep separate the sheets for vehicles, upgrades and vinyls, otherwise I would be loading in memory a massive amount of unused objects.
Coordinator
Apr 29, 2014 at 5:18 AM
Only 1 texture per batch node. One thing we are considering is the "global Z" that was implemented in the cocos2d-x v3 implementation. That would allow the sprite batch nodes to respect eachother's z ordering of their children.
Apr 30, 2014 at 11:51 AM
totallyevil wrote:
Only 1 texture per batch node. One thing we are considering is the "global Z" that was implemented in the cocos2d-x v3 implementation. That would allow the sprite batch nodes to respect eachother's z ordering of their children.
Yeah guessed so. Then I must keep my sprite sheets separate and avoid using the SpriteBatchNode. Anyway with the latest changes I see now zero crashes due to NULL textures.
I still have some caused by Farseer engine (I don't use box2d), but probably the memory is still the issue there.
Anyway I'm far from the thousands of crashes per day I had before.

Thanks again for your help and your effort in this project.
Coordinator
Apr 30, 2014 at 4:13 PM
You should use our box2d. It's considerably faster than the farseer engine. If you need to use farseer, you should use our version, which also has many optimizations to make it faster.
May 1, 2014 at 3:03 AM
totallyevil wrote:
You should use our box2d. It's considerably faster than the farseer engine. If you need to use farseer, you should use our version, which also has many optimizations to make it faster.
I did not use your version of Farseer because it doesn't implement the WheelJoint, which obviously I need for my game. I assumed then that the original farseer source would have been better/more recent than yours.
It's really complicated to understand which source to use for monogame/cocos2d/box2d/farseer, I'm totally confused :(
Coordinator
May 2, 2014 at 9:45 PM
Our repository is designed to be entirely self contained. We optimized all of that source, including our own MonoGame fork, to work with the framework. If you pull in outside frameworks, then you lose some of our optimizations.

The car test is in both our farseer and box2d.