BTicino Wiring Devices, a Starling-based customisable catalogue

BTicino wiring devices for tablets and smartphones
Hi all
It’s almost a month that the “BTicino wiring devices” app is been released and I would to share my experience with you. In the app the user can choose his favourite BTicino plate, customize it and test on his wall.


My sharing desire comes from the fact that I put all myself in this app and I had the possibility of applying most of my knowledge to achieve the best perfomance possible in a textures-like-hell app (and I’m proud of the result). Sometimes I reinvented the wheel, sometimes I felt “alone” facing some issue but it’s the standard workflow when you work with some “all or nothing” graphic designers, the result is nice, especially if compared to lot of other apps/IUs you find on the stores… but not all went straight…

Download the app (I recomend to install the iOS version so you can avoid the long download wait described below in the document): Link to the download page

If you don’t want to download the app I made a video of the interface from the simulator of my Mac

 

Sorry for the long post, I hope this summary may help you
Summary
Tech specs
About the app and the update releases
More and more images, less and less RAM
Scaling images… by hand?…
The app is too much heavy…
Creating the interface
No one difference between the smartphone and the tablet UI???
The stage bounds
Bitmap fonts
Stealing the TextField BitmapDatas
The first demo doesn’t run
Flash Builder can’t debug on Android 4.2.2+
AIR can’t save photos using CameraRoll on Android
Perfection from the beginning if you want AAA performances
Tweens duration (custom Juggler.advanceTime)
What about ANEs
The infamous Android .obb file
Most of the Android devices download very slow
You must be able to communicate directly with the user
Is AIR the unique framework with these problems?
Developing apps == developing software
More features
In the end

 

Tech specs
At the time I’m writing the app is based on:
• Adobe AIR 3.8
• Starling 1.3 (but some optimizations come from the still unreleased 1.4 version)
• Flash Builder 4.7
• TexturePacker for the atlases creation
• Transporter Chief to install the app directly on iOS devices https://github.com/PrimaryFeather/Starling-Framework/tree/master/util/transporter_chief

 

About the app and the update releases
First of all: the target of the app was iOS and Android (3.0+), every device (from the smallest android/iPhone 3GS to the newest iPad 4/Nexus 10) with every resolution need to be supported. I didn’t like the idea of releasing two versions (standard and HD) of the app for two reasons:
• damn, it’s 2013, the tablets have been around for a lot of years
• more apps means more tests/attention when you have to release them. You can minimize as much as you can the work needed to release an update but you can’t avoid to test ALL of them carefully. Because when the app is on the store… is on the store and you can’t nothing but wait for the next approval (iOS) if you made a mistake. It’s not impossible to reduce the amount of work needed for the release but it depends also on the app… basically you must be sure that:
• the users that upgrade the app mustn’t notice anything
• every features that you supported in the previous version must degrade gracefully if no more supported
• every new feature can’t be influenced from the previous app features (unless you want it)
• every local data you saved in the previous app needs to be migrated to be sure that are compliant with the new update
• if you’re making a catalogue app (like mine) you have to be sure that there aren’t problems removing/adding new items
• you must test and test the update on every device, you will miss some bugs, it’s sure, but you have to minimize them
In my mind the multiplatform development (that have lot of OS-related problems, device-related problems, screen-related problems) isn’t an exact science, I hope (and leave a comment, don’t fear to blame me) it’s easier for you.

 

More and more images, less and less RAM
The app is a catalogue of the BTicino’s plates (yes, the ones surrounding the electric switches) splitted by lines, shapes and colors. More less there are 130 different plates in the app. Managing the images, texturing them etc. has been the most time-spending task (after the development one).
The problem (the first one): how to make the images look nice on all the screens without destroying the device performance?
Answer: easy, you have to create lot of different images, some for every resolution you want to support. While testing on the devices you will notice if you need to support more resolutions. A fitting images set means better performances, no images scaling etc. so you have to maximize this kind of experience for the most possible devices.
The app supports eight different resolutions, three for iOS and six for Android (one is shared). For “resolution” I mean that I group the screens by height (in pixels) because the app is landscape only. So the iPhone 4 and the iPhone 5 could use the same images (both are 640 high).
The chosen resolution are:
• 320: iPhone 3GS and lot of small Android devices
• 480: lot of medium-sized Android devices (like the Nexus S etc.)
• 600: most of the 7” Android tablets have a 1024×600 screen
• 768: the iPad 1 and 2 (the iPhone 4 and 5 use them either scaling them)
• 800: most of the 10” Android tablets have a 1280×800 screen
• 1020: the new popular resolution of the medium-sized Android devices and the tablets is 1920×1080 (Full HD)
• 1536: the iPad 3
• 1600: the Nexus 10 and the next Android tablets
So all the images are scaled by hand (more less…) to support the highest possible number of devices. All the other screens simply bring the nearest images set and scale it (for example the iPhone 4 brings the 768 images and scale them down).

Pay attention: the still-unreleased devices are been considered! If tomorrow Samsung releases a 4K tablet (3840×2160 pixels) the app use the biggest ones (the ones designed to fit the 1600 screens in my case) scaling them up. Visually they won’t be perfect… but it’s better that crashing on a new device, what do you think? 🙂

p.s. FYI the graphic designers of Creeo Studio creates 3200×1800 psds when we have to create multiplatform apps… 1800 because is the highest screen “considered” now (the Mac Book Pro retina) and 3200 because… it’s the 16:9 proportion of 1800. So they can simulate all the ratios (simply cropping the psd) and the developers have already the biggest images possible.
p.p.s. I haven’t used the ATF format because of the images artifacts but moslty because of the lack of time

 

Scaling images… by hand?…
The problem: what I told you above means that I have to create eight versions of every image:
• the UI images: 45
• the wall images: 28
• other stuff: 35
• the plate images: ~1500…………………………..
Well I managed to leave unresized the first two (to avoid bad effects from the Photoshop cropping) and because they’re a very limited amount of images, for the remaining 1535 images I wrote a Photoshop extension that makes it for me in a while. I used the Adobe Extension Builder writing the extension directly in Actionscript, now the Javascript version of the Extension Builder is included into the Creative Cloud so have fun with it! The API are simple, creating an extension for PS is straightforward.
The extension I wrote: https://github.com/NoriSte/multiplatform-images-photoshop-extension

Then I used the command-line version of TexturePacker to create all the texture atlases.
About the plate images: the plates must be empty (without the buttons that could be customised by the user) so I cropped them in four different images to save lot of space in the atlases (so a lot of RAM when loaded).

The original plate and the splitted one

One of the atlases

 

The app is too much heavy…
Well with all those images the app was very heavy at the first demo (from 300MB to 500MB based on the version) so accordingly with the client we removed some of them. We reduced the number of images of the plates because after some tests we realized that on the retina screens the difference between the plate previews was not so different if… if we removed the retina previews, the heaviest ones. This change gave a kick to the app weight, now the apps are from 90MB to 130MB without any big differences for the eye.

(EN have you realized how difficult was managing all the images? Now starts the code chapters finally 🙂 )

 

Creating the interface
Build the interface for a multiplatform app is fascinating, the things when your mind has to get used (compared to a standard website interface) are:
• think in percentage: pixels are dead the day that Apple introduced the iPhone 4 and the iPad 2
think in millimeters: all the interactive parts of the UI (buttons, text inputs etc.) must have a minimum size in millimeters, yes: millimeters… Because a button on the iPad 3 could be the 10% the high of the screen but on a iPhone it can’t. So the button needs to be the 10% of the screen UNLESS it’s small than… 10 mm for example. It’s easy to calculate the millimeters using the Capabilities class.
In the case of my app: it’s a landscape one so everything has a height in percentage related to the height of the screen. The width is not a problem, most of the interface is the same indipendently from the width, some stuff adapt itself to the width… but generally I think in percentage and ratios
One advice: it’s always better to retrieve the correct DPIs of the screen from the serverString instead of using the ad-hoc variable because it rounds too much the original value

urlVariables = new URLVariables();
urlVariables.decode(Capabilities.serverString);
screenDPI = parseInt(urlVariables.DP, 10);

But if you’ve ever developed a mobile website etc. there’s nothing new to you.
think about ratios: I work without caring about the screen ratio, all the UI needs to support every ratio from 3:2 (the iPhone 3 and 4) to 16:9 and so on without limits. Ratios smaller than 3:2 don’t worry me but I haven’t tested them 😉
• think relative: the point is clear, you can’t take anything for granted… and nothing can be referenced by your interface as a saving-anchor

 

No one difference between the smartphone and the tablet UI???
More less… yes! Just the share panel changes based on the screen size.

 

The stage bounds
You base all your life on the stage size, after months of development I can say you
• when the app starts the first dimensions of the stage are almost everytime wrong
• wait a while (300 milliseconds in my case) then you consider the stage size valid
• don’t wait (like me at the beginning) for a second stageResize event… because some devices (like the Samsung Galaxy Tab 10.1) doesn’t fire a new stageResize. The Samsung tablet has the correct stage size since the beginning but it’s an exception.

 

Bitmap fonts
With Starling the best choice are the bitmap fonts, they’re fast and light but… but the only con is that they should be used at the exact size you’ve exported them. So if you export your bitmap font at 200 size (like I made at the beginning of the app based on the UI psds) you can’t use it at 190 size for example… because some unwatchable (both on iOS and Android) pixels appear instead of the sexy curves of the font… and in the multiscreen development you can’t know what will be the dimension of the font when the device will run your app… simply because you can’t know the device that will run it.
So I switched to TrueType fonts, no choices. But, wait: a small amount of texts on the screen can destroy the performances because they’re not grouped in a single atlas… Starling transforms them in a Bitmap and uploads to the GPU but one redraw for every text is needed! So…

 

Stealing the TextField BitmapDatas
Looking in the TextField class of Starling I developed an own solution:
• in the createRenderedContents method the BitmapData instances are created then uploaded to the GPU
• I added a static renderedContentsThief property to the class that “steal” the BitmapData instances of the texts

var texture:Texture = Texture.fromBitmapData(bitmapData, false, false, scale);
if(renderedContentsThief !== null)
	renderedContentsThief.apply(null, [bitmapData, text, color]);

In another class…

TextField.renderedContentsThief = textFieldBitmapDataSaver;

the “thief” function saves them

private final function textFieldBitmapDataSaver(bitmapData:BitmapData, text:String, color:uint):void {
	if(bitmapData.width === 1 || bitmapData.height === 1)
		return;
	textFieldBitmapDatas ||= new Dictionary();
	textFieldBitmapDatas[text] ||= {};
	textFieldBitmapDatas[text][color.toString()] = bitmapData.clone();
}

then it adds all the BitmapData instances to a standard Sprite, it gives them a nane and it creates an atlas using the DynamicAtlas class (https://github.com/emibap/Dynamic-Texture-Atlas-Generator)

sprite = new flash.display.MovieClip();
for(key in textFieldBitmapDatas)
{
	for(colorKey in textFieldBitmapDatas[key])
	{
		bitmapData = textFieldBitmapDatas[key][colorKey];
		bitmap = new Bitmap(bitmapData);
		bitmap.name = key + "_" + colorKey;
		sprite.addChild(bitmap);
	}
}
textsAtlas = DynamicAtlas.fromMovieClipContainer(sprite);

finally it destroys the original TextField and BitmapData instances.
Ta-dah: I have all the texts coming from the same atlas, I need only to put them at the same depth and only one redraw is needed for all the texts.

All you have to do during the original TextField creation is to request the textBounds property

text.textBounds;

You don’t need it but requesting it the TextField render is been forced (and the thief called).

In my app only two texts need to be changed frequently, for them I used the standard starling.text.TextField class.

 

The first demo doesn’t run
AIR is amazing, with a well design-oriented mind the device become more less nothing more than “test it at the end”. You’ll fight “just” with some performance issues based on how perfect your app is. In my case:
• the wall images were too big and not scaled for every device
• the same is valid for other images that need to be optimized to run well on the iPhone 3GS and the iPad 1
• I changed my image loading algorithm: from a “load all” one to a “load them one by one” by one algorithm
Ahead on the development I managed with my colleagues to create some patterns for the wall images to save lot or precious RAM, the final result was great.

 

Flash Builder can’t debug on Android 4.2.2+
Boring but easy to resolve, see here


and then connect your android device as a Camera, Flash Builder will debug on the device without any problems.

 

AIR can’t save photos using CameraRoll on Android
Very boring but even easier to resolve: you need another permission in the manifest

 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

 

Perfection from the beginning if you want AAA performances
Here http://wiki.starling-framework.org/manual/performance_optimization you will find all the optimizations that Starling needs to go fast and fast. They’re a few, Starling makes the developer life easy for real. Apply them since the first line of code (and consider them in the analysis phase) remember that with so many devices in the world with so bad CPUs/GPUs you must write the nearest-to-perfection code for your app. Otherwise your app will run good just on the iPhone 5 and the iPad 4… on Android… no, forget it, a non-perfect app doesn’t run on Android at all.

 

Tweens duration (custom Juggler.advanceTime) [UPDATED]
On low-powered Android devices the UI tweens were pretty nice but too slow, why? Because if you use the Starling provided Tween classes they depends on the Juggler class that counts the duration of a tween in frames and not in milliseconds that’s not right, I made a mistake while writing, the fault is of a snippet of mine that at the beginning didn’t manage the low-FPS case. The considerations are still valid but the fault is not of Starling but of a piece of my code wrote to keep all the control over the tweens duration in my hands.

What does it mean? It means that if you set a tween to last for example 500 milliseconds and your app (setted) frame rate is 60 FPS the tween will last 30 frames no matter if the app is actually running at 15 FPS because of a no performant device. So the tweens are always smooth but they can be very slow when the frame rate drops down.
In my app every “section” had its own Juggler instance so I made a change to speed up the tweens on the worst devices (sacrificing the fluidity of the tweens).


protected function stage_enterFrameHandler(event:Event):void {
var actualFrameRate:Number;
var frameDuration:Number;
var timer:int;

timer = getTimer();
frameDuration = isNaN(_lastTImer) ? StarlingMain.frameDuration : (timer – _lastTImer) / 1000;
actualFrameRate = 1 / frameDuration;
if(actualFrameRate < 30) actualFrameRate = 30; frameDuration = 1 / actualFrameRate; juggler.advanceTime(frameDuration); _lastTImer = timer; }[/AS3] I advance the time of the juggler of the actual frame time instead of the hypothetical one (with a minimum of 30 FPS).   What about ANEs
If I can I avoid them, I hate that a so-subtile project depends from a third-party script that probably isn’t tested on all the existing devices. Writing them all my own… I avoid it if I can because I’m sure that I’ll miss something related to the single platforms (where I don’t have a super-deep experience). Don’t blame me, I’ll use them in a near future but when I can I avoid them.

 

The infamous Android .obb file
When I uploaded the app to the Android Play I got the surprise: Google removed the 50MB limit for the apps… but it doesn’t mean that you can upload a 200MB apk! Google Play accepts a 50MB apk and two additional .obb files (nothing more than two .zip files) both up to 2GB.
The Google Play download the .obb files together with the app when the user downloads the app itself (the user doesn’t know it, he see just a “download in progress”) and your app has to manage the files extraction, so:
• you have to localize the .obb files into the device memory (easy)
• you have to open the .obb files (easy with the FZip class http://codeazur.com.br/lab/fzip/)
• you have to extract all the files and save them in another place to use them in your AIR app (easy)
It’s all so easy… but all the Android devices with less than 1GB of RAM crash extracting a 70MB .obb file… so in the end I managed to download all the exceeding assets from the BTicino servers and save them locally the first time the app runs.
p.s. just because I switched to another solution (because of the device crashes) I haven’t mentioned you that Google says that it’s not 100% sure that the .obb files are automatically downloaded from the Google Play… in this case you should use an ANE, “authenticate” the app with the Android Play and then manage your own the .obb download…

 

Most of the Android devices download very slow
Well, I managed to download all the assetsbut another problem occurred: the network was very fast (both Wi-Fi and 3G) and the device browser downloads at the correct speedrate… but the app downloads the files at a ridiculous rate… after lot of tests and lot of apps checked (The Simpsons Tapped Out and Real Racing 3 suffer the same issue) I’ve given up and I accepted the slowness.
That’s the original Twitter conversation

 

You must be able to communicate directly with the user
I learnt that when I developed my first app… keep open a communication channel between you and the users because if something happens… you need to wait the Apple approval before you can tell something to your users. So we managed to have some json files on the BTicino servers, the app reads them everytime it starts and it can show an alert saying what we want. Straightforward, directly to the users.

 

Is AIR the unique framework with these problems?
I think AIR is amazing (and I know it’s better than lot of other languages/frameworks) but I admit that I feel a bit scared at every AIR update, every Flash Builder update, every new OS release, every… everytime I’m not sure AIR runs well. I’d like to know if someone of you have experience with PhoneGap, Titanium, Corona etc. etc. and if your feeling is the same.

 

Developing apps == developing software
Today you can develop apps easily and fastly but don’t forget that they are SOFTWARE, nothing related to a website or a Flash game. The fact that you can develop easily doesn’t mean that developing is easy. Lot of complications come from the device management, the ratios management, the screens management, the stores management, the update management etc. etc. I don’t want to scare you but just to let you think twice for your next time/money estimates 😉

 

More features
A unordered list of feature the app includes:
• the app status is saved locally (every 5 seconds), when the user reopen the app he finds it just
• I used Google Analytics (the web version, without ANEs) to track the stuff, Flox was in beta when I developed the app
• for the Russian language (still not included because the russian headquarter still have to send us the localizations) I used a different font

<application android:hardwareAccelerated="true" />

in the manifest seems to be needed to make the AIR apps working nice also on the Nexus 10
• I managed the different versions with the conditional compilation parameters of the AS compiler
• last but not least: your app won’t be bug-free, develop it isolating every single block and manage every possible case so if something go wrong the app won’t crash
In the next release
• the Chinese language will be supported
• we decrease the images quality as we can to remedy the Android slow download rate (now the user waits up to 20 minutes…)
• Android 2.3 will be supported

 

In the end
I’m happy I developed the BTicino Wiring Devices app, it’s the nearest-to-perfection (in terms of performance) product I’ve ever developed but the best side of our work is that the perfection moves further everyday.
Starling is simply amazing, an app like that would never have reached such performances without it (and spending so little time worrying about the FPS).
See you 🙂

11 thoughts on “BTicino Wiring Devices, a Starling-based customisable catalogue

  1. I think you could have avoided problems with having multiple atlases for every resolution if your designer had supplied you with AI files. You then import AI to Flash Pro, name every layer as movieclip, publish SWF the app would then process in the runtime with DynamicTextureAtlas It’d be done just once . Then app would save generated texture atlas(es) to local storage and access it later.

    DynmicTextureAtlas saves you time and money! )

    • The problems with atlases don’t come from the UI itself but from the plate images (moreless 1500 for every resolution) that weren’t available in AI (the client provided them) but only in psd.
      Thank you for the reply 🙂

  2. Excellent read, I just passed this specific onto a new colleague who had been doing a small research with that. And he truly bought us lunch because I found it regarding him smile So permit me to rephrase that.

  3. Hi, thank your for great tips. But I have one question on you.

    I’m working on weather application based on Starling Framework.where I need display many information in scrolling item list. As you wrote in article above, if you need more sizes of fonts, different colors etc. bitmap fonts isn’t ideal so I’m using Starling TextField and TrueType fonts but it causing performance issues when I have on stage about 200 Textfields (60FPS falls to just 30 FPS) . So I tried to make very similar solution as you using in your app. I saved in Starling TextField class bitmapData before are uploaded to GPU and created Dynamic Texture Atlas, all worked as I wanted. But when I add some Texture from this dynamic Atlas to stage texture has right width and height as original TextField also has transparent background but text isn’t showed. Please if you have some idea where can be problem I will be very grateful to you.

    Thanks

    • The following is a piece of code of the text creation

      [AS3]
      text = new TextField(1, 1, string, Fonts.BTCOND_REGULAR_FONT, 60 * AssetManager.singleSizeAssetsScale);
      text.autoSize = TextFieldAutoSize.HORIZONTAL;
      text.minMillimetersHeight = 2;
      text.fitText();
      text.color = Color.BLACK;
      text.textBounds;
      [/AS3]

      The last line (text.textBounds;) is very important (sorry if I hand’t written in the post) because forces Starling to render the text with the classic display list then create the bitmap.

      Let me know if it works for you
      Stefano

      • Hi Stefano,

        sorry for my delayed response I was quite busy last few weeks but today I returned to project and I tried your advise. Now it works for me so thank for your help but I have trouble with final textures they are blurred and pixelated when I added them to stage. I thought that problem would be affected by scaling during creation of bitmapData from starlling textfield or during creation of dynamic atlas so I set constant scale factor to 1 but result is the same. But probably it will be mistake on my side. So thank you again for help.

        • Don’t worry. Double-check that the textfields nor one of their parents are scaled, it’s strange that the library used “scale” the atlas/images.
          Anyway thank you if you’ll update us about your progress.
          Stefano

  4. > Because if you use the Starling provided Tween classes they depends on the Juggler class that counts the duration of a tween in frames and not in milliseconds.
    What does it mean? It means that if you set a tween to last for example 500 milliseconds and your app (setted) frame rate is 60 FPS the tween will last 30 frames no matter if the app is actually running at 15 FPS because of a no performant device

    Maybe I’m misunderstanding, but the code in Starling doesn’t seem to agree with you. Every frame, Starling measures how many milliseconds have occurred since the last frame, and it uses that value to advance the juggler, and the juggler passes that value onto the tween. Except for a possibly minor delay for the last frame running long, a tween that is meant to finish in 500 ms will finish in 500 ms regardless of the framerate. If your app were running at only two frames per second, but your desired framerate was set to 60, then the tween would advance to the end after one frame. It would not run for 30 frames.

    • Josh you’re right… I assigned to Starling the fault of a script of mine…
      How it happened:
      – my app depend on a custom Juggler (not the Starling main one) so I can control it all my own (10x the speed for dev purposes, slowing down the speed to simulate bed devices etc.)
      – the first version doesn’t managed the FPS slowing down
      [AS3]
      [Inline]
      protected function stage_enterFrameHandler(event:Event):void {
      juggler.advanceTime(StarlingMain.frameDuration);
      }
      [/AS3]
      – the second version managed the not-60FPS case
      [AS3]
      [Inline]
      protected function stage_enterFrameHandler(event:Event):void {
      var frameDuration:Number;
      var timer:int;

      timer = getTimer();
      frameDuration = isNaN(_lastTImer) ? StarlingMain.frameDuration : (timer – _lastTImer) / 1000;
      juggler.advanceTime(frameDuration);
      _lastTImer = timer;
      }
      [/AS3]
      – then the last version avoid to show very bad tweens in case of very low FPS
      [AS3]
      [Inline]
      protected function stage_enterFrameHandler(event:Event):void {
      var actualFrameRate:Number;
      var frameDuration:Number;
      var timer:int;

      timer = getTimer();
      frameDuration = isNaN(_lastTImer) ? StarlingMain.frameDuration : (timer – _lastTImer) / 1000;
      actualFrameRate = 1 / frameDuration;
      if(actualFrameRate < 30) actualFrameRate = 30; frameDuration = 1 / actualFrameRate; juggler.advanceTime(frameDuration); _lastTImer = timer; } [/AS3] - and, when I wrote the post, I completely forgot that all the tweens depend on my own code and not on Starling I'm sorry, I fix it, thank you for reporting 🙂

Leave a Reply to maho125 Cancel reply

Your email address will not be published. Required fields are marked *