FITC Amsterdam 2012 - HTML5: Life in the trenches

This is a short recap of a presentation made during FITC Amsterdam 2012. Kudos for the presentation content goes to the presenter. Please use the resource links at the end of this post to get more information from the source.

Grant Skinner, owner of gskinner.com, is an internationally recognized leader in the field of rich interactive experiences. After 10 years of Flash development, he and his team has spent the past 2 years trying to deliver similar experiences with HTML5. Grant gave a really interesting presentation about his experiences with HTML5 over these past years and an overview of a set of tools and libraries that he and his team have produced during that time to help ease the development process.

So why should you as a developer care about HTLM5?

• Opportunity, there is a big hype around it and now is the time to catch the HTML5 wave
• Reach, you will have a big reach and the perception is growing rapidly
• Experience, you can leverage your Flash skills to build rich internet experiences using HTML5
• Challenge, HTML5 is still in an early stage and still has many problems, you get to take part in solving these problems, and as a developer, solving problems is what we do best.

So what issues do we still have with HTML5?

• There are still inconsistencies between browsers.
• There is a constant flow of new specifications for HTML5 and it can be tricky to keep up with all of them, for developers and for browser providers.
• Performance is still a bit behind but it is improving a lot with hardware acceleration etc.

Some of the new API:s in the HTML5 world are:

• WebGL, a graphics library for the HTML5 canvas.
• WebAudio, audio library for use with canvas and WebGL.
• WebSockets, a new communication API for web server and browser communication.
• CSS Shaders, an advanced addition to filter effects proposed by Adobe.
• ... and more ...

Gskinner.com recenlty launched technitone.com - an audio experiment that makes use of a lot of these new APIs pushing the boundaries of what can be accomplished with modern browser technologies. Definitely worth a visit.

Grant continued his presentation with an overview of a set of open source libraries and tools that he and his team created to simplify development targeting HTML5. Create JS, available at createjs.com, is a set of libraries designed to be approachable, modular, extensible and open and has the following ingredients:

• EaselJS – A flash like API for working with the HTML5 canvas element. Features a hierarchical display list, DisplayObject's (Text, Container, Stage etc), a Ticker for controlling the progress over time and much more.
• TweenJS – A javascript library for tweens and animations. Uses javascript chaining and features synched timelines, easing, sequencing, overriding, plugins etc.
• SoundJS – Web audio library focusing on usability, features pluggable target APIs, progressive enhancement etc.
• PreloadJS – Library for managing and co-ordinating loading of assets with instantiable load queues, smotthed progress, multiple connections, uses XmlHttpRequest version 2.

All of these libraries are, or will be, open sourced and can be downloaded from createjs.com.

Apart from libraries, to get a better development experience when targeting HTML5, you need good tools. The ones shown by Grant Skinner were focusing on sprite sheet creation and animation:

• Flash CS6, Grant has been working with Adobe to be able to export sprite sheets from Flash CS6 targeting easelJS.
• Zoe, an open source tool from the CreateJS family for creating sprite sheets from SWF:s with scriptable animation etc.

A few things that Grant Skinner and the gskinner.com team are currently working on:

• Multi-surface targeting, rendering easelJS code to different rendering environmens, like canvas 2D, WebGL, SVG or even directly to the dom tree.
• EaselJS for NodeJS, render images on the server using client side code.

Grant finished his presentation with a few resources crucial to anyone who works, or would like to start working, with JavaScript:

• developer.mozilla.org – The closest thing you can find to an online API reference for JavaScript.
• caniuse.com – Lists browser support for various old and new HTML5 specifications.
• W3.org/TR – If you really want to dig into the formal core specifications of any HTML5 and CSS3 feature.

This was another great presentation by Grant Skinner, you can always see that he is very passionate about his work. He and his team are developing some really nice stuff that will prove very useful to any JavaScript developer in general, and will provide a smooth transition for the ones coming over from the Flash world in particular. And they are open sourcing all of it so that it can be leveraged and further improved by the community. That's the spirit!

Resources:

• gskinner.com
• createjs.com

FITC Amsterdam 2012: Open web technologies as an alternative to plugins

This is a short recap of a presentation made during FITC Amsterdam 2012. Kudos for the presentation content goes to the presenter. Please use the resource links at the end of this post to get more information from the source.

Michal Budzynski, a web game developer from Poland, shared some of his HTML5 come backs to typical doubts about HTML5 related technologies coming from Flash developers. Since Michal is not a Flash developer himself he had to ask some of his Flash developer friends about why they still preferred Flash over HTML5. These are some of their most usual reasons for favoring Flash coupled with plausible HTML5 alternatives suggested by Michal.

Flash developer: With Flash you can access the users camera.
Michal: HTML5 has the getUserMedia API.

Flash Developer: Flash can go full screen
Michal: With HTML5 you can call requestFullScreen on any page element

Flash developer: Flash can create desktop apps
Michal: You can also do that using XUL

Flash developer: There are Flash debugging tools
Michal: There are similar tools in browsers, firebug in Firefox, developer tools in Chrome, Venkman javascript debugger in FireFox (https://developer.mozilla.org/en/Venkman).

Flash developer: You can create P2P with Flash
Michal: You can use Web Sockets, Easy Web Sockets http://easywebsocket.org/ or upcoming WebRTC http://www.webrtc.org

Flash developer: For Flash there are services like Mochimedia and Kongregate for integrating with social media etc.
Michal: Similar services are starting to surface also on the HTML side, like bluevia.

Flash developer: Flash has DRM
Michal: Google are working on specs to secure web content. The open web community takes pride in staying open.

Flash developer: Flash has decent development tools
Michal: There are good alternatives for HTML5; cloud9ide.com, bly.sk, gordon/shumway and bikeshed.

All in all another one of those Flash vs HTML5 presentations. On several of the points mentioned above Flash is still ahead, HTML5 is catching up but there is still some time until new specifications turn into stable technologies with decent browser support. What I mostly take with me from this presentation is a few of the useful resources mentioned for HTML5 development, like the cloud9 IDE and the Venkman JavaScript debugger.

Resources:

• michalbe.blogspot.com

FITC Amsterdam 2012: Getting acquainted with jQuery

I believe I promised to post some recaps of the presentations I attended during my visit to FITC Amsterdam the other week. This is the first one, stay tuned for more.

This is a short recap of a presentation made during FITC Amsterdam 2012. Kudos for the presentation content goes to the presenter. Please use the resource links at the end of this post to get more information from the source.

Matt Fisher made a simple but useful introduction to jQuery. For anyone who hasn't used jQuery in a project before, like myself, this was a nice presentation for getting acquainted with the framework. This is what I got out of it.

jQuery is a JavaScript framework for boosting productivity for any JavaScript developer. It was first created by John Resig in 2004 and has been developed and improved since then. Current version is 1.7.1 and its main advantage over the widely used and stable version 1.6 is all about performance improvements.

jQuery is light-weight, widely used and accepted and should, according to Matt, be an integral component in the toolbox of any serious web developer.

The jQuery framework consists of the following parts:

• jQuery core – The core APIs provided by of the jQuery framework
• jQuery UI – provides useful user interface components like tree navigators and grids
• jQuery mobile – a relatively new addition to the jQuery family for supporting development for mobile
• Qunit – A unit test framework to use for java script development
• Sizzle – The selector engine for locating elements in the dom tree of an html page

...and its functionality can be divided into the following main areas of use:

• Modify CSS
• Alter HTML
• Respond to user interaction
• Animate elements
• Retrieve information from external resources

Short examples on each of the aforementioned functionalities are available from Matt Fisher's blog.

Matt also made a strong point about the licensing of jQuery and many other open source libraries out there. jQuery adheres to the MIT Licence and all you basically have to do to freely start using the framework is to keep that copyright notice at the top of the source files. If you remove the copyright notice or start copying and pasting parts of the framework and passing it off as your own you are violating the license and by doing that you are basically breaking the law. Perhaps that goes without saying but it was clear from the reaction of the audience that many of the present developers had been doing just that.

I asked Matt about the development environment that he would suggest for starting to work with jQuery or javascript and HTML development in general. As he had grown accustomed to using eclipse based development environments he recommended Aptana Studio. That description also applies to myself so I'll be checking it out.

Resources:

• fisherinnovation.com

FITC Amsterdam 2012: All about HTML5

There has been a while since my last blog post here. A lot of things have been going on in my life since then, I got a little kid, moved back to Sweden after several years abroad and I moved into a new apartment in Malmo. Finally things have settled a bit and I have been able to find some time to produce more content for the blog between remote working, diaper changing and home styling.

It's been about a year since I first started this blog with a visit report from FITC Amsterdam 2011. I was there again this year and this is what I got out of it.

Last years conference was mostly about the Flash vs HTML5 debacle. At that point HTML5 was still very new, browser support was bad, performance even worse and for application development Flex and Flash still seemed like the better choice.

A lot of things has happened over the past year and HTML5 has gained significant grounds in terms of technology advancements and general acceptance during that period. At the same time the uncertainties surrounding the Flash platform over the past few months have made decision makers more reluctant to choose Flex for their next project and are opting for HTML5 solutions.

This was also reflected in the program of this years conference. The vast majority of the technology related presentations were all focusing on HTML5. Even the keynote presentation by Adobe was all about what they are doing in the HTML5 field and you could see an evident change of focus and strategy from this big player in both fields.

So does this all mean that Flash is dead? Not necessarily. The message from Adobe is that the future of the Flash platform will be focussed on advanced gaming and premium video. The idea behind it being to not try to do everything and support every use case, but focus on one or a few main areas, and do them really well.

So for me as a developer using Flex, what is the way forward? Well I guess whenever you are presented with a new project it is still a question of choosing the right technology for the job. For a decently sized enterprise application project I would probably still go with Flex. Even if most of the things that you can do in Flex can now be accomplished with HTML5 related technologies HTML5 still has a long way to go in terms of development experience, tooling and browser support.

I would also keep an eye on what the community can do for the future of Flex now that it has been donated to the Apache foundation as an open source project. Maybe at some point we could even have a decent Flex to HTML5 compiler with a few tweaks in the language specification. The conference made me start thinking and I'll try to gather my thoughts on the future of Flex in a later blog post.

I had a great time at this years FITC Amsterdam, as did I last year. Interesting and relevant presentations, inspiring atmosphere and some very cool people. Add some free beers to the mix and you have a great event right there. I will definitely try to return again next year.

Stay tuned: Over the coming days I'll be publishing short recaps of the various presentations I attended during the conference, it will be almost as if you had been there yourself;

• Getting acquainted with jQuery, Matt Fisher
• Open web technologies as an alternative to plugins, Michal Budzynski
• HTML5: Life in the trenches, Grant Skinner
• Adobe keynote, Mike Chambers et al.
• Hello Sencha: HTML5 Applications, Ted Patrick
• Deep Dive in the Flash Platform Roadmap, Thibault Imber
• CreativeJS: Visual fun in JavaScript, Seb Lee-Delisle
• The future of HTML5 Motion Design, Mark Anders
• Influxis Voodoo Lounge, various presentations and presenters
• A Conversation with Mr. doob, Mr. doob, Seb Lee-Delisle
• The new playground: Adobe & HTML5, Deepa Subramaniam

Flex: dragDropEvent is not dispatched if feedback is set to none

A few days ago I posted about how setting the useRollOver style to false on a List control also prevents the itemRollOver event from firing on that same control:

Flex: Changing useRollOver style affects itemRollOver event

In other words, what I thought would only affect the visual appearance of a component turned out to also have an impact on the components logic. I realized that some time ago I ran in to that same scenario in the context of drag & drop. I thought I'd share that here as well.

So, basically, I was implementing some drag and drop functionality for our application. When the user started a drag action I changed some parameters in the application to reflect that a dragging was currently under way. When the user stopped his dragging I was relying on the dragDropEvent to restore the application to its original state or update it depending on the drop location.

The above was implemented and was working fine. Next step was to show different cursor icons depending on which areas of the screen the user was currently dragging over, the green plus for allowed drop locations and the red cross for prohibited locations. This was easily accomplished by using the DragManager.showFeedback function passing it DragManager.COPY or DragManager.NONE respectively. Unfortunately it showed that this change also caused the implemented logic to fail. The dragDropEvent handler was just not invoked if the feedback was set to none.

Looking back at it now, I suppose it kind of makes sense and my implementation became a bit more clear after reorganizing it to account for the encountered issue. Still, it took me a while to figure it out and I also couldn't find any mentioning of this 'feature' in the documentation. It just doesn't feel right when changing the visual feedback of your application also affects the functionality. MVC, anyone?

Flex: Changing useRollOver style affects itemRollOver event

The other day I ran into a bug in our application that was a bit unintuitive to track down. Suddenly the context menus used on a Tree component in our application started to behave weird. Anytime the context menu was opened the menu items in it referred to the first item in the Tree rather than to the item below the mouse cursor. To give some context, this is how the context menu logic for the Tree component is setup:

We have a variable called 'lastRollOverIndex' that is updated whenever the user hovers an item in the Tree. This variable is updated with the Tree index of the last rolled over item using the itemRollOver event of the Tree control. Whenever the user then right clicks the Tree control the lastRollOverIndex is used to get the item currently under the mouse cursor and the statuses and captions of the context menu items are updated accordingly before the context menu is shown to the user.

And now, seemingly out of the blue, the above functionality failed to work. So I started going through the recent change sets in our repository to try to find out what could be the cause of this problem. Interestingly, the search showed that no changes had been made to any files containing the logic for the above, the only change between a working and a non-working version of the application was some updates to the stylesheet.

But, could that really be? Updating some styles to changes the visual appearance of the application would also cause the logic of the same to fail? Well, answer is yes.

So one of the changes in the stylesheets was setting useRollOver to false for all List components to turn of the highlighting of items while hovering the list. Since a Tree is an extension of a List the change also applies to Tree components. So far, so good. But now we come to the unexpected part. Turns out, setting the useRollOver style also prevents the itemRollOver event from being dispatched which in turn killed the logic for our context menus. At least to me, this was not what I expected, and from what I could find in the documentation it is also not mentioned there.

So in order to disable the highlighting but still allow the itemRollOver events to fire I chose to override the drawItem function of the Tree control:

protected override function drawItem(item:IListItemRenderer,
    selected:Boolean = false,
    highlighted:Boolean = false,
    caret:Boolean = false,
    transition:Boolean = false):void
{
    super.drawItem(item, selected, false, caret, transition);
}

Flex: Custom context menu

Background

I never really liked the context menus supplied by the flash player that you are bound to use when developing flex applications. Mostly because of the non-customizable look and feel and the fact that your application specific menu items will be in the same list as the built in items (you can't hide all of them). Since we are using context menus in many places in our applications I have been looking at the following solution for a while to replace the built in menus:

http://code.google.com/p/custom-context-menu/

This solution uses javascript to capture the right click event over the SWF to prevent the flash player from opening its built-in context menu. The event is then passed on to a function that communicates with Flash over the ExternalInterface. The function called in action script is then free to do whatever it likes to act upon the performed right click.

The drawback of the solution is that it relies on using wmode='opaque' for the flash object and this does not work reliably across different browsers and operating systems.

Anyway, when running into the following bug with the latest Safari browser (5.0) on Mac:

http://bugs.adobe.com/jira/browse/FP-4825

I decided to go ahead and implement custom context menus, at least for our Mac users.

Implementation

The implementation relies on the solution mentioned above for handling the right click events and passing them on to the Flex application. An action script function then traverses the components currently located below the mouse pointer until it finds one that has a context menu defined. The found ContextMenu object then gets wrapped in a CustomContextMenu wrapper that uses the original context menu to create and show a regular flex menu.

The CustomContextMenu wrapper also makes sure that the enabled and visible statuses of the context menu items are also reflected in the items of the generated menu. Last but not least the wrapper makes sure that any user actions on the generated menu are dispatched as events on the original context menu so that the behavior of the original context menu is preserved in the new menu.

That's it. Enough talking. Let's look at the code, feel free to re-use it in any way you deem fit:

This is the code for the CustomContextMenu wrapper:

CustomContextMenu.as

package com.flexceptional.components
{
    import flash.events.ContextMenuEvent;
    import flash.ui.ContextMenu;
    import flash.ui.ContextMenuItem;
    
    import mx.controls.Menu;
    import mx.events.MenuEvent;
    
    public class CustomContextMenu
    {
        private static var SEPARATOR:Object = {type: "separator"};
        
        private var _menu:Menu;
        private var _contextMenu:ContextMenu;
        
        public function CustomContextMenu(contextMenu:ContextMenu)
        {   
            _contextMenu = contextMenu;
            
            // Create menu
            _menu = Menu.createMenu(null, null);
            
            // Add event listeners
            _menu.addEventListener(MenuEvent.MENU_SHOW, menuShowHandler);
            _menu.addEventListener(MenuEvent.ITEM_CLICK, itemClickHandler);
            
            // Set properties
            _menu.setStyle("openDuration", 0);
            _menu.variableRowHeight = true;
            _menu.id = "contextMenu";
            
            // Create custom menu items
            var menuItems:Array = [];
            for each (var item:ContextMenuItem in contextMenu.customItems)
            {
                if (item.separatorBefore)
                {
                    menuItems.push(CustomContextMenu.SEPARATOR);
                }
                menuItems.push(new CustomContextMenuItem(item));
            }
            _menu.dataProvider = menuItems;
        }
        
        private function menuShowHandler(event:MenuEvent):void
        {
            _contextMenu.dispatchEvent(new ContextMenuEvent(ContextMenuEvent.MENU_SELECT));
        }
        
        private function itemClickHandler(event:MenuEvent):void
        {
            event.item.dispatchEvent(new ContextMenuEvent(ContextMenuEvent.MENU_ITEM_SELECT));
        }
        
        public function show(xShow:Object = null, yShow:Object = null):void
        {
            _menu.show(xShow, yShow);
        }
        
        public function hide():void
        {
            _menu.hide();
        }
    }
}

import flash.events.ContextMenuEvent;
import flash.events.Event;
import flash.events.EventDispatcher;
import flash.ui.ContextMenuItem;

// ContextMenuItem wrapper
class CustomContextMenuItem extends EventDispatcher
{   
    private var _contextMenuItem:ContextMenuItem;
    
    public function CustomContextMenuItem(contextMenuItem:ContextMenuItem)
    {
        _contextMenuItem = contextMenuItem;
        
        // Dispatch event on the proxied ContextMenuItem
        addEventListener(ContextMenuEvent.MENU_ITEM_SELECT, function(event:Event):void
        {
            _contextMenuItem.dispatchEvent(event);
        });
    }
    
    public function get label():String
    {
        return _contextMenuItem.caption;
    }
    
    public function get enabled():Boolean
    {
        return _contextMenuItem.enabled;
    }
    
    public function get visible():Boolean
    {
        return _contextMenuItem.visible;
    }
}

This is the function invoked by the java script, place it in your main application (the optional x and y arguments can be used for automated testing):

private var customContextMenu:CustomContextMenu;

private function rightClickHandler(clickedX:Number = NaN, clickedY:Number = NaN):void {
    // Close previous context menu
    if (customContextMenu)
    {
        customContextMenu.hide();
    }
    
    clickedX = isNaN(clickedX) ? mouseX : clickedX;
    clickedY = isNaN(clickedY) ? mouseY : clickedY;
    
    // Get objects under clicked point
    var objects:Array = systemManager.getObjectsUnderPoint(new Point(clickedX, clickedY));
    if (objects.length > 0)
    {
        // Get the top most object clicked
        var object:Object = objects[objects.length-1];
        
        // Check if any of the parent objects have
        // a context menu defined
        while (object)
        {
            if (object.hasOwnProperty("contextMenu") && object.contextMenu)
            {
                customContextMenu = new CustomContextMenu(object.contextMenu);
                customContextMenu.show(clickedX, clickedY);
                break;
            }
            object = object.parent;
        }
    }
}

The code snippet for making the above function available through the ExternalInterface, place it in the initializeHandler of your main application file:

// Enable custom right click functionality
ExternalInterface.addCallback("rightClick", rightClickHandler);

We use the following javascript function to initialize the right click handling only for Mac users running Safari 5, place it in your html template and call it on the onload event on the body tag:

function initRightClick() {
    if (BrowserDetect.OS == "Mac" &&
        BrowserDetect.browser == "Safari" &&
        BrowserDetect.version == "5") {
        RightClick.init();
    }
}

The BrowserDetect object referenced in the above function comes from a file created using the code posted on this page:

http://www.quirksmode.org/js/detect.html

So, big post, feel free to comment on it if you have any questions or doubts. The solution provides a nice workaround for the Mac Safari 5 context menu bug and since our context menus can now be invoked by calling a function from javascript it also allows for us to do automated testing of our context menus using our integration test setup. More on testing in a later post.