Dynamic Theming in Silverlight with Implicit Styles (plus Custom Build Events)

November 25, 2009

Recently I’ve  had a few questions about how to dynamically set or change the visual theme for a Silverlight application at runtime.  The scenario is generally that an application has many possible themes, and needs to pick one  depending on, for instance, a particular user or client who has logged in.  What if these themes are huge, or there are hundreds of them?  You probably wouldn’t want to bloat the downloaded .xap file by including every theme for every user.  So, some approach is needed that lets us target a specific theme to download – at runtime – that isn’t in the application’s resources.

For our simple example, let’s say we have two different customers for a website – SuperAwesome Co. LLC, and SomeOtherCompany Inc.  Both of these customers have their own internal users that will be using our site, so the site needs to be branded with the company’s name and colour scheme depending on who logs in.  We don’t want to include both custom themes in the application’s .xap file, because then users will always be downloading bits they don’t need! 

As our site scaled out to include more users and companies with custom themes, this would degrade the experience for every user as they waited for downloads (not to mention contributing to universal entropy production).  Clearly we should split the themes out of the application and into separately downloadable components.

So to summarize, I’d like to build an example that supports:

  • Theming an entire Silverlight app at runtime using implicit styles
  • Downloading a theme dynamically, without needing to include the theme in the app’s resources/.xap file
  • Adding/changing themes without recompiling the Silverlight app
  • Showing a “loading” screen while the right theme downloads
  • Automatically compressing and deploying resource dictionaries with a post-build event in Visual Studio

Silverlight enables a few approaches to this problem, and many users have posted or blogged solutions for specific pieces of this scenario.  However, I’m still writing this for a few reasons:

  1. I’ve still had some feedback that it can be a bit tricky to put the pieces together – e.g. downloading a zipped XAML theme using a WebClient, unpacking the zip, reading the XAML theme and instantiating a ResourceDictionary, merging it into the application resources, and applying it across an app.
  2. To give an example of using custom build actions in Visual Studio.
  3. To show off the new implicit styling support in Silverlight 4 Beta!

Step 1: Creating the App

Since we’ll be using a WebClient, we need to have an active webserver to handle requests – so we’ll create a new Silverlight application hosted in a web app (the default).  After everything is done, it will look something like the below.  We’ll get to the various files in a bit.

image

Note I’m using Visual Studio here so I can take advantage of custom build events, but this would work equally well in Blend or another tool if you don’t mind manually compressing the themes (or writing an external script to do it).

Let’s also mock up our sample UI in MainPage.xaml:

   1: <Grid x:Name="LayoutRoot" Background="White">
   2:     <TextBlock Name="textBlock1" Style="{StaticResource headerTextBlockStyle}" HorizontalAlignment="Center" VerticalAlignment="Top" />
   3:
   4:     <StackPanel Margin="0,90,0,94" HorizontalAlignment="Left" Width="167">
   5:         <Button Content="Home" />
   6:         <Button Content="Sales Forecasting" />
   7:         <Button Content="Purchase Order History" />
   8:     </StackPanel>
   9: </Grid>

Pretty simple – a header TextBlock with an explicit style defined (note it’s not in the control’s or application’s resources – we’ll be downloading it!), as well as some buttons which will be styled implicitly.  It looks pretty bland as-is:

image

Step 2: Creating Themes

You’ll notice the two .xaml files in the themes directory in the web project folder – these will be our themes for SuperAwesomeCo and SomeOtherCompany.  They’re pretty standard resource dictionaries that can contain any mix of explicit and implicit styles:

   1: <ResourceDictionary
   2:     xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
   3:     xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
   4:     xmlns:sys="clr-namespace:System;assembly=mscorlib"
   5:     >
   6: 
   7:     <!-- explicit style (has a key) -->
   8:     <Style x:Name="headerTextBlockStyle" TargetType="TextBlock">
   9:         <Setter Property="Text" Value="SuperAwesome Co. LLC" />
  10:         <Setter Property="Foreground" Value="Orange" />
  11:         <Setter Property="FontWeight" Value="Bold" />
  12:         <Setter Property="FontSize" Value="15" />
  13:     </Style>
  14: 
  15:     <!-- implicit style (no key - just a TargetType) -->
  16:     <Style TargetType="Button">
  17:         <Setter Property="Foreground" Value="Orange" />
  18:         <Setter Property="Background" Value="Yellow" />
  19:     </Style>
  20: </ResourceDictionary>

Note the implicit style for Button – in Silverlight 4 beta, creating a style without a key will apply that style to all instances of the target type in an application, unless they already have an explicit style set (or if they’re in another control’s template).

Step 3: Custom Build Events: Packaging Themes as .zip Files

Our sample themes are pretty trivial, but they could potentially be quite large – a theme for a big app might have a large number of explicit styles, and could easily reach hundreds of kilobytes in size.  Since they’re just .xaml files that contain text, they should compress well – so let’s turn them into standard .zip files, which the web readily recognizes as application/zip MIME types.

We could compress the files manually, but given that we’ve built the themes in Visual Studio anyway, why not create a custom build event to do it automatically.

Steps to accomplish this:

  1. Get a commandline tool for compressing files.  I prefer 7-Zip, which is an open-source program that has a standalone .exe for command line use.  Make sure to get the Command Line version (7za.exe).  Notice I added this to the web project in the solution explorer screenshot above.
  2. Add a custom build action to the web project to compress each theme using Z-Zip.  In the Build Events section of the web project properties, add the following:

for %%a in (”$(ProjectDir)themes\*.xaml”) do “$(ProjectDir)7za.exe” a -tzip “$(ProjectDir)ClientBin\CompressedThemes\%%~na.zip” “$(ProjectDir)themes\%%~na.xaml”

The result should look like:

image

Breaking this down into parts:

for %%a in      -loop through each file, storing filename as variable “a”

    (”$(ProjectDir)themes\*.xaml”) – find .xaml in <projectdir>/themes

do

   “$(ProjectDir)7za.exe” a –tzip  – call 7-zip to add (“a”) to zip file (“tzip”)

          “$(ProjectDir)ClientBin\CompressedThemes\%%~na.zip” – zip file

          “$(ProjectDir)themes\%%~na.xaml” – source .xaml file

Note that the special syntax “~n” in “%%~na.xaml” means take the only the “name” part of the filename from the %%a variable and omit the .xaml extension.

This will find every .xaml file in the themes folder and turn it into a compressed .zip file in the ClientBin/CompressedThemes folder every time we build the project.

Step 4: Downloading and Applying Theme

We’ll need to determine what theme to download when the app starts – this could be based on a login form, policy, or any number of things.  For the purposes of our example, let’s just assume we’re SuperAwesomeCo and want “theme 1.xaml”.

There’s a few things to do here – download a compressed theme zip file, uncompress it, parse out the theme and create a ResourceDictionary, and apply the theme across the application.  Thankfully, we can do all that with a few lines of code in App.xaml.cs – no need to change the xaml or code for any content pages.  The code is fairly straightforward:

   1: private string _themeZipFilePath = "CompressedThemes/theme 1.zip";
   2: private string _themeContentFileName = "theme 1.xaml";
   3: 
   4: private void Application_Startup(object sender, StartupEventArgs e)
   5: {   ...
   6:     // download a target compressed theme file
   7:     WebClient webclient = new WebClient();
   8:     webclient.OpenReadCompleted += new OpenReadCompletedEventHandler(webclient_OpenReadCompleted);
   9:     webclient.OpenReadAsync(new Uri(App.Current.Host.Source, _themeZipFilePath));
  10: }
  11: 
  12: void webclient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
  13: {
  14:     if (e.Cancelled)
  15:         return;
  16:     else if (e.Error != null)
  17:         return;  // error handling here
  18: 
  19:     // retrieve XAML file from the zip package
  20:     StreamResourceInfo zipInfo = new StreamResourceInfo(e.Result, "application/zip");
  21:     StreamResourceInfo themeInfo = Application.GetResourceStream(zipInfo, new Uri(_themeContentFileName, UriKind.Relative));
  22: 
  23:     // read and load the XAML file as a ResourceDictionary
  24:     string themeXaml = new StreamReader(themeInfo.Stream).ReadToEnd();
  25:     ResourceDictionary theme = XamlReader.Load(themeXaml) as ResourceDictionary;
  26: 
  27:     // merge ResourceDictionary into the application's resources            
  28:     Application.Current.Resources.MergedDictionaries.Add(theme);
  29:     ...
  30: }

When the app starts, we create a new WebClient instance to download the zip file.  Note this is why we needed the web project while debugging – there has to be a server to handle the request.  We could also have used a webservice to retrieve the themes, but a direct download with WebClient serves our purposes nicely for this example.

Once the file has been downloaded, we initialize a StreamResourceInfo with an application/ zip content type.  Since we then know it’s a zip file, we can grab the xaml file out of the zip file by calling GetResourceStream, and read it using a StreamReader.

Finally, we can use XamlReader to parse the file as a ResourceDictionary, and merge it into the application resources MergedDictionaries collection – this will make its content available to the entire application (making the headerTextBlockStyle static resource lookup for an explicit style in MainPage.xaml valid), and automatically apply any implicit styles it contains! 

Step 5: Niceties: Displaying a Splash Screen While Downloading

Even a zipped theme might take a perceptible amount of time to download.  To mitigate this, we can make a splash screen that will display while the theme is downloading so the user doesn’t see the ugly unthemed page during the initial load time.

For testing purposes let’s just create a simple SplashScreen control, and in SplashScreen.xaml we’ll just show a “loading” message:

<Grid x:Name="LayoutRoot" Background="White">
    <TextBlock Text="Loading..." HorizontalAlignment="Center" VerticalAlignment="Center" FontSize="18" />
</Grid>

Now, let’s add to our theme-loading code in App.xaml.cs in order to display the splash screen while the theme is loading:

private void Application_Startup(object sender, StartupEventArgs e)
{
    Grid root = new Grid();
    root.Children.Add(new SplashScreen());
    this.RootVisual = root;
 
    // download theme
    ...
}
 
void webclient_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
    // apply theme
    ...
 
    Grid root = this.RootVisual as Grid;
    root.Children.Clear();
    root.Children.Add(new MainPage());
}

Note we can only set the RootVisual once, so we’ll initialize it to an empty grid.  When the app first starts, we’ll add a SplashScreen to the grid.  After the theme is downloaded and merged into the application’s resources, we’ll clear the SplashScreen and create our real application root (MainPage). 

Voilà!  SuperAwesomeCo employees will now see a custom-themed application.

image

We can modify these themes (or add new ones) at any time without requiring any recompilation just by modifying the contents of the theme zip files.

Download

Source code:

NB: I didn’t include the 7-Zip executable, so you’ll want to grab the command line version from the 7-Zip download page and add it to the ExternalResourceSample.Web project if you want to use the custom build event to automatically create zip files.

Note this is a Silverlight 4 Beta project built in Visual Studio 2010 Beta 2, so content and APIs are subject to change.

5

Surface multitouch gestures and inertia sample for Silverlight

November 23, 2009

I just wanted to call attention to the fact that the Microsoft Surface team recently released a manipulation (gestures) and inertia sample specifically for Silverlight!

On the back of the Microsoft Surface SDK 1.0 SP1 Workstation Edition that was recently announced at PDC for creating and testing Surface applications on a regular desktop computer, this separate Silverlight-specific example shows off a great sample managed library, System.Windows.Input.Manipulations.dll, which supports a similar set of gestures as my previous sample – Rotate, Scale, and Translate.  It also includes support for inertia mechanics – smoothly decelerating a moving element after it has been released by the user, plus handling boundary feedback (when a moving element “bounces” off an application-defined boundary such as a window edge).

The API is similar to the regular version of the Surface SDK, and is equivalent to the .NET Framework 4.0 version – it implements the Manipulation Processor and InertiaProcessor, plus supporting classes.  Note it’s just a sample, so is intended only for educational purposes.

Get the library and sample code here:

http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=4b281bde-9b01-4890-b3d4-b3b45ca2c2e4

I’ll be posting a more in-depth look at this, plus some sample code of my own using the new library, in the near future.

2

Implementing a ContextMenu in Silverlight 4 Beta

November 18, 2009

The Silverlight team receives a lot of feedback about features that users would love to see, and right click support was quite high on the list (currently #2).    So, I’m pleased to say that right click events are fully supported in the new Silverlight 4 beta release recently announced at PDC!

It’s clear there are a number of useful applications for this feature aside from context menus, but menus are definitely one of the major uses cases – so, I’d like to go through a sample implementation of a reusable end-to-end context menu implementation that can be used in any Silverlight application.

Highlights of the features the custom menu should have:

  • An attached property that lets us apply a working ContextMenu to an element in XAML without writing code.
  • ContextMenu, MenuItem, MenuItemSeparator controls that can be retemplated to suit your application’s look and feel.
  • Click events for MenuItems, plus Command and CommandParameter support for MVVM applications.
  • Support interaction with both mouse and keyboard.  For example:
    • Moving between menu items: mouse, tab key, up/down arrow keys
    • Selecting a menu item: clicking, space key, enter key
    • Closing menu: click (right/left) anywhere else in app, escape key
    • MenuItemSeparators shouldn’t be selectable
  • For familiarity’s sake, an API similar to the WPF ContextMenu.

Sample look and feel:

image

1.  The Events

The two new events that are raised for all UIElements are MouseRightButtonDown and MouseRightMouseButtonUp.  These are routed events, similar to the well-known left mouse button events.  The first step to implementing a context menu will be to hook up handlers for these events, and do something with them.

MouseRightButtonDown

  • In order to suppress the normal “Silverlight” context menu that will still pop up by default, we need to mark this event as handled to stop the routed event from bubbling up the visual tree.  We can do this by setting the Handled property to true in event args:
private static void element_MouseRightButtonDown(object sender, MouseButtonEventArgs e)
{
    e.Handled = true;
}

You might notice this differs slightly from WPF, which requires you to handle the Up event – the difference here is to support platforms where context menus are opened on button down instead of up.

MouseRightButtonUp

  • This is when we’ll open the context menu.  Read on for details..

2.  Class Overview

Here’s a simplified class diagram showing the main classes we’ll create:

image

  •  ContextMenu
    • This is the main class that will contain and display the menu items.  Notice the ContextMenu attached property – this will allow us to attach a menu to any element and automatically hook up mouse event listeners as needed to show and hide the menu without writing any code in the app.  This class will have a custom template that can be modified/overriden to suit any application.
  • MenuItemBase
    • This is the base class for any item added to a ContextMenu.
  • MenuItemSeparator
    • By default this element should draw a single horizontal line to separate menu items (customizable by changing the template).  It shouldn’t be selectable.
  • MenuItem
    • This class represents a selectable option in a ContextMenu.  Its content can be set using the Text property, and you can either listen to the Click event or supply a Command (and CommandParameter) which will automatically be executed.

3.  Sample Usage

Once you include the ContextMenu assembly in your application, all you have to do to hook up a context menu is set the ContextMenu.ContextMenu attached property:

   1: <TextBox Text="hi there, world!">
   2:    <menu:ContextMenu.ContextMenu>
   3:        <menu:ContextMenu DataContext="{StaticResource myViewModel}">
   4:            <menu:MenuItem Text="option 1" Command="{Binding MyCommand1}" />
   5:            <menu:MenuItem Text="option 2" Command="{Binding MyCommand2}" />
   6:            <menu:MenuItemSeparator />
   7:            <menu:MenuItem Text="option 3" Click="MenuItem_Click" />
   8:        </menu:ContextMenu>
   9:    </menu:ContextMenu.ContextMenu>
  10: </TextBox>

Pretty clean – no app code needed!  Notice items can either specify the Command property (in this case bound to commands in my sample ViewModel), or handle the Click event (don’t worry, keyboard selection counts as clicking).  Don’t forget to declare the prefix for the ContextMenu control assembly:

<UserControl x:Class="ContextMenuExample.MainPage"
    xmlns:menu="clr-namespace:ContextMenuControls;assembly=ContextMenuControls"
    ...
>

Since ContextMenu, MenuItem, and MenuItemSeparator are custom controls, you can also retemplate them in XAML to change the look and feel.

4.  Drilldown: ContextMenu Class

Here’s a simplified listing of the main members of the ContextMenu class (some  omitted for brevity):

   1: [ContentProperty("Items")]
   2: public class ContextMenu : ToolTip
   3: {
   4:     #region ContextMenu attached property
   5:     public static readonly DependencyProperty ContextMenuProperty =
   6:         DependencyProperty.RegisterAttached("ContextMenu",
   7:                                             typeof(ContextMenu),
   8:                                             typeof(ContextMenu),
   9:                                             new PropertyMetadata(null, OnContextMenuChanged));
  10: 
  11:     private static void OnContextMenuChanged(DependencyObject obj, DependencyPropertyChangedEventArgs e);
  12:     #endregion
  13: 
  14:     private ObservableCollection<MenuItemBase> _items;
  15:     private ItemsControl _itemContainerList;
  16:
  17:     private void OnLoaded(object sender, RoutedEventArgs e);
  18:     private void OnOpened(object sender, RoutedEventArgs e);
  19:     private void OnClosed(object sender, RoutedEventArgs e);
  20: 
  21:     public ObservableCollection<MenuItemBase> Items { get; }
  22:     private void _items_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e);
  23: 
  24:     private void MenuItem_Selected(object sender, EventArgs e);
  25:     private static void element_MouseRightButtonDown(object sender, MouseButtonEventArgs e);
  26:     private static void element_MouseRightButtonUp(object sender, MouseButtonEventArgs e);
  27: 
  28:     private void OnKeyDown(object sender, KeyEventArgs e);
  29:     protected override void OnMouseRightButtonDown(MouseButtonEventArgs e);
  30:     private void OnMouseDownElsewhere(object sender, MouseButtonEventArgs e);
  31: }

Starting off: ContextMenu inherits from ToolTip.  That gives us a good starting point, since the positioning logic and “popup” behavior are already done (thanks to Dave Relyea for some pointers on ToolTip use here).  Also note the Items property is attributed as the ContentProperty, so that adding items to the ContextMenu in XAML is supported without needing to add the <ContextMenu.Items> tags around them every time.

ContextMenu attached DependencyProperty, OnContextMenuChanged

This allows us to attach a ContextMenu to any given FrameworkElement.  Thanks to the OnContextMenuChanged method, which is invoked whenever this property is set or unset, we can automatically start or stop handling the right mouse button events whenever this property is changed on a given element.

Items, ItemContainerList

The Items collection, which contains zero or more MenuItemBase objects to be displayed, is implemented as an ObservableCollection so that we can receive notifications when items are added or removed from the collection and update the displayed items accordingly.  ItemContainerList is a reference to an ItemsControl in the ContextMenu’s template, which will be used to display the items inside the menu. 

We could have made this a bit easier by using a ListBox to get the selection and update notification behavior for free, but implementing our own collection change handling with a generic ItemsControl gives us better control over styling, selection behavior, and API exposure.

OnLoaded

One quirk that results from using a ToolTip is that the menu will default the IsHitTestVisible property to false every time it is loaded.  Since we obviously want to be able to click on the menu, we set IsHitTestVisible to true here.

OnOpened, OnClosed

The main work to be done here is to attach and detach event handlers to the plugin’s root element when the menu opens and closes (respectively), because we want to close the menu whenever the user interacts with something else in the application.  Using the UIElement.AddHandler(..,.., true) API, we can ensure we’ll still receive the events even if they’re already handled by other elements.

OnKeyDown

Most events will be handled at the MenuItem level, but some keyboard routed events that bubble up will be handled by the ContextMenu, since they deal with moving focus or closing the menu.  The code is fairly self-explanatory:

   1: private void OnKeyDown(object sender, KeyEventArgs e)
   2: {
   3:     switch (e.Key)
   4:     {
   5:         case Key.Escape:
   6:             // close menu
   7:             IsOpen = false;
   8:             break;
   9:         case Key.Up:
  10:             // select next item in up direction
  11:             ChangeFocusedItem(false);
  12:             break;
  13:         case Key.Down:
  14:             // select next item in down direction
  15:             ChangeFocusedItem(true);
  16:             break;
  17:     }
  18: }

Since the TabNavigation mode of the ContextMenu is set to Cycle by default, moving “up” past the first item will move focus to the last item, and vice-versa.

Template

The ContextMenu template (defined in themes\generic.xaml) has the ItemsControl used to hold items, and two VisualStates – Open and Closed.

   1: <ControlTemplate x:Key="ContextMenuTemplate" TargetType="local:ContextMenu">
   2:     <Canvas x:Name="Root" Opacity="0">
   3:         <VisualStateManager.VisualStateGroups>
   4:             <VisualStateGroup Name="OpenStates">
   5:                 <VisualStateGroup.Transitions>
   6:                     <VisualTransition GeneratedDuration="0:0:0.2" />
   7:                 </VisualStateGroup.Transitions>
   8:                 <VisualState x:Name="Closed" />
   9:                 <VisualState x:Name="Open">
  10:                     <Storyboard>
  11:                         <DoubleAnimation Storyboard.TargetName="Root" Storyboard.TargetProperty="Opacity" To="1" Duration="0" />
  12:                     </Storyboard>
  13:                 </VisualState>
  14:             </VisualStateGroup>
  15:         </VisualStateManager.VisualStateGroups>
  16:
  17:         <Border BorderBrush="Gray" BorderThickness="1" Width="150">
  18:             <Border.Effect>
  19:                 <DropShadowEffect ShadowDepth="2" Color="Gray" />
  20:             </Border.Effect>
  21:                 <ItemsControl x:Name="ItemList" Background="White" TabNavigation="Cycle" />
  22:         </Border>
  23:     </Canvas>
  24: </ControlTemplate>

 

5.  Drilldown: MenuItem Class

Simplified API listing:

   1: [TemplateVisualState(Name = MenuItem.StateNormal, GroupName = MenuItem.GroupCommon)]
   2: [TemplateVisualState(Name = MenuItem.StateActive, GroupName = MenuItem.GroupCommon)]
   3: public sealed class MenuItem : MenuItemBase
   4: {
   5:     public delegate void ClickHandler(object sender, EventArgs e);
   6:     public event ClickHandler Click;
   7: 
   8:     public static readonly DependencyProperty TextProperty =
   9:         DependencyProperty.Register("Text", typeof(string), typeof(MenuItem), null);
  10:     public static readonly DependencyProperty CommandProperty =
  11:         DependencyProperty.Register("Command", typeof(ICommand), typeof(MenuItem), null);
  12:     public static readonly DependencyProperty CommandParameterProperty =
  13:         DependencyProperty.Register("CommandParameter", typeof(object), typeof(MenuItem), null);
  14: 
  15:     public string Text { get; set; }
  16:     public ICommand Command { get; set; }
  17:     public object CommandParameter { get; set; }
  18: 
  19:     protected override void OnMouseEnter(MouseEventArgs e);
  20:     protected override void OnMouseLeave(MouseEventArgs e);
  21:     protected override void OnMouseMove(MouseEventArgs e);
  22:
  23:     protected override void OnGotFocus(RoutedEventArgs e);
  24:     protected override void OnLostFocus(RoutedEventArgs e);
  25:
  26:     protected override void OnKeyDown(KeyEventArgs e);
  27:
  28:     protected override void OnMouseLeftButtonUp(MouseButtonEventArgs e);
  29:     protected override void OnMouseRightButtonUp(MouseButtonEventArgs e);
  30: }

Here we have the public Click event, as well as the Text, Command, and CommandParameter properties.  The Click event will be raised whenever the item is selected by clicking (with the left or right button), or by selecting with the keyboard (with the Space or Enter key).

The mouse, keyboard, and focus-handling overrides are fairly trivial, and primarily deal with selection and changing the visual state – an item’s visual state will be Active when it has focus, and Normal otherwise.  The main trick here is to focus an item whenever a user highlights it using the mouse or keyboard: we need to re-focus an item not only when it receives a MouseEnter event, but also when it receives a MouseMove event, since keyboard input may have changed the focused item since the last mouse event was received.

5.  Source Code

Full source available from:

Note this is written against Silveright 4 Beta in Visual Studio 2010 Beta 2, so content and APIs are subject to change.  I’ll try to update as needed.

13

Multi-touch Gesture Recognition in Silverlight 3

November 5, 2009

Silverlight 3 provides all the info you need to receive and interpret multi-touch input on Windows 7, but it takes some work to translate these events into standard gestures – like “pinching” to zoom in or out (a list of other common gestures, and some good background info, can be found on the Engineering Windows 7 blog).

Tim Heuer gave a great introduction to the basics of Silverlight 3 multi-touch, covering hardware/software requirements and how to listen to the application-wide Touch.FrameReported event to get touch information.  Building on these basics, I’d like to give a short example of how you can add support for some common gestures – such as Translation, Rotation, and Scaling – to your touch-aware Silverlight 3 application on a per-element basis.

At a high level, the steps to accomplish this are:

  1. Receive touch events at an app level.
  2. Route touch events to the target element(s) being manipulated.
  3. Interpret touch events on a target element as gestures.
  4. Hook up gestures to manipulate the element.

To make things a bit simpler and more reusable, I’d like to create a static class to accomplish #1 and #2 – I’ll call it TouchProcessor.

For #3 and #4, I would also like to be able to reuse this logic – so I’ll create a GestureController class to receive events on behalf of an element, and define an ITouchElement interface that it can use to manipulate elements.

Simplified class diagram:

Gestures Class Diagram

Delving into more detail on each of these:

1.  TouchProcessor

Static class that manages registering for the global Touch.FrameReported event, and passing events on to the GestureControllers of specific touched elements.  It exposes a single public property:

public static class TouchProcessor
{
    public static bool IsTouchEnabled { get; set; }
}

 

 

2.  ITouchElement

An interface describing the available gestures that an element supports.  In this case, we would like to support the following gestures :

  1. Translate
    • This would result from a “panning” gesture – touching an element with a single finger and dragging it around.
  2. Rotate an element
    • This would be accomplished by placing two fingers on an element and rotating them circularly in either direction.
  3. Scale an element
    • This would allow resizing an element by placing two fingers on an element and “pinching” in or out.
  4. Bring to front
    • This would cause an element to be shown overtop all other elements, and would result from tapping an element with one or more fingers, or from starting any of the above gestures.

As you can imagine, these gestures are all readily implemented for any UIElement using the RenderTransform and ZIndex properties.

public interface ITouchElement
{
    void Translate(double x, double y);
    void Rotate(double angle);
    void Scale(double scale);
    void BringToFront();
}

 

 

3.  GestureController

This class is the “brains” behind gesture recognition.  There will be a GestureController instance for each ITouchElement: it will track all touch input a user directs to the element, interpret it, and invoke the appropriate ITouchElement method depending on the gesture.  Since a code listing is worth a thousand words, here’s a snapshot of all the members of this class:

public class GestureController
{
    private ITouchElement _element;
    private Dictionary<int, Point> _points;
    private int _primaryContactId = -1;
 
    public GestureController(ITouchElement element);
    public void TouchPointReported(TouchPoint touchPoint);
    private void InterpretSingleTouchGesture(Point oldPosition, Point newPosition);
    private void InterpretMultiTouchGesture(int Id, Point oldPosition, Point newPosition);
 
    private void TranslateElement(Point oldPosition, Point newPosition);
    private void ScaleElement(Point primaryPosition, Point oldPosition, Point newPosition);
    private void RotateElement(Point primaryPosition, Point oldPosition, Point newPosition);
    private double GetAngleDeltaBetweenPoints(Point referencePoint, Point startPoint, Point endPoint);
    private double GetDistanceBetweenPoints(Point point1, Point point2);
}

You can see it keeps track of a specific target element, a set of active touch points, and which touch point is the “primary” point (for single-touch gestures).

First, some terminology:

  • Touch device
    • A finger.
  • Touch Contact
    • The location of a touch device while it is touching an element on the screen.
  • “Primary” device
    • The first touch device (finger) that hits an element.  This can be reset by lifting all fingers off an element, and touching down again.
  • “Secondary” device
    • Any touch device (finger) that hit an element after the primary one was already down.  These points put the “multi” in “multi-touch”!

Now, let’s take a look at some of the methods in the GestureController class:

TouchPointReported

This is GestureController’s only public method, which is invoked by the TouchProcessor whenever a global touch event is directed at its attached element.  The snippet below shows the implementation of this method:

   1: public void TouchPointReported(TouchPoint touchPoint)
   2: {
   3:     switch (touchPoint.Action)
   4:     {
   5:         case TouchAction.Down:
   6:             // start tracking device position
   7:             _points.Add(touchPoint.TouchDevice.Id, new Point(touchPoint.Position.X, touchPoint.Position.Y));
   8: 
   9:             // if the new point is the only one down, it's the new primary point
  10:             if (_points.Count == 1)
  11:             {
  12:                 _primaryContactId = touchPoint.TouchDevice.Id;
  13:             }
  14: 
  15:             _element.BringToFront();
  16:             break;
  17:         case TouchAction.Move:
  18:             int Id = touchPoint.TouchDevice.Id;
  19:             Point newPoint = new Point(touchPoint.Position.X, touchPoint.Position.Y);
  20:             Point oldPoint = _points[Id];
  21: 
  22:             if (_points.Count == 1)
  23:             {
  24:                 InterpretSingleTouchGesture(oldPoint, newPoint);
  25:             }
  26:             else
  27:             {
  28:                 InterpretMultiTouchGesture(Id, oldPoint, newPoint);
  29:             }
  30: 
  31:             _points[Id] = newPoint;
  32:             break;
  33:         case TouchAction.Up:
  34:             // stop tracking device position
  35:             _points.Remove(touchPoint.TouchDevice.Id);
  36:             break;
  37:     };
  38: }

The operation here depends on which type of action was reported for the given touch point:

  • Down
    • A new touch device (finger) was just placed on the element, so start tracking that device. 
    • If there aren’t any other active devices (i.e., only one finger is on the element), make it the new “primary” contact.
  • Move
    • A touch device that’s already being tracked has moved, so determine what gesture this movement should be interpreted as.  It may be a single-touch gesture (from the “primary” contact) such as a Translate, or a multi-touch gesture such as a Rotate or Scale.
  • Up
    • A touch device that was being tracked has been lifted up, so stop tracking it.  The same finger may touch down again, but it will be treated as a different device.
InterpretSingleTouchGesture, InterpretMultiTouchGesture

These methods will call the appropriate helper methods to translate, scale, or rotate an element when a touch device moves.  A translation will be performed if the device was the primary contact, otherwise a scale or rotate will be performed.

   1: private void InterpretSingleTouchGesture(Point oldPosition, Point newPosition)
   2: {
   3:     // only one contact is down, so translate element
   4:     TranslateElement(oldPosition, newPosition);
   5: }
   6: 
   7: private void InterpretMultiTouchGesture(int Id, Point oldPosition, Point newPosition)
   8: {
   9:     if (Id == _primaryContactId)
  10:     {
  11:         // the primary contact moved, so translate element
  12:         TranslateElement(oldPosition, newPosition);
  13:     }
  14:     else
  15:     {
  16:         // a secondary contact moved, so scale and rotate if applicable
  17:         ScaleElement(_points[_primaryContactId], oldPosition, newPosition);
  18:         RotateElement(_points[_primaryContactId], oldPosition, newPosition);
  19:     }
  20: }
TranslateElement

This method calculates the distance between the old and new position of the primary contact on the X-Y plane, and tells the element to move accordingly:

   1: private void TranslateElement(Point oldPosition, Point newPosition)
   2: {
   3:     // translation: change in primary contact location
   4:     double xDelta = newPosition.X - oldPosition.X;
   5:     double yDelta = newPosition.Y - oldPosition.Y;
   6: 
   7:     if (yDelta != 0 || xDelta != 0)
   8:     {
   9:         _element.Translate(xDelta, yDelta);
  10:     }
  11: }
ScaleElement

This method will tell an element to grow or shrink depending on the change in distance between the primary contact’s current location and the location of the secondary contact that just moved at two points in time – before the move event, and after the move event.  As you can see, it uses some helper methods that we’ll get to shortly.

   1: private void ScaleElement(Point primaryPosition, Point oldPosition, Point newPosition)
   2: {
   3:     // scaling: calculate change in distance from primary point
   4:     double previousLength = GetDistanceBetweenPoints(primaryPosition, oldPosition);
   5:     double newLength = GetDistanceBetweenPoints(primaryPosition, newPosition);
   6:     double scale = (newLength - previousLength) / newLength;
   7:     if (scale != 0)
   8:     {
   9:         _element.Scale(scale);
  10:     }
  11: }
RotateElement

Similar to ScaleElement, this method tells an element to rotate depending on the change in angle between the primary contact and where the secondary contact moved.  It also uses a helper method to do some math.

   1: private void RotateElement(Point primaryPosition, Point oldPosition, Point newPosition)
   2: {
   3:     // rotation: calculate change in angle relative to primary point       
   4:     double angleDelta = GetAngleDeltaBetweenPoints(primaryPosition, oldPosition, newPosition);
   5:     if (angleDelta != 0)
   6:     {
   7:         _element.Rotate(angleDelta);
   8:     }
   9: }
GetDistanceBetweenPoints, GetAngleDeltaBetweenPoints

These two helper methods take care of some geometry calculations.

  • GetDistanceBetweenPoints simply returns the distance between two points using the Pythagorean theorem:

clip_image002

  • GetAngleDeltaBetweenPoints returns the change in angle, in degrees, of a moving secondary contact relative to a fixed primary contact.  It uses the two-argument inverse tangent function to find the angle in green shown below:

image

   1: private double GetDistanceBetweenPoints(Point point1, Point point2)
   2: {
   3:     return Math.Sqrt(Math.Pow(point1.X - point2.X, 2) + Math.Pow(point1.Y - point2.Y, 2));
   4: }
   5: 
   6: private double GetAngleDeltaBetweenPoints(Point referencePoint, Point startPoint, Point endPoint)
   7: {
   8:     // simple way to calculate angle's magnitude and direction
   9:     // slightly naive implementation since it assumes the reference point remains fixed, but it's still functional
  10:     Double angle = Math.Atan2(endPoint.Y - referencePoint.Y, endPoint.X - referencePoint.X) - Math.Atan2(startPoint.Y - referencePoint.Y, startPoint.X - referencePoint.X);
  11: 
  12:     // convert to degrees
  13:     return angle * 180 / Math.PI;
  14: }

 

 

 

4.  ManipulableElement

This is just a UserControl type that implements ITouchElement.  I added a rectangle to it, but you could use anything you like.  As you can see, it’s pretty simple – the ITouchElement method implementations just operate on some render transforms and the z-index of the control.

XAML Content:

   1: <UserControl.RenderTransform>
   2:     <TransformGroup>
   3:         <!-- note ordering is important here -->
   4:         <ScaleTransform x:Name="scale" />
   5:         <RotateTransform x:Name="rotate" />
   6:         <TranslateTransform x:Name="translate" />
   7:     </TransformGroup>
   8: </UserControl.RenderTransform>
   9: <Grid x:Name="LayoutRoot">
  10:     <Rectangle Name="contentRectangle" Height="200" Width="200" />
  11: </Grid>

Code:

   1: public partial class  ManipulableElement : UserControl, ITouchElement
   2: {
   3:     // initialization omitted
   4:
   5:     public void Translate(double deltaX, double deltaY)
   6:     {
   7:         this.translate.X += deltaX;
   8:         this.translate.Y += deltaY;
   9:     }
  10:
  11:     public void Rotate(double angle)
  12:     {
  13:         this.rotate.Angle += angle;
  14:     }
  15:
  16:     public void Scale(double scale)
  17:     {
  18:         this.scale.ScaleX += scale;
  19:         this.scale.ScaleY += scale;
  20:     }
  21:
  22:     public void BringToFront()
  23:     {
  24:          // small hack to bring a DraggableElement to the front
  25:         this.SetValue(Canvas.ZIndexProperty, zIndexCounter++);
  26:     }
  27: }

And that’s it!

image

Full source code available from:

5