Archive

Archive for the ‘WPF’ Category

Windows 8.1 OutOfMemoryException when rendering Bitmaps

February 8, 2014 Leave a comment

Recently, I have been faced to a strange issue in one of our application.
That application has been validated to work on bigger screen size, and we notice at that time that bitmaps graphics was badly render on such big screen mainly due to different pixel size.

To solve that bad rendering of such bitmap on such screen size, we had to use the BitmapScalingMode property of an Image control.

<StackPanel>
<Image RenderOptions.BitmapScalingMode=”HighQuality” Source=”/Resources/key.png” Width=”40″ Height=”40″ Stretch=”Fill” VerticalAlignment=”Center”/>
</StackPanel>

Has shown is sample code above, setting the property to HighQuality value was rendering the image nicely on bigger screen. So far so good then.

Next steps was to test the application on this same type of screen but under Windows 8, working perfectly well as well until the time it has been updated to Windows 8.1 and get our application crash with an OutOfMemoryException.

After days of testing we have identify that it was occurring when used image was about 8 bits and 256 colors. This has been reported as a bug by Microsoft on Windows 8.1 update.

Different possible Work around:

  1. With your prefer      graphic tool generate a files from your image with more depth
  2. If you have no      control of the image you have and want to still use it you card converted      from code by the following code snippets

    BitmapSource ResizeImage(BitmapSource image, int width, int height)
    {
    var      bmp = new FormatConvertedBitmap(image, PixelFormats.Bgr24, null, 0);
    return      new TransformedBitmap(bmp, new      ScaleTransform((double)width/bmp.PixelWidth,      (double)height/bmp.PixelHeight));
    }

Hope this will save you some times 🙂

Categories: Windows 8, WPF Tags:

PowerPoint Presentation : Overview of Microsoft Surface Application Design

May 13, 2013 1 comment

There are a lot of sources of information on the net where you can get different topics based on Surface application design. For one case where I need to present to a customer an overview on designing Surface Application, I though it would be much easier to get basic information, group in a presentation that I coud reuse at any time.

This is what I have done so far and customer personnal attending such presentation really appreciate . Clear enough, concentrate with the essential.

Note that this is not a support for developping an application but more a starting point on what you should know, mixed with my own experience from the field.

As it has been really usefull work for me, I decide to share it with you.

You can view the presentation here

Good luck !

Starting your PixelSense application efficiently : Loading Controls

May 12, 2013 Leave a comment

When you start to build application for the SUR40 and Microsoft PixelSense, all used to work really perfectly well on the simulator and your development PC. When you start to install and run your application directly on the device in Surface mode, you start to face a challenging but important issue:

Get your application started before the Surface Shell timeout

All of us as found the famous startup time-out handle by the Surface shell. It could happen that your application appears to never start in that mode or it start but not all the time.

Never block the main UI thread

The entry point of your Surface application is the main surface window, and this is also the main UI thread for your application. If you perform some long operation within the loading process of that main Window, you are blocking the main UI thread and you will definitely get the famous time-out message

How does it really works

A Microsoft Surface application must not block the UI thread for more than 5 seconds. If it does so, the system closes and tries to restart the unresponsive applications twice. If it must close the application a third time because of unresponsiveness, Surface returns to Launcher and does not try any further restarts.

Loading your application while using potentially long loading of controls

When you start to build an application, you usually start to fill up your SurfaceMainWindow.xaml grid with the different control that you think your need to load.
The following code snapshot shows that scenario:


Potentially Locked Main Surface windows loading process

<s:SurfaceWindow x:Class=”SurfaceAppStartup1.SurfaceWindow1″

xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation&#8221;

xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml&#8221;

xmlns:s=”http://schemas.microsoft.com/surface/2008&#8243;

Xmlns:controls=”clr-namespace:SurfaceAppStartup1.Controls”

Title=”SurfaceAppStartup1″

>

<Grid x:Name=”Content” Height=”Auto”>

<s:SurfaceScrollViewer x:Name=”_mainHScrollViewer” ScrollChanged=”_mainHScrollViewer_ScrollChanged”

Width=”{Binding Element Name=Main, Path=Width}”

PanningMode=”None”

HorizontalScrollBarVisibility=”Hidden”

VerticalScrollBarVisibility=”Hidden” DataContext=”{Binding}”>

<StackPanel x:Name=”AppPanel” Orientation=”Horizontal” Background=”Transparent” >

<!– Load background Window –>

<controls:BackgroundImage DataContext=”{Binding}”/>

<!– Load WordMapControl  –>

<controls:WorldMapControl/>

</StackPanel>

</s:SurfaceScrollViewer>

</Grid>

</s:SurfaceWindow>

From the sample above, you can see that we are instantiated directly in Xaml the loading of the WorldMapControl UserControl as <controls:WorldMapControl/>. In this case if the loading time that that control is longer than plan your main UI thread will be block and you will get the Surface Shell Time-Out.

Solution : Use of Content control and Binding

From the sample above, we simply replace the <controls:WorldMapControl/> by the code below :

<ContentControl DataContext=”{Binding Source={StaticResource Locator}, Path=WorldMapViewModelStatic}”

Content=”{Binding MapControl, UpdateSourceTrigger=PropertyChanged}” />

You can guess from code above that we are using binding and an MVVM pattern to reach our goal. By doing so, the main UI thread will not be block during the loading of the control. Instead it will prepare a Content control that will have its Content property bind to a MVVM property named MapControl. When the MapControl property change, the control will be injected in the ContentControl.

What to do then while my user will wait for control to load?

This is a really interesting question and there are different approach. First avoid using a simple loading screen without adding value for the user. Remember that as soon as a Surface application has started, the application is interactive and user expect to do something with it:

  1. Provide simple content information in relation with your application that users can already explore
  2. Indicate visually that remaining content are currently loading
  3. When the remaining controls have completed and the full application is ready,bring attention to users with visual component or effect that something more can be done

Important :

When all your application initialization content is done, your application must call the Microsoft.Surface.ApplicationServices.SignalApplicationLoadComplete method to dismiss the opening activity when the application is ready to appear.

You can download sample project which illustrate this from the msdn code sample.

Hoping this small post will be helpfull for you.

TIPS EXPERIENCE FROM THE FIELD ON DESIGNING A PIXELSENSE APPLICATION

May 3, 2013 4 comments

Since the release to market of the first Microsoft Surface device (Aka Surface V1) and the today Samsung SUR40 with Microsoft PixelSense technology, I have been working on building different types of application for such devices. By looking a bit behind (even if I am not used to do that in general) I have explore and validate different form of approach and rules on such technology , in order to bring to ending customers what they were really expecting to get.

Those rules and guide lines that I have experiment during all those past years, this is something that I will describe here. It has perfectly work in all my projects and could be apply or adjust in order to cover why not yours.


Rules 1 : Understanding customer objectives

One of the most important rule in my mind is to really understand main objectives of your customers, on having a dedicated PixelSense application. At first it might sounds an evidence but when you look deeper it is the key point entry. This understanding will indicate the type of application that you are going to build and set up. Depending on customers objectives the application interaction and design goal will be totally different and you will be able to setup a compromise between the “Woww” effect and application performance.


Rules 2 : Always remember basic questions before anything else

  • What is the purpose of integrating the device in a selected location?
  • Does the location is suitable to receive the device? ( lighting condition, traffic,..)
  • Who will be your application users?
  • What type of application will be more suitable based on Rule 1
  • How the environment will be ? ( colors, style, branding,..)


Rules 3 : Learn from those who have tried before

Do not jump too quickly into programming your design, but instead raise your head and takes some time to explore what is already existing. To help developers, the Surface team has written an important set of documents to puts you in the right track. The meeting point is the Microsoft PixelSense Design and Development. The Surface SDK 2.0 is the set of tools and sample that will help you building an application but keep in mind that any good programmer can use and understand an SDK but designing a great application is another story and this is the point where you will spend most of your time in a project.


Rules 4 : Forget what you have previously learned on designing traditional application

For some, it is difficult to jump to a new design approach and they always refer to what they have done before and wish to get the same result. You cannot imagine how many times I had to explain to customers or design agency (most of them coming from the web) that the design approach of an application in PixelSense is totally different. This is something where you need to explain again and again until the point you manage to convince them and established a full trust relationship, relying on the experience you provide.


“It takes time to build a great experience, so start with simple thing that you can validate easily”

Your successful application will start from the Microsoft® Surface® 2 Design and Interaction Guide that you absolutely need to cover and get a better idea of what all of this means.


Rules 5 : Do not hesitate to go back to school with your favorite tools

When you have validated the objectives of your customer application define in Rule1, your first favorite tool on your application design will be a pair of scissors and sheets of papers.

Cisors

During my first application design, I remember the time where I was starting right away my application interface by opening visual studio and start creating windows, controls… When building application for PixelSense this approach will definitely make you lose a lot of time and effort.

  1. Build a preliminary brain storming with your team on the main aspect and key point of the application
  2. List on a white board all crazy ideas you might have on your project ( you cannot imagine how a crazy idea can became an excellent one when it is added to others)
  3. Draw main user interface sketch flow without going in details

Now it’s time to go back to school. Simply remember the time when you was at the kinder garden.

  1. Take your scissors and paper and start to build what you think your interface can be
  2. Use your paper as it was the application itself
  3. Review the size of element if you think they are not correct
  4. Validate your preliminary design based on what you visually see

Measure each validated visual by keeping in mind that 1cm ~= 22 pixels

cut


“Do not imagine that the time spend around the device with your scissors is useless. It will save you a lot of tedious effort on sizing your visual element based on content flow”


Rule 6 : The Woww factor against application performance

Who have never start to think of his application by already imagine the type of different animation that could be done in order to bring to the application the famous Woww effect ?

Let me stop a while on this simple question:  Does “Wow effect” really means animation?

Absolutely not, and this is one of an important point that comes to me when designing an application.


“What my customer is really expecting when he was telling me “Make it wow”

In all of my first PixelSense project, I was permanently thinking of that 3 words and with the time, they disappears from my though for the simple reason that this famous wow factor will come naturally by bringing to live customers mains objectives.

The success of an application will be identify when your customer will start to use your application and will naturally pronounced one word: “Woww”.

Points that make me talk about this are essentially based on followings:

  • Too much animation will kill your application
  • Reserved animation for content transition and make it fast enough for rich experience. For example an opacity rendering should not go over 0.5 s.
  • Do not overload your application for unnecessary things that brings no added value for the general concept
  • Always gives priority to application performance


Rule 7 : Avoid using “Forever” storyboard on TagVisualizer

This is one of the rule I recently try to explain to the final customer on one of my last project. As every potential customers, they took major of their ideas from different videos they could catch on the web and explore different application type that can exist, until one of them could be apply to their own use.

So far nothing special, instead a normal behavior when you are new to this world of interaction. The main part of the application was to interact fully with different objects and bring respective content. Nothing special either in that sense except one single request that makes me start to doubt a bit: Each object visual was requested to be animated during the whole presentation of content.

We start to build the requested animation which was based on quite many different layer of successive animated object in different time line. To reach the customer demand we had to set up no more than 8 different storyboard to reach the goal. So far so good we get the animation in place and it behaves as customer expected to get it:

  • The object his place on the table
  • Different circle object appears around the object and rotate asynchronously
  • So particles moves around illustrating user activities during the whole time object was on the table

That was working quite nicely until the time we place more similar object on the table. At that time we could easily notify the weak response of the application during content manipulation. My first doubt start to come true and I could verify this by exploring the CPU usage which reach over 85%.

After some different testing, by simply modifying the storyboard animation to run only for few seconds, the CPU usage drop to 45%

Of course there is not only the fact of stopping a storyboard with make such measure possible but that was the goal of this single test only. Having effect on storyboard consume a lot of resource as well and can be avoided whenever possible.


Rule 8: Always bring alternative to object interaction

Interacting with object is the must have feature for a PixelSense application offering better user experience in a natural way. Objects gets build according to the impact and image to be deliver to the audience and  they are in close relation to direct information content attached.

I have been face to a personal bad experience in an earlier application during an event where we bring some nice object specially built to interact with the application I was showing. At the beginning it was cool until the time you let visitors playing around the table and suddenly discover that some objects are not anymore present.


“A missing object should not avoid accessing to it’s initial content”

I do not know what I get in mind when I was preparing the application, but I get a strange feeling that I had to be able to access object content without the object. And this is what I have done and saved my demo at that time.


Rule 9 : Keep track of your main application information

PixelSense application goes to different state during the run and provide different useful information. Keeping track of those information will help you during the whole application level to get those states available at any time.

Here is below one sample class model we used to store such information during the whole application life time:

class

The main interesting thing here is that this class is a part of a MVVM architecture and its properties can be accessible at any time.


Rule 10 : Do not hesitate to disable unwanted touch input if needed

After releasing an application and deploy it to the customer location, I always spend time to clearly look at users and the way they interact with the application in order to bring fine tuning during validation process on site. During that phase, I write down all that works as well as all that does not work.

I was of course expected different type of people around my application but also people wearing different type of clothes.


“Why am I talking about clothes here as we are talking about designing an application? “

Well, have you ever present the result of your application to your manager and ask him to try it? I am sure yes and at that time you have also notice that his long tie touching the screen disturbed his interaction with requested content. This is what could happen in real as well, as you are not expected to wear only tee-shirt to use your application correctly.

At this time you need to adjust you application in the environment it is used. For such you simply need from your main application windows to disable all touch input that your application do not need.

The following sample is filtering only finger input and store Finger position and orientation in an application state as seen in Rule 9:


Private
void SurfaceWindow_PreviewTouchDown (object sender, TouchEventArgs e)

{

If (e.Device.GetIsFingerRecognized ())

{

this.contactPosition = e.Device.GetPosition (this);

this.contactDegreeOrientation = e.Device.GetOrientation (this) +90;

AppState.Instance.FingerOrientation = this.contactDegreeOrientation;

AppState.Instance.FingerPosition = this.contactPosition;

}

}

With this simple code, the big tie will not be seen any more as a touch input.

Rule 11 : Use a single SurfaceWindow for better portability and flexibility

In PixelSense application design and WPF in general we should forget the multi window approach and going more to a single Window showing different type of content as illustrated below:

content

This architecture will offer flexibility and easier portability. With such approach we have a single main input entry for the whole application and content type gets displayed based on different criteria and  directly activated through data binding. The power of the MVVM pattern here takes all it’s importance.

Let’s keep In mind as well that the Surface SDK 2.0 is compatible with Windows 7 platform.  By having one single input window for the whole application the compatibility change of the application for Windows 7 compatibility will be resume only by providing a Surface Window for PixelSense app and a standard WPF Windows for Window7.

The only change to switch between those environments will be done from the App.xaml.cs file by overriding the App_Startup as shown below:
Void App_Startup (object sender, StartupEventArgs e)

{

if (InteractiveSurface.PrimarySurfaceDevice.IsTagRecognitionSupported == true)

{

//define Surface environment

StartupUri = new Uri (@”/PropertySense;component/SurfaceMainWindow.xaml”, UriKind.Relative);

}

Else

{

//define Windows 7 environment

StartupUri = new Uri (@”/PropertySense;component/MainWindow.xaml”, UriKind.Relative);

}

}

Rule 12 : Drag & Drop operation are far from being natural if not shown

During whole application design where drag and drop was in place, I have notice how complicated it is to make such operation intuitive and evident for users when they are in front of your application for the first time.

By this I really mean that you let users manipulate your application and simply see them, without showing or saying anything. We make this simple test where we provide a simple view of photos equivalent as the one below:

propertydetails2
How do you think user does in front of such photos collection?

First they see an horizontal slider so they understand easily that sliding from left to right can give access to more content. The next thing they most all do is touching a picture and wait. All users were doing exactly the same thing until I decide to show them that they can drag.

From that simple test and great feedback results, you need to start to reconsider that interaction. It does not mean that the drag/ drop interaction is bad in this condition but more how to have it more intuitive to users.  I start to explore different approach as:

  • Add to the interaction the Tap gesture on all element in the list
  • Display a Notification message to the user at a certain time
  • Display text information inside the visual element being manipulated
  • Simulate the drag operation when no activities on the list

From those possible solution, I came from the fact that simulating the interaction with a small storyboard sequence was the most effective way to bring users to repeat that gesture

This scenario keeps me remember all time that people never do what you are expecting them to do in some situation.


Rule 13 : Users are the main source of inspiration

When you have completed the deployment of your application, after a certain time of use, always collect feedback of users manipulating your application. If you have the possibility to go at the location, do not hesitate and simply open your eyes. You will surely see something that you will have never think of or never seen.


“By looking at others using your app, you are already thinking of your next application design”

Rule 14 : Users are the main source of inspiration

When you build a Surface application for Microsoft PixelSense device and have in mind the multi user approach, cleaning teh screen is not necessary the first thing you are thinking of. This comes once again when you look at users the way they use the app. In all application I have made I could notice that users allways open content, but most of the time never close it after they have seen it. This is the reason why after some time of user your main application screen is full of content, the the next user coming to use your app will be confuse on where to start.

Base on that fact we need to try to follow those simple aspect of your content :

    1. Always provide a close button to be able to close manually your visual content
    2. Provide an inactivity time out on your content in order that after a certain time of no use, content will be closed automatically

Hoping that I could help you a bit in your reflexion based on my succesfull deployement so far where those rules gets extracted. Of course they engaged only my own personal opinion based on proven scenarios.

Categories: PixelSense, SURFACE, WPF Tags: , ,

Sample Nested ScrollViewer interaction design

November 18, 2012 4 comments

In one of my project on a Samsung SUR40 with Microsoft PixelSense, I was facing to an interesting interaction design based on different information groups needed to be logically displayed and accessible in a natural way.

The final idea is illustrated by the picture below:

The visual tree

The most external brown rectangle is a panorama panel which should be able to be scroll in both horizontal directions in order to have freely access to child group control. When the user will reach is interesting group of information represented by the blue rectangle, inside information, in my scenario was to be able to scroll vertically the information. And here comes the interaction logic from where you have nested scrollable control.

In order to reach my goal the layout has been define by the following tree, but you could adjust in according to your need.

Control architecture

The resulting XAML of the main panel :

 

The resulting XAML of the group control :

 

Interaction & design key point:

From the interaction that we have setup and as mentioned earlier, we need to get it smooth, logic and natural. If we look closer to the inside group control, they are nothing much that SurfaceList box control with have been define to handle vertical scroll only in that sample. So in each group control we will be able to naturally scroll for vertical data.

But then how can we scroll the panel horizontally?

The first answer to this question: Easy we just need to scroll the whole panel with the help of the most outside scroll viewer. That is correct. But asking the user to scroll the whole panel by placing is finger at the bottom narrow space of the control, do you thing user will all think of that? Absolutely not.

The best approach and most natural way would be that users initiating a horizontal scroll from any group control should be able to start the main panel to scroll, as illustrate bellow:

Apparently should be possible without any trouble but in fact the scroll does not happen.

As we have seen in our layout, each control group embedded a scrollviewer set for vertical scrolling. Horizontal scroll is disabling, but even if it would be enable it will scroll the group control content only. So why does this not work?

The reason came from the fact that the touch is capture by the scrollwiewer and the handle is set to true, so the event in not propagated up to the visual tree.

Nested scrollViewer behavior comes to the rescue

In order to allows that interaction, we have define a nested scrollviewer behavior class which pass to the parent scrollviewer control interaction which is not dedicated to its child.

By adding this behavior to each group control scrollviewer, the gesture will be passing to the parent scrollviewer which is the main panel and you will have a natural scrolling effect.

You can download the project from msdn sample here.
The sample include MVVM pattern

How to attached an MVVM EventToCommand to an Attached event

August 23, 2012 6 comments

In my daily lif e of developer, I am using every time the MVVM Light tool kit which offer me flexibility and set up of the model in an easy and efficient way. On a recent Microsoft PIxelSense project I get the need to bind my MVVM to an attached event. So at first I thought it should be similar as a normal event so I start to setup my triggers as follow:

<s:SurfaceSlider Name=”timelineSlider” Grid.Row=”1″ >
<i:Interaction.Triggers>
  
<helpers:RoutedEventTrigger RoutedEvent=”Thumb.DragStarted” >
    
<cmd:EventToCommand Command=”{Binding Mode=OneWay, Path=SeekStartedCmd}” PassEventArgsToCommand=”True” />
</helpers:RoutedEventTriggers>

The code above, is based on a player way I need to use a slider for a Seek method. So to track the change on the slider, one of the events I need to catch was the one from the Thumb.

At first look it sounds promising, no error, and code run perfectly but the event get not catch.

 We can’t invoke command on attached events. For example we can’t invoke command on Thumb.DragStarted

 How to handle attached event:

After some research on the web and shaking different type of information I came across the following class which will help to solve my issue. We create a new trigger which fires on attached event by inheriting EventTriggerBase<T> class.

namespace Solatys.Helpers
{

 publicclassRoutedEventTrigger : EventTriggerBase<DependencyObject>
    {
        RoutedEvent _routedEvent;
        publicRoutedEvent RoutedEvent
        {
            get
            {
                return _routedEvent;
            }
            set { _routedEvent = value; }
        }

        public RoutedEventTrigger() { }
        protected override void OnAttached()
        {
            Behavior behavior = base.AssociatedObject asBehavior;
            FrameworkElement associatedElement = base.AssociatedObject asFrameworkElement; if (behavior != null)
            {
                associatedElement = ((IAttachedObject)behavior).AssociatedObject asFrameworkElement;
            } if (associatedElement == null)
            {
                thrownewArgumentException(“Routed Event trigger can only be associated to framework elements”);
            }
            if (RoutedEvent != null)
            {
                associatedElement.AddHandler(RoutedEvent, newRoutedEventHandler(this.OnRoutedEvent));
            }
        }

        void OnRoutedEvent(object sender, RoutedEventArgs args)
        {
            base.OnEvent(args);
        }

        protected override string GetEventName()
        {
            return RoutedEvent.Name;
        }
    }
}

 How to use it :

<s:SurfaceSlider Name=”timelineSlider” Grid.Row=”1″>
 
<i:Interaction.Triggers>
   <helpers:RoutedEventTrigger RoutedEvent=”Thumb.DragStarted” >
    <cmd:EventToCommand Command=”{Binding Mode=OneWay, Path=SeekStartedCmd}”
      PassEventArgsToCommand
=”True” />
  </i:Interaction.Triggers>
</s:SurfaceSlider>

 Happy coding

Categories: WPF Tags:

Date and Time in WPF

November 17, 2011 Leave a comment

Recently, i was fighting with a date/Time interpretation which makes me a kind of crazy.The scenario was collecting data from a return dataset based on SQL serve table. When looking in that dataset, the date/Time format of dat fields was formatted correctly based on the culture I get set. But as long as the data was rendering by my WPF gridview, Date/Time value gets disply in different manner.

After browsing around I met Mike Danes on a forums which gets similar trouble.

When you convert a date time to string in code the current (correct) culture is used. However WPF doesn’t use the current culture, it takes the culture name from the Language property of the FrameworkElement and then gets the culture by using GetCultureInfoByIetfLanguageTag. This method returns a standard CultureInfo object, not the user customized version. This has been reported as a bug by Mike already.

To overcome this, it was needed to apply a converter to my date/Time column field.

I would never seen that problem if my users did not start to change cutsom settings of standard culture.

Hope it could save you time as well.
Thanks to Mike as well helping me isolate the issue

Categories: Uncategorized, WPF Tags: