CodeProject

Controlling a Myo Armband with C#

Controlling a Myo Armband with C#

Background

Thalmic Labs has started shipping their Myo armband that allows the wearer’s arm movements and gestures to control different pieces of integrated technology. How cool is that? My friend and I decided we wanted to give one a whirl and see what we could come up with. We’re both C# advocates, so we were a bit taken back when we saw the only C# support in the SDK was made for Unity. We decided to take things into our own hands and open source a Myo C# library. We’re excited to introduce the first version of MyoSharp!

The underlying Myo components are written in C++, and there’s only several functions that are exposed from the library that we can access. In order to do this, we need to leverage platform invocation (PInvokes) from C# to tap into this functionality. Once you have the PInvokes set up you can begin to play around!

The Workflow

Getting setup with the Myo is pretty straightforward, but it wasn’t obvious to us right away. We didn’t have anyone to walk us through how the different components were supposed to work together (just some good ol’ fashioned code) so we had to tinker around. Once we had everything mapped out, it was quite simple though.

  1. The first step is opening a communication channel with the Bluetooth module. You don’t need to worry about the implementation here since it’s all done in C++ by the Thalmic devs. Calling the correct methods using PInvokes from C# allows us to tap into a stream of “events” that come through the Bluetooth module.
  2. Now that we can intercept events, we need to be able to identify a Myo. After all, working with Myos is our main goal here! There’s a “pair” event that we can listen to from the Bluetooth module that notifies us of when a Myo has paired and provides us a handle to the device. This handle gets used for identifying events for a particular Myo or sending a particular Myo messages.
  3. There’s a connect event that will fire when a Myo connects after it’s been paired with the Bluetooth module. A Myo can be paired but disconnected.
  4. Now that we can uniquely identify a Myo, the only things we need to do are intercept events for a particular Myo and make sense of the data coming from the devices! Orientation change? Acceleration change? There’s a host of information that the device sends back, so we need to interpret it.
  5. When a Myo disconnects, there’s an event that’s sent back for that as well.

Getting Started with MyoSharp

I’m going to start this off with some simple code that should illustrate just how easy it is to get started with MyoSharp. I’ll describe what’s going on in the code immediately after.


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you the basics for setting up and working with
    /// a Myo using MyoSharp. Primary communication with the device happens
    /// over Bluetooth, but this C# wrapper hooks into the unmanaged Myo SDK to
    /// listen on their "hub". The unmanaged hub feeds us information about
    /// events, so a channel within MyoSharp is responsible for publishing
    /// these events for other C# code to consume. A device listener uses a
    /// channel to listen for pairing events. When a Myo pairs up, a device
    /// listener publishes events for others to listen to. Once we have access
    /// to a channel and a Myo handle (from something like a Pair event), we
    /// can create our own Myo object. With a Myo object, we can do things like
    /// cause it to vibrate or monitor for poses changes.
    /// </summary>
    internal class BasicSetupExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.Vibrate(VibrationType.Short);
                    e.Myo.PoseChanged += Myo_PoseChanged;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.PoseChanged -= Myo_PoseChanged;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_PoseChanged(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo detected {1} pose!", e.Myo.Arm, e.Myo.Pose);
        }
        #endregion
    }
}

In this example, we create a hub instance. A hub will manage a collection of Myos that come online and go offline and notify listeners that are interested. Behind the scenes, the hub creates a channel instance and passes this into a device listener instance. The channel and device listener combination allows for being notified when devices come online and is the core of the hub implementation. You can manage Myos on your own by completely bypassing the Hub class and creating your own channel and device listener if you’d like. It’s totally up to you.

In the code above, we’ve hooked up several event handlers. There’s an event handler to listen for when Myo devices connect, and a similar one for when the devices disconnect. We’ve also hooked up to an instance of a Myo device for when it changes poses. This will simply give us a console message every time the hardware determines that the user is making a different pose.

When devices go offline, the hub actually keeps the instance of the Myo object around. This means that if you have device A and you hook up to it’s PoseChanged event, if it goes offline and comes back online several times, your event will still be hooked up to the object that represents device A. This makes managing Myos much easier compared to trying to re-hook event handlers every time a device goes on and offline. Of course, you’re free to make your own implementation using our building blocks, so there’s no reason to feel forced into this paradigm.

It’s worth mentioning that the UserInputLoop() method is only used to keep the program alive. The sample code on GitHub actually lets you use some debug commands to read some Myo statuses if you’re interested. Otherwise, you could just imagine this line is replaced by Console.ReadLine() to block waiting for the user to press enter.

Pose Sequences

Without even diving into the accelerometer, orientation, and gyroscope readings, we were looking for some quick wins to building up on the basic API that we created. One little improvement we wanted to make was the concept of pose sequences. The Myo will send events when a pose changes, but if you were interested in grouping some of these together there’s no way to do this out of the box. With a pose sequence, you can declare a series of poses and get an event triggered when the user has finished the sequence.

Here’s an example:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user has performed a series of poses. A 
    /// <see cref="PoseSequence"/> can monitor a Myo for a series of poses and
    /// notify you when that sequence has completed.
    /// </summary>
    internal class PoseSequenceExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // for every Myo that connects, listen for special sequences
                    var sequence = PoseSequence.Create(
                        e.Myo, 
                        Pose.WaveOut, 
                        Pose.WaveIn);
                    sequence.PoseSequenceCompleted += Sequence_PoseSequenceCompleted;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Sequence_PoseSequenceCompleted(object sender, PoseSequenceEventArgs e)
        {
            Console.WriteLine("{0} arm Myo has performed a pose sequence!", e.Myo.Arm);
            e.Myo.Vibrate(VibrationType.Medium);
        }
        #endregion
    }
}

The same basic setup occurs as the first example. We create a hub that listens for Myos, and when one connects, we hook a new PoseSequence instance to it. If you recall how the hub class works from the first example, this will hook up a new pose sequence each time the Myo connects (which, in this case, isn’t actually ideal). Just for demonstration purposes, we were opting for this shortcut though.

When creating a pose sequence, we only need to provide the Myo and the poses that create the sequence. In this example, a user will need to wave their hand out and then back in for the pose sequence to complete. There’s an event provided that will fire when the sequence has completed. If the user waves out and in several times, the event will fire for each time the sequence is completed. You’ll also notice in our event handler we actually send a vibrate command to the Myo! Most of the Myo interactions are reading values from Myo events, but in this case this is one of the commands we can actually send to it.

Held Poses

The event stream from the Myo device only sends events for poses when the device detects a change. When we were trying to make a test application with our initial API, we were getting frustrated with the fact that there was no way to trigger some action as long as a pose was being held. Some actions like zooming, panning, or adjusting levels for something are best suited to be linked to a pose being held by the user. Otherwise, if you wanted to make an application that would zoom in when the user makes a fist, the user would have to make a fist, relax, make a fist, relax, etc… until they zoomed in or out far enough. This obviously makes for poor usability, so we set out to make this an easy part of our API.

The code below has a similar setup to the previous examples, but introduces the HeldPose class:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user is still holding a pose and not just that they've 
    /// transitioned from one pose to another. The <see cref="HeldPose"/> class
    /// monitors a Myo and notifies you as long as a particular pose is held.
    /// </summary>
    internal class HeldPoseExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // setup for the pose we want to watch for
                    var pose = HeldPose.Create(e.Myo, Pose.Fist, Pose.FingersSpread);

                    // set the interval for the event to be fired as long as 
                    // the pose is held by the user
                    pose.Interval = TimeSpan.FromSeconds(0.5);

                    pose.Start();
                    pose.Triggered += Pose_Triggered;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Pose_Triggered(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo is holding pose {1}!", e.Myo.Arm, e.Pose);
        }
        #endregion
    }
}

When we create a HeldPose instance, we can pass in one or more poses that we want to monitor for being held. In the above example, we’re watching for when the user makes a fist or when they have their fingers spread. We can hook up to the Triggered event on the held pose instance, and the event arguments that we get in our event handler will tell us which pose the event is actually being triggered for.

If you take my zoom example that I started describing earlier, we could have a single event handler responsible for both zooming in and zooming out based on a pose being held. If we picked two poses, say fist and fingers spread, to mean zoom in and zoom out respectively, then we could check the pose on the event arguments in the event handler and adjust the zoom accordingly. Of course, you could always make two HeldPose instances (one for each pose) and hook up to the events separately if you’d like. This would end up creating two timer threads behind the scenes–one for each HeldPose instance.

The HeldPose class also has an interval setting. This allows the programmer to adjust the frequency that they want the Triggered event to fire, provided that a pose is being held by the user. For example, if the interval is set to be two seconds, as long as the pose is being held the Triggered event will fire every two seconds.

Roll, Pitch, and Yaw

The data that comes off the Myo can become overwhelming unless you’re well versed in vector math and trigonometry. Something that we’d like to build up and improve upon is the usability of data that comes off the Myo. We don’t want each programmer to have to write similar code to get the values from the Myo into a usable form for their application. Instead, if we can build that into MyoSharp, then everyone will benefit.

Roll, pitch, and yaw are values that we decided to bake into the API directly. So… what exactly are these things? Here’s a diagram to help illustrate:

Roll, Pitch, and Yaw - MyoSharp

Roll, pitch, and yaw describe rotation around one of three axes in 3D space.

The following code example shows hooking up to an event handler to get the roll, pitch, and yaw data:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you how to hook onto the orientation events on
    /// the Myo and pull roll, pitch and yaw values from it.
    /// </summary>
    internal class OrientationExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.OrientationDataAcquired += Myo_OrientationDataAcquired;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.OrientationDataAcquired -= Myo_OrientationDataAcquired;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_OrientationDataAcquired(object sender, OrientationDataEventArgs e)
        {
            Console.Clear();
            Console.WriteLine(@"Roll: {0}", e.Roll);
            Console.WriteLine(@"Pitch: {0}", e.Pitch);
            Console.WriteLine(@"Yaw: {0}", e.Yaw);
        }
        #endregion
    }
}

Of course, if we know of more common use cases that people will be using the orientation data for, then we’d love to bake this kind of stuff right into MyoSharp to make it easier for everyone.

Closing Comments

That’s just a quick look at how you can leverage MyoSharp to make your own C# application to work with a Myo! As I said, MyoSharp is open source so we’d love to see contributions or ideas for suggestions. We’re aiming to provide as much base functionality as we can into our framework but designing it in a way that developers can extend upon each of the individual building blocks.


IronPython: A Quick WinForms Introduction

IronPython: A Quick WinForms Introduction

Background

A few months ago I wrote up an article on using PyTools, Visual Studio, and Python all together. I received some much appreciated positive feedback for it, but really for me it was about exploring. I had dabbled with Python a few years back and hadn’t really touched it much since. I spend the bulk of my programming time in Visual Studio, so it was a great opportunity to try and bridge that gap.

I had an individual contact me via the Dev Leader Facebook group that had come across my original article. However, he wanted a little bit more out of it. Since I had my initial exploring out of the way, I figured it was probably worth trying to come up with a semi-useful example. I could get two birds with one stone here–Help out at least one person, and get another blog post written up!

The request was really around taking the output from a Python script and being able to display it in a WinForm application. I took it one step further and created an application that either lets you choose a Python script from your file system or let you type in a basic script directly on the form. There isn’t any fancy editor tools on the form, but someone could easily take this application and extend it into a little Python editor if they wanted to.

Leveraging IronPython

In my original PyTools article, I mention how to get IronPython installed into your Visual Studio project. In Visual Studio 2012 (and likely a very similar approach for other versions of Visual Studio), the following steps should get you setup with IronPython in your project:

  • Open an existing project or start a new one.
  • Make sure your project is set to be at least .NET 4.0
    • Right click on the project within your solution explorer and select “Properties”
    • Switch to the “Application” tab.
    • Under “Target framework”, select  “.NET Framework 4.0”.
  • Right click on the project within your solution explorer and select “Manage NuGet Packages…”.
  • In the “Search Online” text field on the top right, search for “IronPython”.
  • Select “IronPython” from within the search results and press the “Install” button.
  • Follow the instructions, and you should be good to go!

Now that we have IronPython in a project, we’ll need to actually look at some code that gets us up and running with executing Python code from within C#. If you followed my original post, you’ll know that it’s pretty simple:


var py = Python.CreateEngine();
py.Execute("your python code here");

And there you have it. If it seems easy, that’s because it is. But what about the part about getting the output from Python? What if I wanted to print something to the console in Python and see what it spits out? After all, that’s the goal I was setting out to accomplish with this article. If you try the following code, you’ll notice you see a whole lot of nothing:


var py = Python.CreateEngine();
py.Execute("print('I wish I could see this in the console...')");

What gives? How are we supposed to see the output from IronPython? Well, it all has to do with setting the output Stream of the IronPython engine. It has a nice little method for letting you specify what stream to output to:


var py = Python.CreateEngine();
py.Runtime.IO.SetOutput(yourStreamInstanceHere);

In this example, I wanted to output the stream directly into my own TextBox. To accomplish this, I wrote up my own little stream wrapper that takes in a TextBox and appends the stream contents directly to the Text property of the TextBox. Here’s what my stream implementation looks like:


private class ScriptOutputStream : Stream
{
  #region Fields
  private readonly TextBox _control;
  #endregion

  #region Constructors
  public ScriptOutputStream(TextBox control)
  {
    _control = control;
  }
  #endregion

  #region Properties
  public override bool CanRead
  {
    get { return false; }
  }

  public override bool CanSeek
  {
    get { return false; }
  }

  public override bool CanWrite
  {
    get { return true; }
  }

  public override long Length
  {
    get { throw new NotImplementedException(); }
  }

  public override long Position
  {
    get { throw new NotImplementedException(); }
    set { throw new NotImplementedException(); }
  }
  #endregion

  #region Exposed Members
  public override void Flush()
  {
  }

  public override int Read(byte[] buffer, int offset, int count)
  {
    throw new NotImplementedException();
  }

  public override long Seek(long offset, SeekOrigin origin)
  {
    throw new NotImplementedException();
  }

  public override void SetLength(long value)
  {
    throw new NotImplementedException();
  }

  public override void Write(byte[] buffer, int offset, int count)
  {
    _control.Text += Encoding.GetEncoding(1252).GetString(buffer, offset, count);
  }
  #endregion
}

Now while this isn’t pretty, it serves one purpose: Use the stream API to allow binary data to be appended to a TextBox. The magic is happening inside of the Write() method where I take the binary data that IronPython will be providing to us, convert it to a string via code page 1252 encoding, and then append that directly to the control’s Text property. In order to use this, we just need to set it up on our IronPython engine:


var py = Python.CreateEngine();
py.Runtime.IO.SetOutput(new ScriptOutputStream(txtYourTextBoxInstance), Encoding.GetEncoding(1252));

Now, any time you output to the console in IronPython you’ll get your console output directly in your TextBox! The ScriptOutputStream implementation and calling SetOutput() are really the key points in getting output from IronPython.

The Application at a Glance

I wanted to take this example a little bit further than the initial request. I didn’t just want to show that I could take the IronPython output and put it in a form control, I wanted to demonstrate being able to pick the Python code to run too!

Firstly, you’re able to browse for Python scripts using the default radio button. Just type in the path to your script or use the browse button:

IronPython - Run script from file

Enter a path or browse for your script. Press “Run Script” to see the output of your script in the bottom TextBox.

Next, press “Run Script”, and you’re off! This simply uses a StreamReader to get the contents of the file and then once in the contents are stored in a string, they are passed into the IronPython engine’s Execute() method. As you might have guessed, my “helloworld.py” script just contains a single line that prints out “Hello, World!”. Nothing too fancy in there!

Let’s try running a script that we type into the input TextBox instead. There’s some basic error handling so if your script doesn’t execute, I’ll print out the exception and the stack trace to go along with it. In this case, I tried executing a Python script that was just “asd”. Clearly, this is invalid and shouldn’t run:

python_error_asd

Python interpreted the input we provided but, as expected, could not find a definition for “asd”.

That should be along the lines of what we expected–The script isn’t valid, and IronPython tells us why. What other errors can we see? Well, the IronPython engine will also let you know if you have bad syntax:

python_error_bad_syntax

Python interpreted the script, but found a syntax error in our silly input.

Finally, if we want to see some working Python we can do some console printing. Let’s try a little HelloWorld-esque script:

python_pass_hello_world

Python interpreted our simple Hello World script.

Summary

This sample was pretty short but that just demonstrates how easy it is! Passing in a script from C# into the IronPython is straight forward, but getting the output from IronPython is a bit trickier. If you’re not familiar with the different parts of the IronPython engine, it can be difficult to find the things you need to get this working. With a simple custom stream implementation we’re able to get the output from IronPython easily. All we had to do was create our own stream implementation and pass it into the SetOutput() method that’s available via the IronPython engine class. Now we can easily hook the output of our Python scripts!

As always, all of the source for you to try this out is available online:

Some next steps might include:

  • Creating your own Python IDE. Figure out some nice text-editing features and you can run Python scripts right from your application.
  • Creating a test script dashboard. Do you write test scripts for other applications in Python? Why not have a dashboard that can report on the results of these scripts?
  • Add in some game scripting! Sure, you could have done this with IronPython alone, but maybe now you can skip the WinForms part of this and just make your own stream wrapper for getting script output. Cook up some simple scripts in a scripting engine and voila! You can easily pass information into Python and get the results back out.

Let me know in the comments if you come up with some other cool ideas for how you can leverage this!


Yield! Reconsidering APIs with Collections

Yield! Reconsidering APIs with Collections (Image by http://www.sxc.hu/)

Yield: A Little Background

The yield keyword in C# is pretty cool. Being used within an iterator, yield lets a function return an item as well as control of execution to the caller and upon next iteration resume where it left off. Neat, right? MSDN documentation lists these limitations surrounding the use of the yield keyword:

  • Unsafe blocks are not allowed.
  • Parameters to the method, operator, or accessor cannot be ref or out.
  • A yield return statement cannot be located anywhere inside a try-catch block. It can be located in a try block if the try block is followed by a finally block.
  • A yield break statement may be located in a try block or a catch block but not a finally block.

So what does this have to do with API specifications?

A whole lot really, especially if you’re dealing with collections. I personally haven’t been a big user of the yield keyword, but I’ve never really been forced to use it. After playing around with it for a bit, I saw a lot of potential. I’ve written before about what I think makes a good API. In my article, I was making a point to discuss two perspectives:

  • Who needs to implement your interface. You want it to be easy for them to implement.
  • Who needs to call your interface. You want it to be easy for them to use.

In my opinion, the IEnumerable<T> interface was a tricky thing to work with as a return value. You can essentially only iterate an IEnumerable, and at the time of calling a function, maybe that’s not what you want to do. The flip side is that for the person implementing the interface, IEnumerable<T> is a really easy interface to satisfy. However, the yield keyword has opened up some new doors.

In this article, I’d like to go over a couple of different approaches for an API and then explain why the yield keyword might be something you consider next time around. Disclaimer: I’m not claiming anything I’m about to present is the only way or the best way–I’m just sharing some of my own findings and perspective.

Interface For Returning Collections

The first type of API I’d like to look at is for returning collections. Based on my own API guidelines, I’d ideally choose an interface or class to return that provides a lot of information to the caller that is also easy to create for the implementer of my interface. The List<T> class is a great choice:

  • It’s easy to construct
  • It’s built-in to the .NET framework
  • It provides many handy functions (All of the IList<T> functionality as well as things like AddRange(), or functions that support delegates)

My next choice might be to have a return type of IList<T>, which would provide a little less ease of use to the caller, but make it even easier for the implementer of the interface. They could return arrays of type T, since an array implements the IList<T> interface, or their own custom list implementation that doesn’t inherit from the List<T> class. The differences between using IList<T> and List<T> are arguable pretty small.

A third alternative, which I would have avoided in the past, is to return an IEnumerable<T>. My opinion used to be that this made the life of the interface implementer a bit easier compared to returning an IList<T>, but complicated the life of the caller for a couple of reasons:

  • The caller would have to use the results of the function in a foreach loop.
  • The caller would have to add the items to their own collection to be able to do much more with the items.

My naive implementations of being forced to return an IEnumerable<T> were… well… crap. I would have constructed a collection within the function, fill it up, and then return it as an IEnumerable<T>. Then as the caller of my function, I’d have to re-enumerate the results (or add it to another collection):

public static IEnumerable<T> GetItems()
{
  var collection = new List<T>();
  // add all the items to a collection
  return collection;
}

private static void Main()
{
  var myCollection = new List<T>();
  myCollection.AddRange(GetItems());
  // use myCollection...

  // or.....
  foreach (var item in GetItems())
  {
    // use the items
  }
}

Seems like overkill to me with that implementation. However, we’ll examine how using yield can truly transform this into something… better. So to reiterate, a few potential implementations for an API involving collections might be:

  • Return a List<T> class
  • Return an IList<T> (or even an ICollection<T>) interface
  • Return an IEnumerable<T> interface

Constantly Creating Collections

My design decisions, in the past, were really driven by two guidelines:

  • Make it easier for the person implementing/extending the API
  • Make it easy for the person consuming the API

As I quickly illustrated in the first section, this meant that I would have a method where I would create a collection, fill it with items, and then return it. I could generally pick any concrete collection class and return it since I would usually pick a simple collection as the return type. Easy.

One thing that might be noticeable with this approach is that it looks pretty inefficient to keep creating new collections, fill them, and then return them. I’ll illustrate with a simple example. We’ll create a class that has a method on it called GetItems(). As per my reasoning presented earlier, we’ll have this method return a List<T> instance, and to make this example easier to work with, we’ll pass in an IEnumerable<T> instance. For what it’s worth, the input to this function is really just for demonstration purposes here–We’re really focusing on how we’re creating our return value.

public class CreateNewListApi<T>
{
  public List<T> GetItems(IEnumerable<T> input)
  {
    var newCollection = new List<T>();

    foreach (var item in input)
    {
      newCollection.Add(item);
    }

    return newCollection;
  }
}

And now that we have our simple class we can mock up a little test for performance… Just how inefficient is creating new lists every time?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Creating New Collections");
    var api = new CreateNewListApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 1.73 seconds. The memory printout I get when running is 1615908864 bytes. So is that slow? Is that a lot of memory usage? I think it’s pretty hard to say conclusively without being able to compare it against anything. So let’s keep this number in mind as we continue to investigate the alternatives.

Side Note: At this point, some readers may be saying “Well, if the input to our function was also a list (or if whatever our function has to work with was otherwise equivalent to our return value) then we wouldn’t have to go populate a new collection every time… We can just return the underlying collection”! And I would say you are absolutely correct. If your function has access to an instance of the same type as the return type, then you could always just return that instance. But what implications does this have? You’re now giving people access to your underlying internals, and they can go modify them as they please. So, if you need to control access to items being added or removed, then it might not make sense for you to expose your internal collections like this.

Yield to Incoming API Alternatives

We’ve seen how my past implementations may have looked, so how might we tweak this? If we tweak our API a bit, we can make our method return an IEnumerable<T> instead. Let’s see what that might look like:

public class YieldingApi<T>
{
  public IEnumerable<T> GetItems(IEnumerable<T> input)
  {
    foreach (var item in input)
    {
      yield return item;
    }
  }
}

So in this API implementation, all we’ll be doing is iterating over some type of collection and then yielding each result. If we run it through the same type of test as out previous API implementation, what kind of results do we end up with?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Yielding");
    var api = new YieldingApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 2.80 seconds. The memory printout I get when running is 449409024 bytes. How does this relate back to our first implementation? Well, it’s certainly slower. It takes about 1.62x as long to enumerate using the yield implementation as it did with the first API we created. However, yield also uses less than 1/3 (about 27.8%, actually) of the memory footprint when compared to the first implementation. Pretty cool results!

Site Note: So yield was a bit slower according to our results, but what happens if print the elapsed time before we run that foreach loop? Well, on my machine it averages at about one millisecond. Now that’s fast, right?! The cool thing about using yield with the IEnumerable<T> interface is that the work is deferred. That is, not until the program goes to actually run the enumeration do we get our performance hit. Try it out! Try moving the time printout from after the foreach loop to before the foreach loop. Try sticking breakpoints in on the line that yields. You’ll see what I mean.

Summary

In this article, I’ve explored two different ways of implementing an API (specifically focusing on the return value). We saw a brief performance analysis between the two and I highlighted some differences in both approaches. Let’s recap though:

  • Approach 1: Returning a List<T> and creating the collection ahead of time
    • Appeared to be overall a bit faster then yielding.
    • Consumed much more memory than yielding.
    • Callers can use the results immediately for enumeration, checking count, or as a collection to add more things to
    • The return type of List<T> is a bit more restrictive than an IEnumerable<T> like in the second API implementation
  • Approach 2: Return type of IEnumerable<T> and yielding results
    • Appeared to be overall a bit slower than the List<T> implementation
    • Lazy. We don’t actually execute any enumeration code until the caller actually enumerates
    • Consumed significantly less memory than the first approach using List<T>
    • Callers can enumerate the results immediately, but they need to add the results to a collection class to do much more than enumerate

So next time you’re designing an API for your interfaces and classes, try keeping these things in mind!

EDIT (December 30th, 2013):
As per some comments on Google+ by Dan Nemec, I figured I’d add a bit more here in the summary. IEnumerable<T> on it’s own is certainly not useless, especially if you’re leveraging LINQ or extension methods. My main beef in the past was that the consumer of an API with a IEnumerable<T> return value can only iterate over the results… And that’s just because that’s all that IEnumerable<T> lets you do. Dan made a great point though–If you are leveraging things like extension methods, or LINQ (which introduces tons of handy extension methods for working with IEnumerable<T>) then you get all of that functionality tacked on to IEnumerable<T>.

So if you’re not fortunate enough to be working with LINQ or extension methods (i.e. working with legacy code in old .NET framework versions… and yes I am familiar with the attribute you can add in to allow extension methods provided you have a compiler version high enough to support it), then IEnumerable<T> sometimes just plain sucks. I’d wager the majority of C# developers aren’t in this boat though, so I’d like to thank Dan again for his comments.


Lambdas: An Example in Refactoring Code

Lambdas: An Example in Refactoring Code

Background: Lambdas and Why This Example is Important

Based on your experience in C# or other programming languages, you may or may not be familiar with what a lambda is. If the word “Lambda” is new and scary to you, don’t worry. Hopefully after reading this you’ll have a better idea of how you can use them. My definition of a lambda expression is a function that you can define in local scope to pass as an argument provided it meets the delegate signature. It’s probably pretty obvious to you that you can pass in object references and value types into all kinds of functions… But what about passing in a whole function as an argument? And what if you just want to declare a simple anonymous method right when you want to provide it to a function? Lambdas.

So now you at least have a basic idea of what a Lambda is. What’s this article all about? I wanted to discuss a real-world coding experience that helped demonstrate the value of lambdas to me. In my honest opinion, I think having real world programming topics to learn from is more beneficial than many of the “ideal” scenario examples/tutorials you end up reading on the Internet. We can argue and debate that certain things are better or worse in an ideal sense, but when you have a real practical example, it really helps to drive the point home.

So for me, I love working with events. I’m very comfortable with the concept of delegation in C#. I can have one object that may notify anyone that’s interested that something is happening, and the other objects that do care are able to handle the event. Thus, actions can get delegated to those objects that care to be notified. One of my weaknesses at this point in my development experience is leveraging the concept of delegation outside of the realm of events. Delegation is powerful, but it’s certainly not limited to hooking onto events with event handlers.

The particular example I want to illustrate is a parallel of a real coding scenario. I was refactoring some code that was leveraging close to zero OOP practices. I wanted to create a nice extensible framework and class hierarchy to replace it. Once I was done, a few colleagues of mine at Magnet Forensics picked up on a bit of a code smell. We all agreed the new framework and class hierarchy was better, but there seemed to be a lot of boiler plate code going on. We got into the discussion of how lambdas could reduce a lot of the light-weight classes I had introduced. After taking their thoughts and refactoring my changes just a little bit more, the benefits of the lambdas were obvious to me.

So obvious, I had to write about it to share with all of you! Feel free to skip ahead to the downloads section to get the code and follow along with it. There are plenty of options for downloading.

The Scenario

I mentioned that this was a real world scenario. I’ve contrived a parallel example that hopefully demonstrates some of the real world issues while illustrating how lambdas are useful. Let’s imagine we have some big chunk of logic that does data processing. In my real-world scenario, this may have existed as one monolithic function. I would have one big function that, based on all the parameters I provide, can figure out how to process the data I feed it.

Problems:

  • Hard to test (You need to test the whole function even if you’re really just wanting to target a small part of it)
  • Error prone (Any small change to one part can potentially break an entire other part of the function as it grows in complexity)
  • Not extensible (As soon as you need to deviate a little bit from the structure that’s existed, suddenly things get really complicated)

By switching to more of an OOP approach, I can start to address all of the above problems. So in this example, I’ll illustrate what my initial refactoring would have looked like by introducing classes. Afterward, I’ll show what my second refactor may have looked like after taking lambdas into account. In order to stay true to some of the real world problems you might encounter when performing a big refactor like this, I’ve opted to include some fictitious dependency. I refer to this at the “mandatory argument” or “important reference”. You’ll notice in the code that I don’t really use it to do any work, but it’s demonstrating having to pass down some other critical information to my classes that the original function may have had easy access to.

Pre-Refactor: No Lambdas Here!

Let’s start with our new OOP layout. I want to have a factory that can create data processor instances for me. So let’s define what those look like.

First, we have the interface for our data processors:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public interface IProcessor
  {
    bool TryProcess(object input);
  }
}

And then a simple interface for a factory that can create the data processor instances for us:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public interface IProcessorFactory
  {
    IProcessor Create(ProcessorType type, object mandatoryArgument, object value);
  }
}

As you may have noticed, the factory interface I’ve provided above takes a ProcessorType enumeration. You may or may not agree that using an enumeration as an argument for the factory is good practice, but I’m using it to make my example simple. Here’s what our enumeration will look like:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public enum ProcessorType
  {
    GreaterThan,
    LessThan,
    NumericEqual,
    StringEqual,
    StringNotEqual,
    /* we could add countless more types of processors here. realistically,
     * an enum may not be the best option to accomplish this, but for
     * demonstration purposes it'll make things much easier.
     */
  }
}

And now we have a definition for all of the basic building blocks defined. These will also be used later when we refactor, so I wanted to get them out of the way right in the beginning.

Right. So, let’s create an extensible IProcessor implementation. We can address some of our basic requirements (like our artificial dependency) and create something that can easily be built on top of. All of our child classes will just have to handle validating their constructor input and overriding a single method. Easy!

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PreRefactor
{
  public abstract class Processor : IProcessor
  {
    private readonly object _importantReference;

    public Processor(object mandatoryArgument)
    {
      if (mandatoryArgument == null)
      {
        throw new ArgumentNullException("mandatoryArgument");
      }

      _importantReference = mandatoryArgument;
    }

    public bool TryProcess(object input)
    {
      if (input == null)
      {
        return false;
      }

      return Process(_importantReference, input);
    }

    protected abstract bool Process(object importantReference, object input);
  }
}

And now let’s provide the factory that’s going to be making all of these instances for us. Please not that the factory is left incomplete on purpose. I’ll only be providing two actual processor implementations and I’ll leave it up to you to try and fill out the rest!

using System;
using System.Collections.Generic;
using System.Text;

using LambdaRefactor.Processing.PreRefactor.Numeric;
using LambdaRefactor.Processing.PreRefactor.String;

namespace LambdaRefactor.Processing.PreRefactor
{
  public class ProcessorFactory : IProcessorFactory
  {
    public IProcessor Create(ProcessorType type, object mandatoryArgument, object value)
    {
      switch (type)
      {
        case ProcessorType.GreaterThan:
          return new GreaterProcessor(mandatoryArgument, value);
        case ProcessorType.StringEqual:
          return new StringEqualsProcessor(mandatoryArgument, value);
        /*
         * we still have to go implement all the other classes!
         */
        default:
          throw new NotImplementedException("The processor type '" + type + "' has not been implemented in this factory.");
      }
    }
  }
}

And now that we have a factory that can easily create our processors for us, let’s actually define some of our processor implementations.

We’ll start off with a simple processor for checking if some input is greater than a defined value. It should really only work with numeric values, but one of the challenges we need to work with is that our data is only provided to us as an object. As a result, we’ll have to do some type checking on our own.

using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace LambdaRefactor.Processing.PreRefactor.Numeric
{
  public class GreaterProcessor : Processor
  {
    private readonly decimal _value;

    public GreaterProcessor(object mandatoryArgument, object value)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      _value = Convert.ToDecimal(value, CultureInfo.InvariantCulture); // will throw exception on mismatch
    }

    protected override bool Process(object importantReference, object input)
    {
      decimal numericInput;
      try
      {
        numericInput = Convert.ToDecimal(input, CultureInfo.InvariantCulture);
      }
      catch (Exception)
      {
        return false;
      }

      return numericInput > _value;
    }
  }
}

And to put a spin on things, let’s implement a processor that operates on string values only. We’ll implement the processor that checks if strings are equal. Like the GreaterProcessor, we’re forced to get object references passed in. We’ll need to convert these to strings to work with them.

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PreRefactor.String
{
  public class StringEqualsProcessor : Processor
  {
    private readonly string _value;

    public StringEqualsProcessor(object mandatoryArgument, object value)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      _value = (string)value; // will throw exception on mismatch
    }

    protected override bool Process(object importantReference, object input)
    {
      return Convert.ToString(input, System.Globalization.CultureInfo.InvariantCulture).Equals(_value);
    }
  }
}

Where can we go from here?

  • We can make simple inverse processors by overriding others and inverting the return value on the Process() function. Want a StringDoesNotEqual processor? It’s just as easy as  inheriting from the StringEqualsProcessor and then modifying the return of Process(). Then we add this to our factory.
  • Adding other various types of processors is easy. We just have to extend our base class and add a couple of lines to our factory.
  • This code is much easier to test than one monolithic function that does all types of processing. We can now put a nice testing framework around this, and test each method on each class individually.

Post-Refactor: All of the Lambdas!

So… Why don’t we stop here? Because we can do better.

I mentioned that to make a simple inverse processor, all I had to do was override a class and invert the return value of Process(). That’s pretty easy to do… Except I need an entire new class to do it. If I want to make more types of numeric processing, I need to provide similar type checking and conversion. This code gets duplicated every time I go to add another simple class.

I also have my factory class responsible for creating my processor instances. They’re relatively coupled already, but I want developers to have to use my factory to construct instances of processor interface and not worry about the specific implementations. So what if my factory had a bit more say in the construction if the processors? I could use lambdas to pass in the logic that’s unique to each type of processor, and keep each type of processor pretty bare bones. This would move more logic into the factory, but reduce the number of processor implementations I have to make.

So let’s do better!

Let’s start with our new IProcessor implementation. We’ll provide a delegate signature that will be the basis for the lambda expressions we pass in:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PostRefactor
{
  public abstract class Processor : IProcessor
  {
    private readonly object _importantReference;

    public Processor(object mandatoryArgument)
    {
      if (mandatoryArgument == null)
      {
        throw new ArgumentNullException("mandatoryArgument");
      }

      _importantReference = mandatoryArgument;
    }

    public delegate bool ProcessDelegate<T>(object importantReference, T processorValue, T input);

    public bool TryProcess(object input)
    {
      if (input == null)
      {
        return false;
      }

      return Process(_importantReference, input);
    }

    protected abstract bool Process(object importantReference, object input);
  }
}

From here, we can come up with some child classes that that are generic enough for us to work with using lambas that still provide enough functionality for them to exist on their own. We can break our processors up based on the type of data they’ll be working with. That is, we can have a processor for numeric values and a processor for string values. This will cover a lot of the duplicated functionality that exists in the current state of our refactor if we wanted to keep creating new IProcessor implementations.

Let’s start with our NumericProcessor:

using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace LambdaRefactor.Processing.PostRefactor.Numeric
{
  public class NumericProcessor : Processor
  {
    private readonly decimal _value;
    private readonly ProcessDelegate<decimal> _processDelegate;

    public NumericProcessor(object mandatoryArgument, object value, ProcessDelegate<decimal> processDelegate)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      if (processDelegate == null)
      {
        throw new ArgumentNullException("processDelegate");
      }

      _value = Convert.ToDecimal(value, CultureInfo.InvariantCulture); // will throw exception on mismatch
      _processDelegate = processDelegate;
    }

    protected override bool Process(object importantReference, object input)
    {
      decimal numericInput;
      try
      {
        numericInput = Convert.ToDecimal(input, CultureInfo.InvariantCulture);
      }
      catch (Exception)
      {
        return false;
      }

      return _processDelegate(importantReference, _value, numericInput);
    }
  }
}

And similarly, a StringProcessor:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PostRefactor.String
{
  public class StringProcessor : Processor
  {
    private readonly string _value;
    private readonly ProcessDelegate<string> _processDelegate;

    public StringProcessor(object mandatoryArgument, object value, ProcessDelegate<string> processDelegate)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      if (processDelegate == null)
      {
        throw new ArgumentNullException("processDelegate");
      }

      _value = (string)value; // will throw exception on mismatch
      _processDelegate = processDelegate;
    }

    protected override bool Process(object importantReference, object input)
    {
      return _processDelegate(importantReference, _value, Convert.ToString(input, System.Globalization.CultureInfo.InvariantCulture));
    }
  }
}

With these two basic child classes built upon our new IProcessor implementation, we can restructure a new IProcessorFactory implementation. As I mentioned, we can leverage lambdas to move some logic back into the factory class and keep the processor implementations relatively basic.

Here’s the new factory:

using System;
using System.Collections.Generic;
using System.Text;

using LambdaRefactor.Processing.PostRefactor.Numeric;
using LambdaRefactor.Processing.PostRefactor.String;

namespace LambdaRefactor.Processing.PostRefactor
{
  public class ProcessorFactory : IProcessorFactory
  {
    public IProcessor Create(ProcessorType type, object mandatoryArgument, object value)
    {
      switch (type)
      {
        case ProcessorType.GreaterThan:
          return new NumericProcessor(mandatoryArgument, value, (_, x, y) => x <; y);
        case ProcessorType.StringEqual:
          return new StringProcessor(mandatoryArgument, value, (_, x, y) => x == y);
        /*
         * Look how easy it is to add new processors! Exercise for you:
         * implement the remaining processors in the enum!
         */
        default:
          throw new NotImplementedException("The processor type '" + type + "' has not been implemented in this factory.");
      }
    }
  }
}

As you can see, our new factory is simple like our first implementation. The major difference? We’re passing very simple lambdas that would have otherwise been functionality defined in a very light-weight child class. This allows us to move away from having many potentially very bare-bones classes and minimizes the amount of boilerplate duplication.

Summary

I didn’t post it here, but the original implementation that this example paralleled  in real life was a pain to deal with. It was hard to test, brittle to modify/extend, and just downright unwieldly. It was obvious to me that switching to a refactored object-oriented implementation was going to make this style of code easy to extend and easy to test.

The initial refactor posted in this example was a great step in the right direction. The code became easy to build upon by relying on simple OOP principals, and granular parts of the functionality became really easy to test. If I just wanted to test certain types of numeric processing, I didn’t have set up a test for my entire massive “process” function. All I’d have to do is make an instance of the processor I want to test, and call the methods I’d like to cover. Incredibly easy.

Lambdas took this to the next level though. By leveraging lambads, I could refactor even more common code into a base class. This meant that  in order to use my processors properly, the final factory class implementation definitely became required to use. It caused a paradigm shift where instead of making lots of light-weight child classes for additional processor implementations, I’d only need to implement some logic in the factory. All of my existing processors could be refactored into a handful of generic processor classes, and the factory would be responsible for passing in the necessary lambdas.

Lambdas let you accomplish some pretty powerful things, and this refactoring example was one case where they made code much easier to manage. Hopefully you can find a good use for lamba expressions in your next up-coming programming task!

Code Downloads


Fragments: Creating a Tabbed Android User Interface

Fragments: Creating a Tabbed Android User Interface

Fragments: A Little Background

Update: The actual application is available on the Google Play store.

Once upon a time, Android developers used only two things called activities and views in order to create their user interfaces. If you’re like me and you come from a desktop programming environment, an Activity is sort of like a form or a window. Except it’s more like a controller for one of these classes. With that analogy in place, a view is then similar to a control. It’s the visual part you’re interacting with as a user. I remember the learning curve being pretty steep for me being so stuck in my desktop (C# and WPF) development, but once I came up with these analogies on my own, it seemed pretty obvious. So to make an Android application, one would simply put some views together and chain some activities to show these views. Pretty simple.

Something changed along the way though. It was apparent that the Activity/View paradigm was a bit lacking so something new was added to the mix: The Fragment. Fragments were introduced in Android 3.0 (which is API level 11). Fragments added the flexibility to be able to swap out parts of an activity without having to completely redefine the whole view. This means that having an application on a mobile phone with a small screen can appear differently than when it’s on a large tablet, and as a developer you don’t have to redesign the whole bloody thing. Awesome stuff!

So, to clarify, a fragment is just a part of the activity. By breaking up activities into fragments, you get the modular flexibility of being able to swap in and out components at will. If you’re like me and you took a break from Android when fragments were introduced, then you may have another little learning curve. The goal of this article is to create a tabbed Android user interface using fragments.

For what it’s worth, when I first tried putting together a tabbed UI with fragments, it was a complete mess. I was surfing the net for examples, but I couldn’t find anything that really hit it home for me. Once I had it working, I decided I should redo it and document the process. That’s how this article came to be! Another side note… I’m a C# developer by trade and I haven’t developed with Android/Java within a team. If you don’t like my coding conventions then please try to look past that to get the meat of the article!

As per usual, you can follow along by downloading all of the code ahead of time. Please check out the section at the end of the article and pick whichever option you’d like to get the source!

Setting Up: Getting Your Project Together

I’m going to make a few assumptions here. You should have Eclipse installed with the latest Android Development Tools. There are plenty of examples out there for how to get your environment put together, but I’m not going to cover that here.

You’re going to want to start by making a new Android Application in eclipse. By going to the “File” menu, then the “New” sub menu, then the “Other” sub menu, you should get a dialog letting you pick Android application. You’ll get a wizard that looks like the following (where I’ve filled in the information with what I’ll be using for this entire example):

Tab Fragment Tutorial - New Android Application

The first part of the wizard is setting up your Android project.

The wizard gives you some options for what you want to have it generate for you. In this case, I opted out of having a custom icon (since that’s not really important for this tutorial) and I chose to have it create an activity for me.

Tab Fragment Tutorial - Create Activity

The second step in the wizard lets you choose what to create. I wanted just the activity made.

Our activity is actually going to be pretty light-weight. The bulk of what we’re going to be doing is going to be inside of our fragments. Because of this, we should be totally fine just making our main activity a simple blank activity.

Tab Fragment Tutorial - Create Blank Activity

We won’t have much code in our main activity. Let’s just opt for the blank activity.

The final step in the wizard just wants you to confirm the naming for your generated code.

Tab Fragment Tutorial - Activity Naming

Let’s create our “MainActivity” activity with a layout called “activity_main”. Pretty straight forward.

At this point, we actually have an Android application that we can deploy to a phone or a virtual device. If you’re new to Android programming, I suggest you try it out. It’s pretty exciting to get your first little application running.

The Layouts

The layout XML files in Android provide the hierarchies of views that will get shown in the UI. If you haven’t modified the one that was created by default, it will probably look like this:

Tab Fragment Tutorial - Initial Main Activity Layout XML

The default Main Activity XML will look like this. It’s really just a text view that says “Hello World”.

What does that give us? Well, we get a RelativeLayout view that acts as a container for a TextView. The TextView says “Hello World”. Amazing, right?

Let’s switch up our main activity’s layout a bit. Instead of a RelativeLayout, let’s drop in a linear layout that has a vertical orientation. We’ll blow away the TextView too, and drop in a Fragment. Our fragment will need to point to our custom fragment class (which we haven’t created yet). For now, make the class “com.devleader.tab_fragment_tutorial.TabsFragment”. Later in the example, we’ll create the TabsFragment class and put it within this package. When the application runs, it will load up our custom fragment (specified by the full class name) and place it within our LinearLayout.

The layout XML for the main activity looks like the following:


<LinearLayout
 xmlns:android="http://schemas.android.com/apk/res/android"
 android:orientation="vertical"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent">

 <fragment
 class="com.devleader.tab_fragment_tutorial.TabsFragment"
 android:id="@+id/tabs_fragment"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent" />
</LinearLayout>

We’re going to need a layout for our tabs fragment. This is going to be the view portion of the UI that gets dropped in to our main activity. It’s going to be responsible for showing the tabs at the top of the UI and then providing container views for the contents that each tab will want to show.

In order to create this layout, right click on your “layout” folder nested within the “res” folder in the Eclipse IDE. Go to “new”, and then click on the “Other” child menu. Pick “Android XML Layout File” from your list of options. Select “TabHost” as the layout’s root element. Let’s call this file “fragment_tabs.xml”.

The top level component in this layout will be a TabHost. We’ll put our TabWidget in next, which is going to contain the actual tab views, and then a FrameLayout with two nested FrameLayouts inside of it for holding the contents that we want to show for each tab. To clarify, the user will be clicking on views within the TabWidget to pick the tab, and the contents within the tab1 and tab2 FrameLayouts will show the corresponding user interface for each tab.

The layout XML for the tabs fragment looks like the following:


<TabHost
 xmlns:android="http://schemas.android.com/apk/res/android"
 android:id="@android:id/tabhost"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent"
 android:background="#EFEFEF">

 <LinearLayout
 android:orientation="vertical"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent">

 <TabWidget
 android:id="@android:id/tabs"
 android:layout_width="fill_parent"
 android:layout_height="wrap_content" />

 <FrameLayout
 android:id="@android:id/tabcontent"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent">

 <FrameLayout
 android:id="@+id/tab1"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent"
 android:background="#FFFF00" />

 <FrameLayout
 android:id="@+id/tab2"
 android:layout_width="fill_parent"
 android:layout_height="fill_parent"
 android:background="#FF00FF" />

 </FrameLayout>
 </LinearLayout>
</TabHost>

You may have noticed I used some pretty aggressive hard-coded colors in the layout file. I highly advise you switch these to be whatever you want for your application, but when I’m debugging UI layouts I like to use really high contrasting colors. This helps me know exactly where things are (as opposed to having 10 views all with the same background). Maybe I’m a bit crazy, but I find it really helpful.

Now that we have the main activity done and the tab fragment all set up, the last thing we need is to create some sort of layout for our individual tab views. This will be the view that is placed inside of the TabWidget on our tabs fragment layout. These views will have the title of the tab and they’ll be what the user actually interacts with in order to switch tabs.

The layout XML for our simple tab view looks like the following:


<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
 android:layout_width="wrap_content"
 android:layout_height="wrap_content"
 android:orientation="vertical" >

<TextView
 android:id="@+id/tabTitle"
 android:layout_width="fill_parent"
 android:layout_height="wrap_content"
 android:textAppearance="?android:attr/textAppearanceLarge" />

</LinearLayout>

And that’s it for layouts! Just these three simple files. Now, we need to fill out our classes!

The Classes

If we start from the beginning with the classes, the first (and only) class that gets generated for you is the MainActivity class. If you left it untouched (hopefully you did since there was no indication to change it yet!) then you should have a class that looks like:

Tab Fragment Tutorial - Initial Main Activity Class

The default MainActivity class that gets generated after we complete the steps in the wizard.

In order to make this example work, we barely even need to modify this class at all. You’ll notice our MainActivity extends the Activity class. Because we’re going to be using fragments in our application, we need to modify this class to extend the FragmentActivity. In this entire example, I opted to use the Android v4 Support Library. Thus, in order to make this example work, please ensure you’re using FragmentActivity from the package “android.support.v4.app.FragmentActivity“.

Once you’ve made this replacement (“Activity” for “FragmentActivity”) we’re all done in this class. Great stuff, right? Let’s move on.

We’re going to want to make a class that defines what a tab is. In order to make some nice re-usable code that you can extend, I decided to make a base class that defines minimum tab functionality (at least in my opinion). Feel free to extend upon this class later should your needs exceed what I’m offering in this tutorial.

The base TabDefinition class will:

  • Take in the ID of the view where the tab’s content will be put. In our example, this will be the ID for tab1 or tab2’s FrameLayout.
  • Provide a unique identifier to look up the tab.
  • Be required to provide the Fragment instance that will be used when the tab is activated.
  • Be required to create the tab view that the user will interact with in order to activate the tab.

Let’s add a new class called “TabDefinition” to the package “com.devleader.tab_fragment_tutorial”, just like where our MainActivity class is. The code for the TabDefinition class is as follows:


package com.devleader.tab_fragment_tutorial;

import java.util.UUID;

import android.support.v4.app.Fragment;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;

/**
 * A class that defines a UI tab.
 */
public abstract class TabDefinition {
 //
 // Fields
 //
 private final int _tabContentViewId;
 private final String _tabUuid;

 //
 // Constructors
 //
 /**
 * The constructor for {@link TabDefinition}.
 * @param tabContentViewId The layout ID of the contents to use when the tab is active.
 */
 public TabDefinition(int tabContentViewId) {
   _tabContentViewId = tabContentViewId;
   _tabUuid = UUID.randomUUID().toString();
 }

 //
 // Exposed Members
 //
 /**
 * Gets the ID of the tab's content {@link View}.
 * @return The ID of the tab's content {@link View}.
 */
 public int getTabContentViewId() {
   return _tabContentViewId;
 }

 /**
 * Gets the unique identifier for the tab.
 * @return The unique identifier for the tab.
 */
 public String getId() {
   return _tabUuid;
 }

 /**
 * Gets the {@link Fragment} to use for the tab.
 * @return The {@link Fragment} to use for the tab.
 */
 public abstract Fragment getFragment();

 /**
 * Called when creating the {@link View} for the tab control.
 * @param inflater The {@link LayoutInflater} used to create {@link View}s.
 * @param tabsView The {@link View} that holds the tab {@link View}s.
 * @return The tab {@link View} that will be placed into the tabs {@link ViewGroup}.
 */
 public abstract View createTabView(LayoutInflater inflater, ViewGroup tabsView);
}

Now that we have the bare-minimum definition of what a tab in our UI looks like, let’s make it even easier to work with. In my example, I just want to have my tabs have a TextView to display a title–They’re really simple. I figured I’d make a child class of TabDefinition called SimpleTabDefinition. The goal of SimpleTabDefinition is really just to provide a class that takes the minimum amount of information to get a title put onto a custom view.

Please keep in mind that there are many ways to accomplish what I’m trying to illustrate here, but I personally felt having a base class with a more specific child class would help illustrate my point. You could even put in a second type of child class that would make a graphical tab that shows a graphical resource instead of a string resource. Tons of options!

Let’s add another new class called “SimpleTabDefinition” to the package “com.devleader.tab_fragment_tutorial”. The code for SimpleTabDefinition is as follows:


package com.devleader.tab_fragment_tutorial;

import android.support.v4.app.Fragment;
import android.view.Gravity;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.LinearLayout;
import android.widget.TextView;
import android.widget.LinearLayout.LayoutParams;

/**
 * A class that defines a simple tab.
 */
public class SimpleTabDefinition extends TabDefinition {
  //
  // Fields
  //
  private final int _tabTitleResourceId;
  private final int _tabTitleViewId;
  private final int _tabLayoutId;
  private final Fragment _fragment;

  //
  // Constructors
  //
  /**
  * The constructor for {@link SimpleTabDefinition}.
  * @param tabContentViewId The layout ID of the contents to use when the tab is active.
  * @param tabLayoutId The ID of the layout to use when inflating the tab {@link View}.
  * @param tabTitleResourceId The string resource ID for the title of the tab.
  * @param tabTitleViewId The layout ID for the title of the tab.
  * @param fragment The {@link Fragment} used when the tab is active.
  */
  public SimpleTabDefinition(int tabContentViewId, int tabLayoutId, int tabTitleResourceId, int tabTitleViewId, Fragment fragment) {
    super(tabContentViewId);

    _tabLayoutId = tabLayoutId;
    _tabTitleResourceId = tabTitleResourceId;
    _tabTitleViewId = tabTitleViewId;
    _fragment = fragment;
  }

  //
  // Exposed Members
  //
  @Override
  public Fragment getFragment() {
    return _fragment;
  }

  @Override
  public View createTabView(LayoutInflater inflater, ViewGroup tabsView) {
    // we need to inflate the view based on the layout id specified when
    // this instance was created.
    View indicator = inflater.inflate(
      _tabLayoutId,
      tabsView,
      false);

    // set up the title of the tab. this will populate the text with the
    // string defined by the resource passed in when this instance was
    // created. the text will also be centered within the title control.
    TextView titleView = (TextView)indicator.findViewById(_tabTitleViewId);
    titleView.setText(_tabTitleResourceId);
    titleView.setGravity(Gravity.CENTER);

    // ensure the control we're inflating is layed out properly. this will
    // cause our tab titles to be placed evenly weighted across the top.
    LinearLayout.LayoutParams layoutParams = new LinearLayout.LayoutParams(
      LayoutParams.WRAP_CONTENT,
      LayoutParams.WRAP_CONTENT);
    layoutParams.weight = 1;
    indicator.setLayoutParams(layoutParams);

    return indicator;
  }
}

Awesome stuff. Now we can define tabs easily in our application. We just have one more class left, I promise! In the following section, I’ll re-iterate over everything, so if you’re feeling a bit lost… Just hang in there.

The one part we’re actually missing is the fragment that will manage all of our tabs. We created the layout for it already, which has a TabHost, a TabWidget (to contain the clickable tab views), and some FrameLayouts (that contain the content we show when we press a tab). Now we just need to actually attach some code to it!

The TabsFragment class that we’re going to want to add to the package “com.devleader.tab_fragment_tutorial” is responsible for a few things. First, we’re going to be defining our tabs in here. This class will be responsible for taking those tab definitions and creating tabs that get activated via the TabHost. As a result, this fragment class is going to have to implement the OnTabChangedListener interface. This will add a method where we handle switching the fragment shown to match the fragment for the contents of the tab that was pressed.

The code for our TabsFragment class looks like the following:

package com.devleader.tab_fragment_tutorial;

import android.os.Bundle;
import android.support.v4.app.Fragment;
import android.support.v4.app.FragmentManager;
import android.view.LayoutInflater;
import android.view.View;
import android.view.ViewGroup;
import android.widget.TabHost;
import android.widget.TabHost.OnTabChangeListener;
import android.widget.TabHost.TabSpec;

/**
 * A {@link Fragment} used to switch between tabs.
 */
public class TabsFragment extends Fragment implements OnTabChangeListener {
  //
  // Constants
  //
  private final TabDefinition[] TAB_DEFINITIONS = new TabDefinition[] {
    new SimpleTabDefinition(R.id.tab1, R.layout.simple_tab, R.string.tab_title_1, R.id.tabTitle, new Fragment()),
    new SimpleTabDefinition(R.id.tab2, R.layout.simple_tab, R.string.tab_title_2, R.id.tabTitle, new Fragment()),
   };

  //
  // Fields
  //
  private View _viewRoot;
  private TabHost _tabHost;

  //
  // Exposed Members
  //
  @Override
  public void onTabChanged(String tabId) {
    for (TabDefinition tab : TAB_DEFINITIONS) {
      if (tabId != tab.getId()) {
        continue;
      }

      updateTab(tabId, tab.getFragment(), tab.getTabContentViewId());
      return;
    }

    throw new IllegalArgumentException("The specified tab id '" + tabId + "' does not exist.");
  }

  @Override
  public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
    _viewRoot = inflater.inflate(R.layout.fragment_tabs, null);

    _tabHost = (TabHost)_viewRoot.findViewById(android.R.id.tabhost);
    _tabHost.setup();

    for (TabDefinition tab : TAB_DEFINITIONS) {
      _tabHost.addTab(createTab(inflater, _tabHost, _viewRoot, tab));
    }

    return _viewRoot;
  }

  @Override
  public void onActivityCreated(Bundle savedInstanceState) {
    super.onActivityCreated(savedInstanceState);
    setRetainInstance(true);

    _tabHost.setOnTabChangedListener(this);

    if (TAB_DEFINITIONS.length > 0) {
      onTabChanged(TAB_DEFINITIONS[0].getId());
    }
  }

  //
  // Internal Members
  //
  /**
  * Creates a {@link TabSpec} based on the specified parameters.
  * @param inflater The {@link LayoutInflater} responsible for creating {@link View}s.
  * @param tabHost The {@link TabHost} used to create new {@link TabSpec}s.
  * @param root The root {@link View} for the {@link Fragment}.
  * @param tabDefinition The {@link TabDefinition} that defines what the tab will look and act like.
  * @return A new {@link TabSpec} instance.
  */
  private TabSpec createTab(LayoutInflater inflater, TabHost tabHost, View root, TabDefinition tabDefinition) {
    ViewGroup tabsView = (ViewGroup)root.findViewById(android.R.id.tabs);
    View tabView = tabDefinition.createTabView(inflater, tabsView);

    TabSpec tabSpec = tabHost.newTabSpec(tabDefinition.getId());
    tabSpec.setIndicator(tabView);
    tabSpec.setContent(tabDefinition.getTabContentViewId());
    return tabSpec;
  }

  /**
  * Called when switching between tabs.
  * @param tabId The unique identifier for the tab.
  * @param fragment The {@link Fragment} to swap in for the tab.
  * @param containerId The layout ID for the {@link View} that houses the tab's content.
  */
  private void updateTab(String tabId, Fragment fragment, int containerId) {
    final FragmentManager manager = getFragmentManager();
    if (manager.findFragmentByTag(tabId) == null) {
      manager.beginTransaction()
        .replace(containerId, fragment, tabId)
        .commit();
    }
  }
}

And that’s it! Just four classes in total, and one of them (MainActivity) was almost a freebee!

Putting It All Together

Let’s recap on all of the various pieces that we’ve seen in this example. First, we started with the various layouts that we’d need. Our one and only activity is pretty bare bones. It’s going to contain our tabs fragment view. The tabs fragment view is responsible for containing the individual tabs a user clicks on as well as the content that gets displayed for each tab. We also added a layout for really simplistic tab views that only really contain a TextView that shows the tab’s title.

From there, we were able to look at the classes that would back up the views. To use our fragment implementation, we only had to modify our parent class of our only activity. I opted to create some classes that define tab functionality to make extending the UI a bit easier, and adding additional child classes that fit in this pattern is simple. The TabsFragment class was the most complicated part of our implementation, and truth be told, that’s where most of the logic resides. This class was responsible for defining the tabs we wanted to show, and what fragments we would swap in when each tab was clicked.

In order to extend this even more, the things you’ll want to consider are:

  • Defining your own type of tab definition classes. Maybe you want to look at graphical tabs, or something more complicated than just a title.
  • Implementing your own fragment classes that you display when your tabs are clicked. In the example, the contents of the tabs are empty! This is definitely something you’ll want to extend upon.
  • Adding more tabs! Maybe you need three or four tabs instead of two.

Summary

Fragments in Android really aren’t all that complicated. As a new Android developer or transitioning from the pre-API level 11 days, they might seem a bit odd. Hopefully after you try out this example they’re a lot more clear. Hopefully by following along with this tutorial you found that you were easily able to set up a tabbed user interface in Android and get a basic understanding for how fragments work.

Source and Downloads

I like being able to provide the source in as many formats as possible… so here we go:

Update: The actual application is available on the Google Play store.


Dynamic Programming with Python and C#

Dynamic Coding with C# and Python

Dynamic Code: Background

Previously, I was expressing how excited I was when I discovered Python, C#, and Visual Studio integration. I wanted to save a couple examples regarding dynamic code for a follow up article… and here it is! (And yes… there is code you can copy and paste or download).

What does it mean to be dynamic? As with most things, wikipedia provides a great start. Essentially, much of the work done for type checking and signatures is performed at runtime for a dynamic language. This could mean that you can write code that calls a non-existent method and you wont get any compilation errors. However, once execution hits that line of code, you might get an exception thrown. This Stack Overflow post’s top answer does a great job of explaining it as well, so I’d recommend checking that out if you need a bit more clarification. So we have statically bound and dynamic languages. Great stuff!

So does that mean Python is dynamic? What about C#?

Well Python is certainly dynamic. The code is interpreted and functions and types are verified at run time. You won’t know about type exceptions or missing method exceptions until you go to execute the code. For what it’s worth, this isn’t to be confused with a loosely typed language. Ol’ faithful Stack Overflow has another great answer about this. The type of the variable is determined at runtime, but the variable type doesn’t magically change. If you set a variable to be an integer, it will be an integer. If you set it immediately after to be a string, it will be a string. (Dynamic, but strongly typed!)

As for C#, in C# 4 the dynamic keyword was introduced. By using the dynamic keyword, you can essentially get similar behaviour to Python. If you declare a variable of type dynamic, it will take on the type of whatever you assign to it. If I assign a string value to my dynamic variable, it will be a string. I can’t perform operations like pre/post increment (++) on the variable when it’s been assigned a string value without getting an exception. If I assign an integer value immediately after having assigned a string value, my variable will take on the integer type and my numeric operators become available.

Where does this get us with C# and Python working together then?

Example 1: A Simple Class

After trying to get some functions to execute between C# and Python, I thought I needed to take it to the next level. I know I can declare classes in Python, but how does that look when I want to access it from C#? Am I limited to only calling functions from Python with no concept of classes?

The answer to the last question is no. Most definitely not. You can do some pretty awesome things with IronPython. In this example, I wanted to show how I can instantiate an instance of a class defined within a Python script from C#. This script doesn’t have to be created in code (you can use an external file), so if you need more clarification on this check out my last Python/C# posting, but I chose to do it this way to have all the code in one spot. I figured it might be easier to show for an example.

We’ll be defining a class in Python called “MyClass” (I know, I’m not very creative, am I?). It’s going to have a single method on it called “go” that will take one input parameter and print it to the console. It’s also going to return the input string so that we can consume it in C# and use it to validate that things are actually going as planned. Here’s the code:

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Scripting.Hosting;

using IronPython.Hosting;

namespace DynamicScript
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Enter the text you would like the script to print!");
            var input = Console.ReadLine();

            var script =
                "class MyClass:\r\n" +
                "    def __init__(self):\r\n" +
                "        pass\r\n" +
                "    def go(self, input):\r\n" +
                "        print('From dynamic python: ' + input)\r\n" +
                "        return input";

            try
            {
                var engine = Python.CreateEngine();
                var scope = engine.CreateScope();
                var ops = engine.Operations;

                engine.Execute(script, scope);
                var pythonType = scope.GetVariable("MyClass");
                dynamic instance = ops.CreateInstance(pythonType);
                var value = instance.go(input);

                if (!input.Equals(value))
                {
                    throw new InvalidOperationException("Odd... The return value wasn't the same as what we input!");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! There was an exception while running the script: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

Not too bad, right? The first block of code just takes some user input. It’s what we’re going to have our Python script output to the console. The next chunk of code is our Python script declaration. As I said, this script can be loaded from an external file and doesn’t necessarily have to exist entirely within our C# code files.

Within our try block, we’re going to setup our Python engine and “execute” our script. From there, we can ask Python for the type definition of “MyClass” and then ask the engine to create a new instance of it. Here’s where the magic happens though! How can we declare our variable type in C# if Python actually has the variable declaration? Well, we don’t have to worry about it! If we make it the dynamic type, then our variable will take on whatever type is assigned to it. In this case, it will be of type “MyClass”.

Afterwards, I use the return value from calling “go” so that we can verify the variable we passed in is the same as what we got back out… and it definitely is! Our C# string was passed into a Python function on a custom Python class and spat back out to C# just as it went in. How cool is that?

Some food for thought:

  • What happens if we change the C# code to call “go1” instead of “go”? Do we expect it to work? If it’s not supposed to work, will it fail at compile time or runtime?
  • Notice how our Python method “go” doesn’t have any type parameters specified for the argument “input”? How and why does all of this work then?!

Example 2: Dynamically Adding Properties

I was pretty excited after getting the first example working. This meant I’d be able to create my own types in Python and then leverage them directly in C#. Pretty fancy stuff. I didn’t want to stop there though. The dynamic keyword is still new to me, and so is integrating Python and C#. What more could I do?

Well, I remembered something from my earlier Python days about dynamically modifying types at run-time. To give you an example, in C# if I declare a class with method X and property Y, instances of this class are always going to have method X and property Y. In Python, I have the ability to dynamically add a property to my class. This means that if I create a Python class that has method X but is missing property Y, at runtime I can go right ahead and add property Y. That’s some pretty powerful stuff right there. Now I don’t know of any situations off the top of my head where this would be really beneficial, but the fact that it’s doable had me really interested.

So if Python lets me modify methods and properties available to instances of my type at runtime, how does C# handle this? Does the dynamic keyword support this kind of stuff?

You bet. Here’s the code for my sample application:

using System;
using System.Collections.Generic;
using System.Text;

using Microsoft.CSharp.RuntimeBinder;

using IronPython.Hosting;

namespace DynamicClass
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Press enter to read the value of 'MyProperty' from a Python object before we actually add the dynamic property.");
            Console.ReadLine();

            // this script was taken from this blog post:
            // http://znasibov.info/blog/html/2010/03/10/python-classes-dynamic-properties.html
            var script =
                "class Properties(object):\r\n" +
                "    def add_property(self, name, value):\r\n" +
                "        # create local fget and fset functions\r\n" +
                "        fget = lambda self: self._get_property(name)\r\n" +
                "        fset = lambda self, value: self._set_property(name, value)\r\n" +
                "\r\n" +
                "        # add property to self\r\n" +
                "        setattr(self.__class__, name, property(fget, fset))\r\n" +
                "        # add corresponding local variable\r\n" +
                "        setattr(self, '_' + name, value)\r\n" +
                "\r\n" +
                "    def _set_property(self, name, value):\r\n" +
                "        setattr(self, '_' + name, value)\r\n" +
                "\r\n" +
                "    def _get_property(self, name):\r\n" +
                "        return getattr(self, '_' + name)\r\n";

            try
            {
                var engine = Python.CreateEngine();
                var scope = engine.CreateScope();
                var ops = engine.Operations;

                engine.Execute(script, scope);
                var pythonType = scope.GetVariable("Properties");
                dynamic instance = ops.CreateInstance(pythonType);

                try
                {
                    Console.WriteLine(instance.MyProperty);
                    throw new InvalidOperationException("This class doesn't have the property we want, so this should be impossible!");
                }
                catch (RuntimeBinderException)
                {
                    Console.WriteLine("We got the exception as expected!");
                }

                Console.WriteLine();
                Console.WriteLine("Press enter to add the property 'MyProperty' to our Python object and then try to read the value.");
                Console.ReadLine();

                instance.add_property("MyProperty", "Expected value of MyProperty!");
                Console.WriteLine(instance.MyProperty);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! There was an exception while running the script: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

Let’s start by comparing this to the first example, because some parts of the code are similar. We start off my telling  the user what’s going to happen and wait for them to press enter. Nothing special here. Next, we declare our Python script (again, you can have this as an external file) which I pulled form this blog. It was one of the first hits when searching for dynamically adding properties to classes in Python, and despite having limited Python knowledge, it worked exactly as I had hoped. So thank you, Zaur Nasibov.

Inside our try block, we have the Python engine creation just like our first example. We execute our script right after too and create an instance of our type defined in Python. Again, this is all just like the first example so far. At this point, we have a reference in C# to a type declared in Python called “Properties”. I then try to print to the console the value stored inside my instances property called “MyProperty”. If you were paying attention to what’s written in the code, you’ll notice we don’t have a property called “MyProperty”! Doh! Obviously that’s going to throw an exception, so I show that in the code as well.

So where does that leave us then? Well, let’s add the property “MyProperty” ourselves! Once we add it, we should be able to ask our C# instance for the value of “MyProperty”. And… voila!

Some food for thought:

  • When we added our property in Python, we never specified a type. What would happen if we tried to increment “MyProperty” after we added it? What would happen if we tried to assign an integer value of 4 to “MyProperty”?
  • When might it be useful to have types in C# dynamically get new methods or properties?

Summary

With this post, we’re still just scratching the surface of what’s doable when integrating Python and C#. Historically, these languages have been viewed as very different where C# is statically bound and Python is a dynamic language. However, it’s pretty clear with a bit of IronPython magic that we can quite easily marry the two languages together. Using the “dynamic” keyword within C# really lets us get away with a lot!

Source code for these projects is available at the following locations:


Python, Visual Studio, and C#… So. Sweet.

Python, Visual Studio, and C#

Python & C# – Background

Let’s clear the air. Using Python and C# together isn’t anything new. If you’ve used one of these languages and at least heard of the other, then you’ve probably heard of IronPython. IronPython lets you use both C# and Python together. Pretty legit. If you haven’t tried it out yet, hopefully your brain is starting to whir and fizzle thinking about the possibilities.

My development experiences is primarily in C# and before that it was VB .NET (So I’m pretty attached to the whole .NET framework… We’re basically best friends at this point). However, pretty early in my career (my first co-op at Engenuity Corporation, really) I was introduced to Python. I had never really used a dynamic or implicitly typed language, so it was quite an adventure and learning experience.

Unfortunately, aside from my time at EngCorp, I hadn’t really had a use to continue on with Python development. Lately, I’ve had a spark of curiosity. I’m comfortable with C#, sure, but is that enough? There’s lots of great programming languages out there! It’s hard for me to break out of my comfort zone though. I’m used to C# and the awesomeness of Visual Studio, so how could I ever break free from these two things?

Well… I don’t have to yet.

Python Tools for Visual Studio

This was a nice little treasure to stumble upon:

But I didn’t really know what it was all about. I had heard of IronPython, and I knew I could use Python with C# together, so what exactly is “Python Tools“?

After I watched the video that the Visual Studio team tweeted out, I was captivated. Did this mean I could revisit python without having to leave the comfort of my favourite IDE? You bet. First thing I did after watching this video (and yes, I somehow managed to hold back the excitement and wait until the video was done) was fire up Visual Studio. I run with Visual Studio 2012 (the dark theme too) so in my screenshots that’s what you’ll be seeing. Once Visual Studio has loaded:

  • Go to the “Tools” menu at the top of the IDE.
  • Select the “Extensions and Updates…” menu item.
  • You should see the “Extensions and Updates” dialog window now.

You’re going to want to search for “Python Tools” after you’ve selected the “Online” option on the left side of the dialog. It should look something like this:

Python Tools - Visual Studio Extensions and Updates

Installing Python Tools for Visual Studio is pretty easy. Make sure you’re searching online and search for “Python Tools”.

After you’ve followed all of the installation instructions, it’s time to make sure the installation worked. Simple enough!

  • Go to the “File” menu at the top of the IDE.
  • Go to the “New” menu item.
  • Select the “Project…” menu item.
  • You should now see the “New Project” dialog

To ensure Python is now available, try seeing if you have Python project templates available:

Verify Python in Visual Studio

To verify that Python is now available in Visual Studio, check under the installed templates. It should be under “Other Languages”.

Hopefully it’s there. If not, or if you have any other install questions, I highly recommend you refer to the official site and follow along there. This is what got me up and running with my current machine, but if your setup is slightly different you should definitely follow their instructions. That’s it! You have Python Tools! But what else would make your C#, Python, and Visual Studio experience EVEN BETTER? The answer to that question is of course IronPython. Head on over to this page and get yourself setup with the latest cut of IronPython. Once that’s setup, you should have all the fancy tools you need!

Print to Console – Your First C#/Python Application

I’m sure you feel the excitement building. I’ll start by saying the code is all available online, so even though I’ll have snippets and pictures here, you can download all of the source and follow along that way if you want. Otherwise, I’ll do my best to walk you through how I set things up! This application is going to be pretty simple. It’s a tiny bit bigger than a “Hello World” application, with the difference being that you tell Python what you want to print to the console. Easy-peasy, right?

First up, let’s make a new C# console project.

  • From Visual Studio, go to the “File” menu at the top of the IDE.
  • Select the “New” menu item.
  • Select the “Project” menu item.
  • You should see the “New Project” dialog.
  • Select the “Visual C#” template on the left of the dialog.
  • Select “Console Application”.
  • In the framework dropdown at the top of the dialog, select .NET 4.5
  • Fill in the details for where you want to save your project.
  • Press “OK”! And we’re off!

Now that you have a console application you’re going to want to add in all the dependencies we need. If you look at the project in your solution explorer, you’re going to want to add the following dependencies:

IronPython Dependencies in Visual Studio

Add the IronPython and Microsoft.Scripting dependencies through the solution explorer in Visual Studio.

If you’re having trouble getting the dependencies set up, remember you can always download the source projects I’ve put together. Now that you have all the necessary dependencies, here’s the source for our little application:

using System;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;

using IronPython.Hosting;

namespace PrintToConsole
{
    internal class Program
    {
        private static void Main()
        {
            Console.WriteLine("What would you like to print from python?");
            var input = Console.ReadLine();

            var py = Python.CreateEngine();
            try
            {
                py.Execute("print('From Python: " + input + "')");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! We couldn't print your message because of an exception: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

Let’s walk through what this code is doing:

  • First we’re getting input from the user. This is some pretty basic C# stuff, but we’re simply printing a message to the console and taking in the text the user enters before they press enter.
  • Next, we create a Python engine instance. This is the class that’s going to be responsible for executing python for us!
  • The code that exists within the try block tells our engine instance to execute some python code.
    • The print() method that you see being passed to the engine is the syntax since Python 3.0.
    • The parameter that we’re passing into the print() method is a python string… but we’re sticking our user input inside of it as well!
    • It’s also important to note that we’re building up a C# string that contains all of the Python code that will be executed and passing that to the engine.
  • I have a catch block here to catch any unexpected problems. Can you think of any?
    • What happens if your user input some text with a single quote?
  • The last part of the application just asks the user to press enter when they are all done.

Simple! There’s your first C# + Python application! You can see the source for the whole thing over here.

Run External Script

So this is great: you can now run some python code from within C#. Totally awesome. But what about all those python scripts you have written up already? Do you need to start copying and pasting them into C# code files and start to try and format them nicely? The answer is no, thankfully! Let’s start by following the exact same steps as outlined in the first example. You should be able to set up a new .NET 4.5 C# console project and add in all the same dependencies. Once you have that put together, you can use the following source code:

using System;
using System.Collections.Generic;
using System.Text;

using IronPython.Hosting;

namespace RunExternalScript
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Press enter to execute the python script!");
            Console.ReadLine();

            var py = Python.CreateEngine();
            try
            {
                py.ExecuteFile("script.py");
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! We couldn't execute the script because of an exception: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

This script looks similar, right? Before I explain what it does, let’s add in the Python script that you’ll be executing from this console application.

  • Right click on your project in the solution explorer.
  • Select the “Add” menu item from the context menu.
  • Select the “New Item…” menu item.
  • You should see the “Add New Item” dialog.
  • You’ll want to add a new text file called “script.py”.

It should look a little something like this:

Add new Python script in Visual Studio

In the “Add New Item” dialog, select “Text File” and rename it to “script.py”.

The next really important step is to ensure that this script gets copied to the output directory. To do this, select your newly added script file in the solution explorer and change the “Copy to Output Directory” setting to “Copy Always”. Now when you build your project, you should see your script.py file get copied to the build directory. Woo! You can put any python code you want inside of the script file, but I started with something simple:

print('Look at this python code go!')

Okay, so back to the C# code now. This example looks much like the first example.

  • Wait for the user to press enter before executing the Python script. Just to make sure they’re ready!
  • Create our engine instance, just like in the first example.
  • In the try block, we tell the engine to execute our script file. Because we had the file copy to the output directory, we can just use a relative path to the file here.
  • Again, we’ve wrapped the whole thing inside of a try/catch to ensure any mistakes you have in your python script get caught.
    • Try putting some erroneous Python code in the script file and running. What happens?
  • Finally, make sure the user is content with the output and wait for them to press Enter before exiting.

Look how easy that was! Now you can choose to execute Python code generated in C# OR execute external Python scripts!

Summary

It’s awesome to see that you expressed an interest in trying to marry these two languages together inside of a powerful IDE. We’re only breaking through the surface here, and admittedly I’m still quite new to integrating Python and C# together. I need to re-familiarize myself with Python, but I can already see there is a ton of potential for writing some really cool applications this way.

In the near future, I’ll be discussing how the dynamic keyword in C# can actually allow you to create classes in Python and use them right inside of C#… Dynamically!

Both of these pages were helpful in getting me up and running with C# and Python together:

Source code for these projects is available at the following locations:


Events: Demystifying Common Memory Leaks

Events: Demystifying Common Memory Leaks

Background

If you’ve poked through my previous postings, you’ll probably notice that I love using events when I program. If I can find a reason to use an event, I probably will. I think they’re a great tool that can really help you with designing your architectures, but there are certainly some common problems people run into when they use events. The one I want to address today has to do with memory leaks. That’s right. I said it. Memory leaks in your .NET application. Just because it’s a managed language doesn’t mean your code can’t be leaking memory! And now that I’ve got your attention, let’s see how events might be causing some leakage in your application.

(There is source that you can download and run. Check the summary section at the end!)

Instance-Scope Event Handlers

One of the most common ways to set up an EventHandler in C# is by having them defined for the entire scope of the instance. Consider for a moment the form designer in Visual Studio. When you double click on controls you get some handler created for the default event on that control. See how the EventHandler was declared though? You get a method declared that has a sender and some type of EventArgs. Pretty standard stuff here and there’s nothing ground-breaking about it. So what’s the problem with this method?

Well, there’s nothing wrong with it as long as you know how to clean up after yourself. Consider the following two classes:


private class ObjectWithEvent
{
    ~ObjectWithEvent()
    {
        Console.WriteLine(this + " is being finalized.");
    }

    public event EventHandler<EventArgs> Event;

    public void UnhookAll()
    {
        Event = null;
    }
}

private class ObjectThatHooksEvent
{
    public ObjectThatHooksEvent(ObjectWithEvent objectWithEvent)
    {
        objectWithEvent.Event += ObjectWithEvent_Event;
    }

    ~ObjectThatHooksEvent()
    {
        Console.WriteLine(this + " is being finalized.");
    }

    private void ObjectWithEvent_Event(object sender, EventArgs e)
    {
        // some fancy event
    }
}

The first class has an event that our second class can hook onto. You’ll notice in the second class that I’ve defined an instance-scope handler that we can hook up. This is the exact same syntax for declaring an event handler that you’d get from the form designer if you’re doing GUI programming.

The danger with this setup is that until you unhook the event, the object that hooks onto the event will not be freed. “Well, no problem!” is what you might be thinking. You know how to solve that. You can just unhook the event in the second class’s finalizer/deconstructor.

…Except that won’t work. The finalizer will not get called on the instance of the second class until the event has been unhooked! It’s a bit of a chicken-or-the-egg problem, but it makes sense. A finalizer will only be called when the reference is being cleaned up, but the instance can’t be marked for cleaning because something is still using its event handler. See why this can get a bit dangerous?

Anonymous Delegate (No Parent Reference)

So this is an example of hooking events where you won’t get a leak. Why am I showing it? Well, in the next section I’ll make a small tweak to it which will make it behave just like the first scenario I described.

Let’s assume we have two classes again. I’ll use the first class from my first example (the object with the event) and this new class here that we’ll use to hook onto the event:

private class HookWithAnonymousDelegate
{
    public HookWithAnonymousDelegate(ObjectWithEvent objectWithEvent)
    {
        objectWithEvent.Event += (sender, args) =>
        {
            // handle your event
            // (this one is special because it doesn't use anything related to the instance)
            Console.WriteLine("Event being called!");
        };
    }

    ~HookWithAnonymousDelegate()
    {
        Console.WriteLine(this + " is being finalized.");
    }
}

Notice the difference from the first example? I’ve hooked up an anonymous method (using a lambda expression) to our event instead of declaring an instance-scope event handler. It’s a small change, and for the most part, I might argue that this is just a stylistic thing. If you don’t ever plan on unhooking the event then it’s not such a big deal to go with anonymous methods, but if your method body grows pretty big the code can definitely get unsightly.

Anyway… sweet! We just hooked up to our event and we don’t have the scary leak situation that we did in the first scenario. How cool is that? Well…

Anonymous Delegate (With Parent Reference)

The second method I described works great… until you go to put it into practice. It’s clearly not an impossible situation, but it’s pretty unlikely that you’ll write event handlers within an object that don’t use any of that object’s state (or even other methods on the object). Again, not impossible but just not the common use case. And since it’s not the common use case, you need to be concerned with the potentially problematic common use case 🙂

Let’s consider two classes (yes, again, two classes). We’ll use the first class I described above in both examples that has an event that we can hook onto, and a second class that looks similar to the class I introduced in the second example:

private class HookWithAnonymousDelegate2
{
    public HookWithAnonymousDelegate2(ObjectWithEvent objectWithEvent)
    {
        objectWithEvent.Event += (sender, args) =>
        {
            // handle your event and use something that's part of this instance
            SomeInnocentLittleMethod();
        };
    }

    ~HookWithAnonymousDelegate2()
    {
        Console.WriteLine(this + " is being finalized.");
    }

    private void SomeInnocentLittleMethod()
    {
        Console.WriteLine("... Not so innocent after all!");
    }
}

See the difference compared to example 2? The event handler in this class calls an instance method. This would be a pretty common thing to do (unless you like to duplicate all of your code and not use methods ever :P) and it doesn’t look like it should cause problems. And really, it won’t if you understand the implications of hooking an event handler up to an event. So once you’re done handling your events, make sure you clean up and remove your handlers!

In my opinion, the really interesting part of this example is that the event handler is only calling an instance method. It’s not even using any variables or properties of the instance. Still, the .NET framework is going to hold onto this second instance until we unhook.

Summary

Well, hopefully I haven’t scared you away from using events. The take-away point here is that you need to be mindful of hooking up your events and when/where you unhook them. Personally, unless you always plan to have two objects exist for the same lifetime, I wouldn’t hook up events in the constructors like I’ve done in my examples. Some closing tips:

  • Try only hooking onto events when you need to. If you don’t need to hook up all your events when initializing something, then don’t!
  • Be mindful of how you’re going to clean up your event hooking. Whenever you add an event handler, try to think of where you’ll be cleaning it up.
  • Hooking events onto singletons or global instances can make this problem a lot worse. Since your singleton will be around for the lifetime of your application, if you forget to unhook from your event then you’ll start accumulating a lot of garbage.

I’ve written up a little sample application that uses the example classes and walks you through the three examples I’ve outlined. All of them involve instantiating the classes, hooking up the events, and then how they behave differently when you try to clean them up. You can grab the source code from:

Hope you enjoyed! Remember to follow Dev Leader:


API: Don’t Forget About The Non-Public API!

Image courtesy of FreeDigitalPhotos.net

Background

From an object oriented programming perspective, an application programming interface (API) is often referred to as the way other developers can interact with the public members of your class(es) and interface(s). Of course, API can be used to describe how one interacts with a web service (or other types of services), but for this discussion I’m limiting the scope to that of interfaces and classes. Limiting the definition of API to public members (or the equivalent of C#’s “public” in other languages) is omitting one huge part of what it encompasses. The purpose of this post is to clarify, in my opinion, why I think forgetting about the non-public API can lead to bad framework and API designs.

API And The Audience

I’ve written before about what I think makes a good API, and I had some comments on Code Project on the same posting that lead me to this topic: the audience. There’s a lot to consider when you’re writing what many people would consider good, clean code. It’s impossible to please everyone because everyone has some sort of best practice, guideline, or convention that they follow because they believe it’s the best. So, I’m not going to tell you that some way is the best… I just want you to be conscious of your audience so that you can make better decisions.

Okay, okay… So what do I mean by audience? I’m going to generalize your audience (the consumers of your API) into two distinct categories. The first category is the group of developers who will be using references to things that implement your interfaces and your concrete classes. They’ll use the instances of the things you define. For example, let’s consider a something built-in to the .NET framework: the List<T> class. Your first group of API consumers are just going to use instances of this class exactly as provided by the framework. They’ll make new instances of them and pass them around to be used in functions, or make properties that return instances of List<T>, or declare variables of type List<T>, etc… They use it as is, as provided.

The second general group of API consumers are the ones who are going to be extending your interfaces and classes. A great example of this is the EventArgs class. This class is super basic; It doesn’t do anything! Anyone who wants to use the EventArgs class essentially has to make their own class that inherits from EventArgs. Another example of this is the Exception class. Again, this class is built for people to extend it with their own implementations. I suppose both of these examples are pretty primitive because the base classes don’t offer much functionality, but what if your class wanted to let child classes override the default behaviour? I’ll have some more concrete examples of this later on.

Intentions Of The Audience

With these two general types of consumers defined, it’s a bit easier to consider how people may want to use your API differently. The first class of consumer wants to able to easily call your methods that you’ve defined and easily create the objects/interfaces that are core to your API. This means that the input they need to provide should be extremely basic, so using things like built in interfaces, or other classes that you’ve defined that are easy to create. These consumers also want information-rich return values and classes. Why? Because it makes their life easy! If they only need to provide a little bit of information and they get a lot back, they’re able to do a lot more with the data that they have access to. They don’t need to (and they certainly don’t want to) call 10 different things in intricate ways to get a little bit of data back.

The second class of consumer takes the exact opposite perspective. Here’s why. In the first case where the first group of consumers want to pass in only minimal information to methods, the second class of consumer wants lots of information passed in. This class of consumer is required to do some job or return some data, so the more information they are provided the easier it is for them to perform their job. Similarly, they want the return values of methods to be as simplistic as possible. Why? It makes their job easier! If they are responsible for returning some incredibly complex class from a method defined in your interface, and the input to that method is only minimal information, this makes the job of the second class of consumer quite difficult.

The best way to remember these similarities and differences is that the flow of data is opposite depending on what type of API consumer you’re talking about. The first type of consumer wants to provide a little and get a lot, and the second type of consumer wants to get a lot and provide a little. Makes sense right? Everyone wants to do the easy thing.

The take-away here is that depending on who you think your main audience is going to be for your API, it will affect how you structure it. And this is exactly why forgetting about the non-public API can be a huge mistake. Forgetting this part of the API makes the whole thing difficult to extend because your base classes cannot as easily be built on top of. You actually make the lives of the second class of API consumers very difficult if you ignore their needs.

Example: WinForms

If you’ve done desktop application development in C#, you’ve likely used (or at least heard of) WinForms. Some people new to desktop application development might have actually started with Windows Presentation Foundation (WPF), but the same concepts will apply here. In my opinion WinForms is a great example of an API that has both public and non-public components that were designed with both audience types in mind. Let’s start with the first class of API consumers.

If you’ve dabbled in WinForms, you’re likely familiar with the Windows Form Designer offered in Visual Studio. If you are… then congrats! You’re the first class of API consumer that I’ve described. By using the built in classes like Buttons, TextBoxes and Labels, you’re using the out-of-the-box components offered in the framework and consuming the public API offered by these controls. You’ll be using things like the Text property offered on these controls and interacting with them via their events (i.e. hooking onto click events or text changed events). You’ll be using the public API. Nothin’ wrong with that!

//
// MyButton
//
this.MyButton.Location = new System.Drawing.Point(146, 84);
this.MyButton.Name = "MyButton";
this.MyButton.Size = new System.Drawing.Size(75, 23);
this.MyButton.TabIndex = 0;
this.MyButton.Text = "Click Me!";
this.MyButton.UseVisualStyleBackColor = true;
this.MyButton.Click += new System.EventHandler(this.MyButton_Click);

private void MyButton_Click(object sender, EventArgs e)
{
    // do stuff!
}

Now, the creators of WinForms weren’t dummies. They knew that they weren’t going to be able to offer you every possible control you’d ever need. They made the API in WinForms have great support for the non-public members of their base classes! So, what do I mean by that?

Let’s pretend we want to have our own fancy button class. Because our button is fancy, we always want to tell the user just how fancy it is when they click it. Now if the framework you were using had a poorly designed API, you might be forced to code a button from scratch. That would be pretty lame considering the complexity of the built-in controls offered to you already. For this example, you could surely just create one button and hook an event handler onto the click event (using the public API), but what if you wanted to re-use this everywhere throughout your user interface? You’d want your own FancyButton class that has this behaviour built-in so you can easily reuse it. No problem.

private class FancyButton : Button
{
    protected override void OnClick(EventArgs e)
    {
        base.OnClick(e);
        this.Text = "The fancy button was clicked.";
    }
}

The non-public API in WinForms gives you access to built-in behaviour of the base classes. You don’t need to hook onto events to get the job done, you can actually override the OnClick method and prevent the click event from even firing! The focus on the non-public API allows developers to extend the built-in classes without having to design their own from the ground up.

Summary

It takes a lot of practice and experience to be able to write a good API. There’s also plenty of different opinions on what constitutes a well-designed API. In my opinion, you need to give a lot of thought as to how your API will be used. Consider the two general types of API consumers I’ve defined: The consumers that use your API with the public parts of the interfaces and classes you’ve defined, and the consumers that want to extend your defined classes to provide their own related implementations. These two types of consumers will want very different things, and in order to please the second class of consumer, you absolutely cannot forget about the non-public API.

Some food for thought:

  • How can I best guess how developers will use my API?
  • I’m providing base classes with my framework and API. Can people easily extend them through inheritance? Will people want to extend them?
  • What would make my public API easier for other developers to use?
  • What would make my non-public API easier for other developers to build on to my base classes?

Singletons: Why Are They “Bad”?

Background

The very first thing I want to say is that I don’t think singletons are inherently bad–even if it means I am cast away from the rest of the programming world. There’s a time and a place for the singleton. It’s really common for people to get caught up with their perspective on something that they outright refuse to acknowledge the other side of it. I’d also like to point out that if you have a strong opinion on something and you find that other people also have a strongly opposing opinion on the same thing, there’s probably good take away points from either side. In this post though, I’m going to focus on why singletons are “bad”, because for me it means acknowledging one of the two main perspectives–that they are the best thing since cat videos met The Internet or they are the worst thing since Justin Bieber.

Let’s clarify what a singleton is here so we’re all on the same page. Maybe you’re under the impression it’s something slightly different than what I’m about to be talking about, so I’d rather make it clear from the beginning. And for what it’s worth, if this isn’t the exact meaning as set in stone by the singleton gods, then that’s sort of unfortunate… because this is going to be what I’m discussing.  From Design Patterns: Elements of Reusable Object-Oriented Software by the Gang of Four, a singleton must:

Ensure a class has only one instance and provide a global point of access to it.

So, with that incredibly complex definition, let’s get into it.

 

Global Dependencies

There’s approximately 3.2 billion articles on The Internet that will tell you that global variables are the enemy. I mean, I could do some of the work for you, but you could check out this, or this, or this… There’s still billions more. Usually when we have ~99% of people agreeing on something, they probably have a pretty good point, right? You’ll notice after a bit of searching that one *big* problem with singletons is the fact that they are just a different way to dress a global variable. Thus, all of the arguments for why global variables are bad could be used against singletons.

There are a few (quite a few) major problems with having global dependencies:

  • You’ll be putting yourself at risk for dealing with deadlocks if you need to lock your resources
  • Your singleton can become the resource bottleneck in your application
  • It’s hard to know who you’ll affect by modifying the global variable
  • Testing becomes difficult because the tests may depend on the state of the singleton
  • Dependencies aren’t obvious from examining the API

Quite simply, global dependencies can be pretty scary. If you and you’re team are experienced enough, trying to weave in some new features or heavily modify code relying on singletons may not be that challenging for you. Maybe. But testing can get super messy.

When you write unit tests, you want to ensure that each test can be run independently of the others. You need complete control over your state. This can be a big problem if you run two tests in a row that depend on state provided by a singleton. Consider the following set of tests that depend on a singleton in a sample run:

  • Test1 increases the count property on a singleton by 1 as part of the side effect of the test.
  • Test2 also increases the count property of the same singleton instance by 1, just as Test1 did. However, Test2 is explicitly testing this property as part of it’s validation to ensure the value transitioned from 0 to 1.

When you run Test1 all by itself, it behaves as you expect. Great! Now you move onto Test2. Simple. You got it working. You’re all excited so you run them all together. Uh oh… The tests failed! But how? You don’t believe it, so you run them all again. Now they pass! This is a crappy spot to be in. If the test suite executes Test2 before Test1, your tests will work. If the test suite executes them in the opposite order, your tests will fail because the singleton instance will have had its count property increased twice. The global state of the singleton comes back to haunt you.

 

Tight Coupling

Most experienced developers know that you want to decrease coupling and increase cohesion because this, generally speaking, makes for a good API and extensible code base. Now this point is related to the global dependency points I was previously stating, but it deserves to be addressed on it’s own. The global dependency topic was more of a focus on the application at run time. That is, it becomes difficult to know and manage how your application behaves at run time when you share global state across the entire thing. It’s challenging to to start adding new functionality to your application or modify existing components of your application that depend on the global state because the dependencies just keep growing.

Coupling, in my opinion, is more of an issue with compile time. With singletons, you start to design classes that depend on singletons and thus you are relying on a specific implementation (Maybe you could check out dependency injected singletons?). Singletons, generally speaking, are a single instance of a concrete class. You start to code classes and object hierarchies that are then depending on this concrete class, and this can potentially (and I say potentially, because I would never claim it’s “always”) lead you to a dead end. Let’s consider a scenario:

We’re coding an application that uses a singleton for the data access layer of our application. We have a MySQL data model that we’ve coded as a singleton. We expose a few methods to read and write records to the database using some SQL, and things are great. Then, one day, we speak to our customers like we usually do to ensure we’re meeting their expectations. They inform us that we really need to be able to support a document database like MongoDB in addition to MySQL. Hold on. Wait. You mean our code that uses our data layer needs to be agnostic to the database under the hood?! But… But our singleton is only able to deal with MySQL… (I wrote about how to get yourself out of this situation here, although I would not claim it’s the ideal scenario). If we would have been passing around references to something that met a nice and clean model interface spec, we’d likely be able to hide the implementation details and completely avoid this problem.

 

Singletons Disrupt The API

Because I like to design complex systems in code, API and architecture is something I like to focus on. There’s an awesome posting over here by Miško Hevery about this very problem. He does a great job describing the problem with examples, so I’ll only try to summarize some of the main take-away points I got from it.

There’s nothing to stop someone from putting some heavy initialization logic in a singleton. I mean, you shouldn’t (because you can’t guarantee when this is going to happen!) but there’s nothing that prevents it. As such, simply calling a constructor on some class (which you might assume to happen pretty quickly) actually ends up taking seconds. Not a couple milliseconds… But seconds. Oh. I guess you didn’t realize your class was trying to call a singleton instance that was connecting over The Internet to some host on the other side of the planet during its initialization. Surprise. Singletons can mask this kind of stuff because it’s not explicit in the API. It kind of goes back to coupling but I wanted to point out that this kind of stuff can get scary.

This can be completely mitigated by incorporating the dependencies right into the API. There’s no hiding crazy database initialization or downloading data from the internet in a constructor (well… it just reduces the likelihood of you doing it so easily I guess). You use interfaces and construct classes in the order that you need them, and this doesn’t end up being some magical process that happens behind a curtain. If one class depends on another, so be it (it’d be nice if it just depended on the interface…), but pass in the reference that you require. Singletons often end up being the shortcut, but what’s easier for you to code now may not be easier for someone to extend upon and understand in the future.

 

Summary

Singletons. You’ve likely seen them. You’ve likely heard bad things about them. You may have even used one yourself. Shame on you. Like all debatable things though, there’s always going to be another side to it. If you take away anything from reading this, I hope it’s that you question what the other perspective is. Fully understanding something requires you to look at all sides.

 

References

This article was based on information I obtained from the following sources (as well as my own experiences, of course!):


  • Nick Cosentino

    Nick Cosentino

    I work as a team lead of software engineering at Magnet Forensics (http://www.magnetforensics.com). I'm into powerlifting, bodybuilding, and blogging about leadership/development topics over at http://www.devleader.ca.

    Verified Services

    View Full Profile →

  • Copyright © 1996-2010 Dev Leader. All rights reserved.
    Jarrah theme by Templates Next | Powered by WordPress