Programming

ProjectXyz: Enforcing Interfaces (Part 2)

Enforcing Interfaces

This is my second installment of the series related to my small side project that I started. I mentioned in the first post that one of the things I wanted to try out with this project is coding by interfaces. There’s an article over at CodeProject that I once read (I’m struggling to dig it up now, arrrrrghh) that really gave me a different perspective about using interfaces when I program. Ever since then I’ve been a changed man. Seriously.

The main message behind the article was along the lines of: Have your classes implement your interface, and to be certain nobody is going to come by and muck around with your class’s API, make sure they can’t knowingly make an instance of the class. One of the easiest ways to do this (and bear with me here, I’m not claiming this is right or wrong) is to have a hidden (private or protected) constructor on your class and static methods that let you create new instances of your class. However, the trick here is your static method will return the interface your class implements.

An example of this might look like the following:


public interface IMyInterface
{
    void Xyz();
}

public sealed class MyImplementation : IMyInterface
{
    // we hid the constructor from the outside!
    private MyImplementation()
    {
    }

    public static IMyInterface Create()
    {
        return new MyImplementation();
    }

    public void Xyz()
    {
        // do some awesome things here
    }
}

Interesting Benefits

I was pretty intrigued by this article on enforcing interfaces for a few reasons and if you can stick around long enough to read this whole post, I’ll hit the cons/considerations I’ve encountered from actually implementing things this way. These are obviously my opinion, and you can feel free to agree or disagree with me as much as you like.

  • (In theory) it keeps people from coming along and tacking on random methods to my classes. If I have an object hierarchy that I’m creating, having different child classes magically have random public APIs changing independently seems kind of scary. People have a harder time finding ways to abuse this because they aren’t concerned with the concrete implementation, just the interface.
  • Along with the first point, enforcing interfaces makes people think about what they’re doing when they need to change the public API. Now you need to go change the interface. Now you might be affecting X number of implementations. Are you sure?
  • Sets people up nicely to play with IoC and dependency injection. You’re already always working with interfaces because of this, now rolling out something like Moq or Autofac should be easier for you.
  • Methods can be leveraged to do parameter checks BEFORE entering a constructor. Creating IDisposable implementations can be fun when your constructor fails but and your disposable clean up code was expecting things to be initialized (not a terribly strong argument, but I’ve had cases where this makes life easier for me when working with streams).

Enforcing Interfaces in ProjectXyz

I’ve only implemented a small portion of the back-end of ProjectXyz (from what I imagine the scope to be) but it’s enough where I have a couple of layers, some different class/interface hierarchies that interact with each other, and some tests to exercise the API. The following should help explain the current major hierarchies a bit better:

  • Stats are simple structures representing an ID and a value
  • Enchantments are simple structures representing some information about modifying particular stats (slightly more complex than stats)
  • Items are more complex structures that can contain enchantments
  •  Actors are complex structures that:
    • Have collections of stats
    • Have collections of enchantments
    • Have collections of items

Okay, so that’s the high level. There’s obviously a bit more going on with the multi-layered architecture I’m trying out here too (since the hierarchies are repeated in a similar fashion in both layers). However, this is a small but reasonable amount of code to be trying this pattern on.

I have a good handful of classes and associated interfaces that back them. I’ve designed my classes to take in references to interfaces (which, are of course backed by my other classes) and my classes are largely decoupled from implementations of other classes.

Now that I’ve had some time to play with this pattern, what are my initial thoughts? Well, it’s not pure sunshine and rainbows (which I expected) but there are definitely some cool pros I hadn’t considered and definitely some negative side effects that I hadn’t considered either. Stay tuned

(The previous post in this series is here).


ProjectXyz: Why I Started a Side Project (Part 1)

ProjectXyz

Alright, I’ll admit it… Even for a placeholder name on a side project it’s pretty terrible, right? Well, my apologies. So, if you made it to this post you might be wondering what ProjectXyz is and why I started it up. From a high level, I started working on ProjectXyz so that I could have a hobby programming project to tinker with and I figured I’d blog about my adventures in bringing it all together. I plan on making this a mini-series documenting some of the things I’m learning or experimenting with, so this will serve as the intro to the series.

Before we get too far, here’s the link to the GitHub site: https://github.com/ncosentino/ProjectXyz

Why Have a Side Project?

Here’s the main thing I want to talk about in part 1 of this series: Why should you have a side project?

From my experiences in high school and university, I found that having a side project that I could code in was probably the best way that I could continue to learn. It’s a low stress activity that encourages you to be highly creative. You can involve as many other people as you want, or work on it completely solo so that you can minimize external stress and maximize your own creativity. It’s your project, so you have the freedom to do what you want.

I leveraged various small side projects to learn things like how sockets work or how I’d try and structure a multi-tier application. I was able to work on refactoring large amounts of code and learn how to use source control to my advantage. Was there a design pattern I wanted to learn about implementing? Great! Then I could try and find a way to incorporate it into my projects. The goal was always about trying to implement new things and not about cutting corners to try and deliver something to an end user.

Once I started work full-time, I gave up on my programming side projects. I started up this blog which has been fun and I try to take on as much work for my career as I can because I actually enjoy my job… But I stopped coding my own hobby projects. I’ve found that I’m missing out on two major things as a result of that:

  • The ability to experiment with patterns, technologies, and frameworks
  • The ability to get really creative and try out completely new things

Work has been a great opportunity to learn, but it’s learning out of necessity. Myself or my team will hit blocks and we have to work together to try and overcome them. It doesn’t give me the opportunity to go completely into left field to try something new out. Having a side project gives me a bit of freedom to try and learn all sorts of really neat things.

So… Why ProjectXyz?

Okay, well, if you mean why the name… Then I don’t have a great answer. I wanted to start coding but I didn’t want to waste time thinking about a name. I’ll think of something better later, I promise.

Otherwise, you might be wondering why I decided to build ProjectXyz to be what it is (or, what it will be). ProjectXyz is the back end (i.e. not the pretty graphical part) for a role playing game, and I’ve played around with this kind of thing before. I like playing games like this, and I’ve had a lot of fun trying to create a game like this in the past. As a result, I can really focus on what I’m building and not trying to figure out what to build. I’m not pouring energy into wondering “how do I solve this great big popular problem with this piece of software?”, but instead I can just get as creative as I’d like. It’s not about “What does the customer want?”, but instead, I can ask “What do I want to make?”.

The things I’m looking to try out with ProjectXyz (to start with, at least) are:

  • More LINQ usage
  • Coding by interfaces
  • API design
  • Dependency injection and IoC
  • TDD and coded test designs
  • Moq for mocking my classes
  • … GitHub! To make some publicly visible code.

As I work through ProjectXyz, I’ll write more posts on the various things I’m learning as I go! Check out the GitHub page and drop some comments!

https://github.com/ncosentino/ProjectXyz
(The second post in this series is here)


Controlling a Myo Armband with C#

Controlling a Myo Armband with C#

Background

Thalmic Labs has started shipping their Myo armband that allows the wearer’s arm movements and gestures to control different pieces of integrated technology. How cool is that? My friend and I decided we wanted to give one a whirl and see what we could come up with. We’re both C# advocates, so we were a bit taken back when we saw the only C# support in the SDK was made for Unity. We decided to take things into our own hands and open source a Myo C# library. We’re excited to introduce the first version of MyoSharp!

The underlying Myo components are written in C++, and there’s only several functions that are exposed from the library that we can access. In order to do this, we need to leverage platform invocation (PInvokes) from C# to tap into this functionality. Once you have the PInvokes set up you can begin to play around!

The Workflow

Getting setup with the Myo is pretty straightforward, but it wasn’t obvious to us right away. We didn’t have anyone to walk us through how the different components were supposed to work together (just some good ol’ fashioned code) so we had to tinker around. Once we had everything mapped out, it was quite simple though.

  1. The first step is opening a communication channel with the Bluetooth module. You don’t need to worry about the implementation here since it’s all done in C++ by the Thalmic devs. Calling the correct methods using PInvokes from C# allows us to tap into a stream of “events” that come through the Bluetooth module.
  2. Now that we can intercept events, we need to be able to identify a Myo. After all, working with Myos is our main goal here! There’s a “pair” event that we can listen to from the Bluetooth module that notifies us of when a Myo has paired and provides us a handle to the device. This handle gets used for identifying events for a particular Myo or sending a particular Myo messages.
  3. There’s a connect event that will fire when a Myo connects after it’s been paired with the Bluetooth module. A Myo can be paired but disconnected.
  4. Now that we can uniquely identify a Myo, the only things we need to do are intercept events for a particular Myo and make sense of the data coming from the devices! Orientation change? Acceleration change? There’s a host of information that the device sends back, so we need to interpret it.
  5. When a Myo disconnects, there’s an event that’s sent back for that as well.

Getting Started with MyoSharp

I’m going to start this off with some simple code that should illustrate just how easy it is to get started with MyoSharp. I’ll describe what’s going on in the code immediately after.


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you the basics for setting up and working with
    /// a Myo using MyoSharp. Primary communication with the device happens
    /// over Bluetooth, but this C# wrapper hooks into the unmanaged Myo SDK to
    /// listen on their "hub". The unmanaged hub feeds us information about
    /// events, so a channel within MyoSharp is responsible for publishing
    /// these events for other C# code to consume. A device listener uses a
    /// channel to listen for pairing events. When a Myo pairs up, a device
    /// listener publishes events for others to listen to. Once we have access
    /// to a channel and a Myo handle (from something like a Pair event), we
    /// can create our own Myo object. With a Myo object, we can do things like
    /// cause it to vibrate or monitor for poses changes.
    /// </summary>
    internal class BasicSetupExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.Vibrate(VibrationType.Short);
                    e.Myo.PoseChanged += Myo_PoseChanged;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.PoseChanged -= Myo_PoseChanged;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_PoseChanged(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo detected {1} pose!", e.Myo.Arm, e.Myo.Pose);
        }
        #endregion
    }
}

In this example, we create a hub instance. A hub will manage a collection of Myos that come online and go offline and notify listeners that are interested. Behind the scenes, the hub creates a channel instance and passes this into a device listener instance. The channel and device listener combination allows for being notified when devices come online and is the core of the hub implementation. You can manage Myos on your own by completely bypassing the Hub class and creating your own channel and device listener if you’d like. It’s totally up to you.

In the code above, we’ve hooked up several event handlers. There’s an event handler to listen for when Myo devices connect, and a similar one for when the devices disconnect. We’ve also hooked up to an instance of a Myo device for when it changes poses. This will simply give us a console message every time the hardware determines that the user is making a different pose.

When devices go offline, the hub actually keeps the instance of the Myo object around. This means that if you have device A and you hook up to it’s PoseChanged event, if it goes offline and comes back online several times, your event will still be hooked up to the object that represents device A. This makes managing Myos much easier compared to trying to re-hook event handlers every time a device goes on and offline. Of course, you’re free to make your own implementation using our building blocks, so there’s no reason to feel forced into this paradigm.

It’s worth mentioning that the UserInputLoop() method is only used to keep the program alive. The sample code on GitHub actually lets you use some debug commands to read some Myo statuses if you’re interested. Otherwise, you could just imagine this line is replaced by Console.ReadLine() to block waiting for the user to press enter.

Pose Sequences

Without even diving into the accelerometer, orientation, and gyroscope readings, we were looking for some quick wins to building up on the basic API that we created. One little improvement we wanted to make was the concept of pose sequences. The Myo will send events when a pose changes, but if you were interested in grouping some of these together there’s no way to do this out of the box. With a pose sequence, you can declare a series of poses and get an event triggered when the user has finished the sequence.

Here’s an example:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user has performed a series of poses. A 
    /// <see cref="PoseSequence"/> can monitor a Myo for a series of poses and
    /// notify you when that sequence has completed.
    /// </summary>
    internal class PoseSequenceExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // for every Myo that connects, listen for special sequences
                    var sequence = PoseSequence.Create(
                        e.Myo, 
                        Pose.WaveOut, 
                        Pose.WaveIn);
                    sequence.PoseSequenceCompleted += Sequence_PoseSequenceCompleted;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Sequence_PoseSequenceCompleted(object sender, PoseSequenceEventArgs e)
        {
            Console.WriteLine("{0} arm Myo has performed a pose sequence!", e.Myo.Arm);
            e.Myo.Vibrate(VibrationType.Medium);
        }
        #endregion
    }
}

The same basic setup occurs as the first example. We create a hub that listens for Myos, and when one connects, we hook a new PoseSequence instance to it. If you recall how the hub class works from the first example, this will hook up a new pose sequence each time the Myo connects (which, in this case, isn’t actually ideal). Just for demonstration purposes, we were opting for this shortcut though.

When creating a pose sequence, we only need to provide the Myo and the poses that create the sequence. In this example, a user will need to wave their hand out and then back in for the pose sequence to complete. There’s an event provided that will fire when the sequence has completed. If the user waves out and in several times, the event will fire for each time the sequence is completed. You’ll also notice in our event handler we actually send a vibrate command to the Myo! Most of the Myo interactions are reading values from Myo events, but in this case this is one of the commands we can actually send to it.

Held Poses

The event stream from the Myo device only sends events for poses when the device detects a change. When we were trying to make a test application with our initial API, we were getting frustrated with the fact that there was no way to trigger some action as long as a pose was being held. Some actions like zooming, panning, or adjusting levels for something are best suited to be linked to a pose being held by the user. Otherwise, if you wanted to make an application that would zoom in when the user makes a fist, the user would have to make a fist, relax, make a fist, relax, etc… until they zoomed in or out far enough. This obviously makes for poor usability, so we set out to make this an easy part of our API.

The code below has a similar setup to the previous examples, but introduces the HeldPose class:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user is still holding a pose and not just that they've 
    /// transitioned from one pose to another. The <see cref="HeldPose"/> class
    /// monitors a Myo and notifies you as long as a particular pose is held.
    /// </summary>
    internal class HeldPoseExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // setup for the pose we want to watch for
                    var pose = HeldPose.Create(e.Myo, Pose.Fist, Pose.FingersSpread);

                    // set the interval for the event to be fired as long as 
                    // the pose is held by the user
                    pose.Interval = TimeSpan.FromSeconds(0.5);

                    pose.Start();
                    pose.Triggered += Pose_Triggered;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Pose_Triggered(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo is holding pose {1}!", e.Myo.Arm, e.Pose);
        }
        #endregion
    }
}

When we create a HeldPose instance, we can pass in one or more poses that we want to monitor for being held. In the above example, we’re watching for when the user makes a fist or when they have their fingers spread. We can hook up to the Triggered event on the held pose instance, and the event arguments that we get in our event handler will tell us which pose the event is actually being triggered for.

If you take my zoom example that I started describing earlier, we could have a single event handler responsible for both zooming in and zooming out based on a pose being held. If we picked two poses, say fist and fingers spread, to mean zoom in and zoom out respectively, then we could check the pose on the event arguments in the event handler and adjust the zoom accordingly. Of course, you could always make two HeldPose instances (one for each pose) and hook up to the events separately if you’d like. This would end up creating two timer threads behind the scenes–one for each HeldPose instance.

The HeldPose class also has an interval setting. This allows the programmer to adjust the frequency that they want the Triggered event to fire, provided that a pose is being held by the user. For example, if the interval is set to be two seconds, as long as the pose is being held the Triggered event will fire every two seconds.

Roll, Pitch, and Yaw

The data that comes off the Myo can become overwhelming unless you’re well versed in vector math and trigonometry. Something that we’d like to build up and improve upon is the usability of data that comes off the Myo. We don’t want each programmer to have to write similar code to get the values from the Myo into a usable form for their application. Instead, if we can build that into MyoSharp, then everyone will benefit.

Roll, pitch, and yaw are values that we decided to bake into the API directly. So… what exactly are these things? Here’s a diagram to help illustrate:

Roll, Pitch, and Yaw - MyoSharp

Roll, pitch, and yaw describe rotation around one of three axes in 3D space.

The following code example shows hooking up to an event handler to get the roll, pitch, and yaw data:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you how to hook onto the orientation events on
    /// the Myo and pull roll, pitch and yaw values from it.
    /// </summary>
    internal class OrientationExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.OrientationDataAcquired += Myo_OrientationDataAcquired;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.OrientationDataAcquired -= Myo_OrientationDataAcquired;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_OrientationDataAcquired(object sender, OrientationDataEventArgs e)
        {
            Console.Clear();
            Console.WriteLine(@"Roll: {0}", e.Roll);
            Console.WriteLine(@"Pitch: {0}", e.Pitch);
            Console.WriteLine(@"Yaw: {0}", e.Yaw);
        }
        #endregion
    }
}

Of course, if we know of more common use cases that people will be using the orientation data for, then we’d love to bake this kind of stuff right into MyoSharp to make it easier for everyone.

Closing Comments

That’s just a quick look at how you can leverage MyoSharp to make your own C# application to work with a Myo! As I said, MyoSharp is open source so we’d love to see contributions or ideas for suggestions. We’re aiming to provide as much base functionality as we can into our framework but designing it in a way that developers can extend upon each of the individual building blocks.


Hack The North

Hack The North

Hack The North… What The Hack Is That?

Hack The North is Canada’s largest international hackathon. It’s big. It’s bad. It’s awesome. Okay, but what does that even mean?

The idea is that 1000 people get together from all over the world in an event where they’re given 36 hours to create amazing technology hacks. There’s a lack of sleep but no lack of amazing ideas and hacked together proof of concepts that show incredible innovation. Sponsors are present to hand out prizes for best usage of their product or API to competitors as well as mentor them and provide help with problem solving. This year, Hack The North was hosted at The University of Waterloo.

Bro, Do You Even Hack?!

This was my first time ever at a hackathon. I’ve participated in the Ontario Engineering Competition (OEC) which is much smaller scale event with a similar structure–A bunch of students get together and have to come up with a design to solve a problem. I even ran OEC in 2010 with some schoolmates (you might know them at Thalmic Labs) which was a blast. However, this weekend I was sitting on the other side of the table.

I got to hang out with a group of my colleagues at Hack The North at the Magnet Forensics booth. We had a great time representing our company and meeting tons of incredible students from all over the world. We provided assistance where we could with some of the challenges that students were facing, and spoke to them about the important role that our software plays in the life of a forensic examiner/investigator.

What’d You See There?

When I first showed up at Hack The North, it was around 9:00 on Friday night. I got there just in time for the opening ceremonies, which I thought were great. They had live Q&A with a former Facebook employee who has “made it” in the software industry and intros from some of the major sponsors at the event.

Hack The North - Opening Ceremonies

Everyone getting settled for the opening ceremonies at Hack The North.

From that point on, I spent the rest of the time hanging around the Magnet booth… But that doesn’t mean I didn’t see anything awesome. We had people from all over the place coming by to talk to us and pick up some Magnet swag. We handed out an incredible amount of stress balls and couldn’t seem to keep a sufficient supply of them coming. We had our stickers showing up on everything from fuzzy viking hats to laptops to shirts to megaphones. You name it, our stickers made it onto it.

Hack The North - Magnet Viking

Akshay Joshi decorated his hat with Magnet stickers!

Hack The North - Magnet Megaphone

One of the Hack The North organizers was sporting a Magnet sticker on his megaphone.

On the last day of Hack The North, we had participants coming up to our booth to demonstrate some of their awesome hacks. We got to see how Thalmic’s Myo was being incorporated or how other vendors’ APIs were being leveraged to do some really awesome things. We were really impressed with some of the things we saw.

What’s Next?

I think even after the first night of being at Hack The North I was trying to think of what we could do next time or if we even wanted to come back… The answer to the latter was quickly an “absolutely yes!”, so I’m pretty confident we’ll be making an appearance at Hack The North again. There were so many great people at this even that we spoke with that it would be silly not to go back.

I think next year we’d like to participate even more. We learned a lot about the different ways that we could get involved, so things like speaking sessions or workshops would be awesome to get people involved with. We’ll definitely have more stress balls and unique giveaways to please the masses!

Thanks to everyone who made it out to Hack The North. Stay in touch with us! We’d love to hear more from you.

Hack The North - Kelly and Nick

Kelly and I with our #Truth T-shirts and other Magnet gear on at our booth!


Refactoring For Interfaces: An Adventure From The Trenches

puzzle

Refactoring: Some Background

If you’re a seasoned programmer you know all about refactoring. If you’re relatively new to programming, you probably have heard of refactoring but don’t have that much experience actually doing it. After all, it’s easier to just rewrite things from scratch instead of trying to make a huge design change part way through, right? In any mature software project, it’s often the case where you’ll get to a point where your code base in its current state cannot properly sustain large changes going forward. It’s not really anyone’s fault–it’s totally natural. It’s impossible to plan absolutely everything that comes up, so it’s probable that at some point at least part of your software project will face refactoring.

In my real life example, I was tasked with refactoring a software project that has a single owner. I’m close with the owner and they’re a very technical person, but they’re also not a programmer. Because I’m not physically near the owner (and I have a full-time job, among other things I’m doing) it’s often difficult to debug any problems that come up. The owner can’t simply open an editor and get down to the code to fix things up.

So there was an obvious solution which I avoided in the first place… Unit tests. Duh. I need unit tests. So that’s an easy solution right? I’ll bust out my favourite testing framework (I’m a fan of xUnit) and start getting some solid code coverage. Well… in an ideal world, like every programming article is ever written, this would have been the case. But that wasn’t the case. My software project does a lot of direct HTTP/FTP requests and interacts with particular hardware on the machine. How awesome is writing a unit test that contacts an HTTP server? Not very awesome.

What was I to do? I need to be able to write unit tests so that I can validate my software before putting it in my customer’s hands, but I can’t test it with unit tests because I don’t have the hardware!

Refactoring for Interfaces

Okay, so the first step in my master plan is to refactor for interfaces. What do I mean by that? Well, I have a lot of code that will call out and make HTTP requests and it has a specific dependency on System.Net.WebRequest. The same thing holds true for my FTP requests I want to make. Because I have that dependency within my classes, it means I have no choice but to call out to the network and go do these things.

I could design it a different way though. What if I abstracted the web requests away so that I didn’t actually have to call that class directly? What if I could have a reference to some instance that met some API that would just do that stuff for me? I mean, my class knows all about it’s on particular job, but it truthfully doesn’t know the first thing about calling out to the Internet to go post some HTTP requests. This means if someone else is responsible for providing me with a mechanism for giving me the ability to post HTTP requests, this other entity could also fool me and not actually send out HTTP requests at all! That sounds like exactly what my test framework would want to do.

My first step was to look at the properties and methods I was using on the WebRequest class. What was shared between the HTTP and FTP requests that I was creating? The few things I had to consider were:

  • Some sort of Send() method to actually send the request
  • A URI to identify where the request is being sent
  • A timeout property

I then created an interface for a web request that had these properties/methods accessible and created some wrapper classes that implemented this interface but encapsulated the functionality of the underlying web requests. The next step was creating a class and interface for a “factory” that could create these requests. This is because my code that needs to make HTTP/FTP requests only knows it needs to make those requests–It doesn’t have any knowledge on how to actually create one.

With my interfaces for my requests and my factory that creates them, I was able to move onto the next step.

Create Mocks for the Interfaces

Now that I had my classes leveraging interfaces instead of concrete classes internally, I could mock the inner-workings of my classes. This would provide two major benefits:

  • I could create tests that wouldn’t have to actually go out to the Internet/network.
  • I could create instrumented mocks that would let me test whether certain web requests were being made.

I started off my writing up some unit tests. I tried to get as much code coverage as I could by doing simple tests (i.e. create an object, check default values, call a method and check a result, etc…). Once I had exhausted a lot of the simple stuff, I targeted the other areas that I wasn’t hitting. I mean, how was my coded test supposed to test my method that does an HTTP request and an FTP of a file under the hood? Mocks.

So this is where you probably draw the line between your integration tests or unit tests and get all pedantic about it. But I don’t care how you want to separate it: I need coded tests that cover a section of my class so that I can ensure it behaves as I expect. But if I’m mocking my dependencies, how do I know my class is actually doing what I expect?

Instrument your mocks! This was totally cool for me to play around with for the first time. I had to create dummy FTP/HTTP requests that met the interface my class under test expected. Pretty easy. But I could actually assert what requests my class under test was actually trying to send out! This meant that if my method was supposed to try and hit a certain URL, I could assert that easily by instrumenting my mocked instance to check just that. Was it supposed to FTP a certain set of bytes? No problem. Use my mocked instance to assert those bytes are actually the ones my class under test is trying to send.

Wrap Up

This was just a general post, and I didn’t put up any code to go along with it. Sorry. I really just wanted to cover my experience with refactoring, interfaces, mocking, and code coverage because it was a great learning for me.

To recap on what I said in this post:

  • Identify the parts you want to mock. These are the things your class or method probably isn’t responsible for creating directly. Going out to the network? Accessing the disk? Accessing the environment your test is running under? Creating complex concrete classes because they hook into some other system for you? Great candidates for this.
  • Create interfaces by looking at the API you’re accessing. You know what classes you want to mock, so look how you’re using the API. If you need to access a few properties and methods, then make that part of your interface. If you see commonality between a few similar things, you might be able to create a single interface to handle all of the scenarios.
  • Inject factories that can create instances for you. These factories know how to create the concrete classes that meet your interfaces. In a real situation, they can create the classes you expect. In a test environment, they can create your mocks.
  • Write coded tests with your mocks! The last part is the most fun. You can finally inject some mocked classes into your classes/methods under test and then instrument them to ensure your code under test is accessing them in the way you expect. Run some code coverage tools after to prove you’re doing a good job.

I hope my experiences down this path are able to help you out!


IronPython: A Quick WinForms Introduction

IronPython: A Quick WinForms Introduction

Background

A few months ago I wrote up an article on using PyTools, Visual Studio, and Python all together. I received some much appreciated positive feedback for it, but really for me it was about exploring. I had dabbled with Python a few years back and hadn’t really touched it much since. I spend the bulk of my programming time in Visual Studio, so it was a great opportunity to try and bridge that gap.

I had an individual contact me via the Dev Leader Facebook group that had come across my original article. However, he wanted a little bit more out of it. Since I had my initial exploring out of the way, I figured it was probably worth trying to come up with a semi-useful example. I could get two birds with one stone here–Help out at least one person, and get another blog post written up!

The request was really around taking the output from a Python script and being able to display it in a WinForm application. I took it one step further and created an application that either lets you choose a Python script from your file system or let you type in a basic script directly on the form. There isn’t any fancy editor tools on the form, but someone could easily take this application and extend it into a little Python editor if they wanted to.

Leveraging IronPython

In my original PyTools article, I mention how to get IronPython installed into your Visual Studio project. In Visual Studio 2012 (and likely a very similar approach for other versions of Visual Studio), the following steps should get you setup with IronPython in your project:

  • Open an existing project or start a new one.
  • Make sure your project is set to be at least .NET 4.0
    • Right click on the project within your solution explorer and select “Properties”
    • Switch to the “Application” tab.
    • Under “Target framework”, select  “.NET Framework 4.0”.
  • Right click on the project within your solution explorer and select “Manage NuGet Packages…”.
  • In the “Search Online” text field on the top right, search for “IronPython”.
  • Select “IronPython” from within the search results and press the “Install” button.
  • Follow the instructions, and you should be good to go!

Now that we have IronPython in a project, we’ll need to actually look at some code that gets us up and running with executing Python code from within C#. If you followed my original post, you’ll know that it’s pretty simple:


var py = Python.CreateEngine();
py.Execute("your python code here");

And there you have it. If it seems easy, that’s because it is. But what about the part about getting the output from Python? What if I wanted to print something to the console in Python and see what it spits out? After all, that’s the goal I was setting out to accomplish with this article. If you try the following code, you’ll notice you see a whole lot of nothing:


var py = Python.CreateEngine();
py.Execute("print('I wish I could see this in the console...')");

What gives? How are we supposed to see the output from IronPython? Well, it all has to do with setting the output Stream of the IronPython engine. It has a nice little method for letting you specify what stream to output to:


var py = Python.CreateEngine();
py.Runtime.IO.SetOutput(yourStreamInstanceHere);

In this example, I wanted to output the stream directly into my own TextBox. To accomplish this, I wrote up my own little stream wrapper that takes in a TextBox and appends the stream contents directly to the Text property of the TextBox. Here’s what my stream implementation looks like:


private class ScriptOutputStream : Stream
{
  #region Fields
  private readonly TextBox _control;
  #endregion

  #region Constructors
  public ScriptOutputStream(TextBox control)
  {
    _control = control;
  }
  #endregion

  #region Properties
  public override bool CanRead
  {
    get { return false; }
  }

  public override bool CanSeek
  {
    get { return false; }
  }

  public override bool CanWrite
  {
    get { return true; }
  }

  public override long Length
  {
    get { throw new NotImplementedException(); }
  }

  public override long Position
  {
    get { throw new NotImplementedException(); }
    set { throw new NotImplementedException(); }
  }
  #endregion

  #region Exposed Members
  public override void Flush()
  {
  }

  public override int Read(byte[] buffer, int offset, int count)
  {
    throw new NotImplementedException();
  }

  public override long Seek(long offset, SeekOrigin origin)
  {
    throw new NotImplementedException();
  }

  public override void SetLength(long value)
  {
    throw new NotImplementedException();
  }

  public override void Write(byte[] buffer, int offset, int count)
  {
    _control.Text += Encoding.GetEncoding(1252).GetString(buffer, offset, count);
  }
  #endregion
}

Now while this isn’t pretty, it serves one purpose: Use the stream API to allow binary data to be appended to a TextBox. The magic is happening inside of the Write() method where I take the binary data that IronPython will be providing to us, convert it to a string via code page 1252 encoding, and then append that directly to the control’s Text property. In order to use this, we just need to set it up on our IronPython engine:


var py = Python.CreateEngine();
py.Runtime.IO.SetOutput(new ScriptOutputStream(txtYourTextBoxInstance), Encoding.GetEncoding(1252));

Now, any time you output to the console in IronPython you’ll get your console output directly in your TextBox! The ScriptOutputStream implementation and calling SetOutput() are really the key points in getting output from IronPython.

The Application at a Glance

I wanted to take this example a little bit further than the initial request. I didn’t just want to show that I could take the IronPython output and put it in a form control, I wanted to demonstrate being able to pick the Python code to run too!

Firstly, you’re able to browse for Python scripts using the default radio button. Just type in the path to your script or use the browse button:

IronPython - Run script from file

Enter a path or browse for your script. Press “Run Script” to see the output of your script in the bottom TextBox.

Next, press “Run Script”, and you’re off! This simply uses a StreamReader to get the contents of the file and then once in the contents are stored in a string, they are passed into the IronPython engine’s Execute() method. As you might have guessed, my “helloworld.py” script just contains a single line that prints out “Hello, World!”. Nothing too fancy in there!

Let’s try running a script that we type into the input TextBox instead. There’s some basic error handling so if your script doesn’t execute, I’ll print out the exception and the stack trace to go along with it. In this case, I tried executing a Python script that was just “asd”. Clearly, this is invalid and shouldn’t run:

python_error_asd

Python interpreted the input we provided but, as expected, could not find a definition for “asd”.

That should be along the lines of what we expected–The script isn’t valid, and IronPython tells us why. What other errors can we see? Well, the IronPython engine will also let you know if you have bad syntax:

python_error_bad_syntax

Python interpreted the script, but found a syntax error in our silly input.

Finally, if we want to see some working Python we can do some console printing. Let’s try a little HelloWorld-esque script:

python_pass_hello_world

Python interpreted our simple Hello World script.

Summary

This sample was pretty short but that just demonstrates how easy it is! Passing in a script from C# into the IronPython is straight forward, but getting the output from IronPython is a bit trickier. If you’re not familiar with the different parts of the IronPython engine, it can be difficult to find the things you need to get this working. With a simple custom stream implementation we’re able to get the output from IronPython easily. All we had to do was create our own stream implementation and pass it into the SetOutput() method that’s available via the IronPython engine class. Now we can easily hook the output of our Python scripts!

As always, all of the source for you to try this out is available online:

Some next steps might include:

  • Creating your own Python IDE. Figure out some nice text-editing features and you can run Python scripts right from your application.
  • Creating a test script dashboard. Do you write test scripts for other applications in Python? Why not have a dashboard that can report on the results of these scripts?
  • Add in some game scripting! Sure, you could have done this with IronPython alone, but maybe now you can skip the WinForms part of this and just make your own stream wrapper for getting script output. Cook up some simple scripts in a scripting engine and voila! You can easily pass information into Python and get the results back out.

Let me know in the comments if you come up with some other cool ideas for how you can leverage this!


Article Dump #24 – Weekly Article Dump

Article Dump #24 - Dev Leader (Image by http://www.sxc.hu)

Article Dump #24

Welcome to the 24th issue of my (nearly) weekly article dumps. I don’t have a theme or an update this week, so it’s kept pretty short. I hope you find the following articles interesting though! Leave me a comment if you have any opinions on these

Articles

  • The 7 Values That Drive IDEO: In this article, the CEO of IDEO Tim Brown talks about the various values that his organization embraces to have a creative culture. Some of the ideas in the slides seem really high level or like generic fluff, but try thinking about what they would mean in your organization. It’s one thing to glance at IDEO’s list and say “Yeah, yeah… That’s nice…” but when you actually think about how that fits in with your organization, you might actually realize you don’t embody those values. Do you learn from failure? Does your organization promote an ask for forgiveness not permission approach? Would this make sense in your organization? Just some food for thought, but I thought a lot of these values were interesting to think about and how embracing them might change the organization I work in.
  • The 15 Most Annoying Coworkers of All Time: Ilya Pozin put together a pretty funny article on different types of coworkers you’ll encounter in your career. I got worried that I might be #13 on the list… The office comedian who isn’t actually funny. Apparently this post got a lot of flack in the comments on LinkedIn. I guess people were expecting a really serious article on how to deal with these different types of problems in the workplace. I didn’t really have expectations when I read it, aside from not wanting to find myself on the list. Maybe the main take away point here is… don’t annoy your colleagues!
  • Companies Frustrate Innovative Employees: Gijs van Wulfen takes a different perspective on innovation. So many people now are writing about embracing failure (so far as you learn from it). I’m actually a big believer in that approach–take controlled risks and learn from things that don’t go as expected. Gijs’ perspective is a little bit different: forget embracing failure; boost the innovation effectiveness rate! Gijs goes through a workflow for trying to improve innovation at various steps in the process. Pretty interesting!
  • Your Boss is Happier Than You (But Shouldn’t Be): Jeff Haden tells us something we probably all (let’s say in the majority of circumstances) know: your boss is happier than you. Big surprise right? They get to make decisions, have fewer bosses than you, and they make more money. Sounds like a good reason to be happier, no? But if your boss is happier than you, those probably aren’t the exact reasons. Your superiors are likely happier than you because of autonomy. They get a bit more freedom to do accomplish goals in their own way. Jeff has a big list of reasons why your boss is probably happier… and none of them are about money.
  • When is it a Good Idea to write Bad Code?: Rejoice in the first programming article for this week! Tech debt. Ever heard of it? If not, it’s not likely that you’ve never encountered it in your programming career. I’d wager at least one of the last handful of big features you implemented in your code base either had to deal with some tech debt or perhaps even introduced some tech debt. Brad Carleton has put together a big list of different types of tech debt and what they mean in your project. I highly suggest you read it if your a programmer. There’s a lot of things to be aware of with tech debt but it’s important to remember that tech debt isn’t always the worst thing that could happen. Sometimes it’s okay to sacrifice a sub-par design now in order to get some software out the door. Your users might try it out and decide they don’t like the functionality anyway, and you’d end up re-writing it again!
  • “Happiness” vs “Meaningfulness” — The Surprising DifferenceAlex Banayan‘s article discusses the difference between happiness and meaningfulness. It appears as though often happiness and meaningfulness are not necessarily aligned. For example, it might be easy to chase a life of happiness that lacks meaning, or dedicate your life to something meaningful but not be very happy while doing it. The real question is, is it possible to achieve a balance where you’re leading a fulfilling life that keeps you happy? Alex talks briefly about five different categories and how each can sway to something more meaningful or something that provides more happiness. Are you living a happy and fulfilling life? Do you have to balance these five categories carefully?

Follow Dev Leader on social media outlets to get these updates through the week. Thanks!


Be a Better Programmer – Weekly Article Dump

Be a Better Programmer - Weekly Article Dump (Image by http://www.sxc.hu/)

Be a Better Programmer

It’s a new year and that means it’s all about resolutions, right? Well, I’m not a huge fan of keeping around a resolution that needs to wait for a new year, but I am a fan of reflecting on your goals and your skills. If you’re a programmer like me, then maybe this will be a great starting point. In my weekly article dumps I usually would just provide a couple of comments on a link like this, but I felt I should dive in a little bit more. You can find the original article by Amy Jollymore over here. Please have a look! I shared it with the whole dev team at Magnet Forensics because I felt there was a little bit of something for everyone.

Number one on this list, and perhaps the one I’d personally like to focus on more out of this list, is checking your code before blaming others. Blaming other people–in general, not just programming–is an easy way out. When a problem occurs, it’s simple to assume that all of your work is right and that it must be someone else’s fault. But if everyone starts thinking like this, it turns into a nasty blame war. So next time the build breaks or your shiny new feature stops working as expected, don’t go blaming other people. Investigate what the problem is. See what your most recent changes were and if they could have caused the problem. As you start to gain confidence that your changes aren’t responsible for the issue, try sitting down with one or two other people you think might have been around the problem area recently–But don’t go accusing them! Putting your heads together to figure out the problem can speed up the process and might even shed some light on some miscommunication over a design or some assumptions in the code that don’t actually hold true. It’s a lot more embarrassing to blame someone when it’s actually your fault compared to putting in the effort and admitting you might have goofed up. Try it out!

Number two is also a great item. You should never put an end to your learning… especially as an individual in a technology space. There are so many great suggestions listed for this point that there’s no point in me repeating them. Just go read them! An interesting point worth mentioning is using podcasts for learning. This is a great option if you find you’re brain is still spinning when you lay down in bed or if you have a long commute to work (or something else you’re involved in). The author also mentions that you don’t need to be learning programming… What about domain expertise? If you’re writing code for banks, lawyers, or digital forensics… Why  not learn about that too?!

The last point I’ll touch on from the article is number three: don’t be afraid to break things. I love this point. If you’re working on a big piece of software, there are almost certainly areas that seem brittle, scary, or just plain incomprehensible. If your project is still small, it very well get to this point. It doesn’t mean that the code is bad or that you’re working with the worst programmers… It’s just something that happens when you’re continuously trying to build on your software. The real problem occurs when nobody is willing to take the time to go change things. If you have big scary brittle parts of code, then set aside some time, take a deep breath, and go refactor it! It might seem like hell at first, but once you get into it (and especially after it’s done) you’ll feel a million times better. Plus, now your code can continue to be built upon without people running in fear when you mention that section of code. Code can get nasty, but consider using a “tech debt” system or regularly set aside time for refactoring parts of your code base.

Again, the original article is located at: 7 Ways to be a Better Programmer in 2014. Check it out!

Articles

  • How to Manage Dynamic Tensions — and Master the Balancing Act: This was an interesting article on some parts of leadership that often oppose each other. Author Chris Cancialosi does an excellent job in discussing balance between internal and external influences as well as leading and managing. A good take away from this article is at least acknowledging that there are certainly some things to balance. You may want to have the most flexible team, but have you considered if there’s a “too flexible”? Just a bit of perspective that this article might bring to light.
  • A Crash Course In Leadership For 20-Something CEOs: Barry Salzberg‘s article is geared toward young CEOs, but I think that means we can apply the lessons to anyone looking to lead! A few of the points I’d like to mention include being tough on problems and not on people. Your people are the one’s who are going to solve problems and bring great ideas to the table. They’ll invest their time into your organization in order to accomplish great things–so don’t be hard on them. Instead, acknowledge that your problems and challenges are the things you want to crush, and work with your team to make sure you conquer every challenge that gets in the way of your goal. Another point is on taking risks. Never taking risks is a great way to stagnate. You need to learn from your failures, but keep pushing the boundaries. Finally, be ready to adapt. As your organization grows or as the market you’re working within evolves, you need to be ready to adapt and change. You might get lucky and things don’t change all that much over a long period of time, but the odds of that are pretty low. Be ready to adapt so when the time comes, you don’t need to worry about everything falling apart.
  • Leading at Scale with Agility: Brad Smith has a few great points on what leading a team should encompass. First, a team should have a goal that it is trying to achieve. If that team is part of a larger organization, the team’s goal should align with the goal of the entire organization. Secondly, decisions for the team should involve those on the team. It’s easy to sit back and speculate what might be best, but why not involve the people directly affected? Of course, this is more difficult for large teams but maybe that’s an indication your teams would be more effective if they were smaller. Next, empower teams to arrive at solutions on their own. If a plan worked out well, try communicating it to others to try out. Conversely, if the plan had some problems, let others on the team (or other teams) know about the hurdles. Finally, Brad has a point on trust. Trust is arguably one of the most important parts of leading a team. Each team member needs to be able to trust the others. There should be an easy assumption that everyone is operating with best intentions.
  • For Leaders, Today is History: In this article by Steven Thompson, he gives a high-level overview of his focus. Specifically, he focuses on the future and not right now. Steven says the teams he is in charge of are often looking at the problems of “right now” and perhaps a little bit in the future. It would be counter productive for him to try and butt-in to try helping with those problems because he’s so far removed from them. Instead, those individuals have been empowered to focus on those problems. Instead, Steven focuses on the future–the direction of the teams. As a leader, it’s important to try and be thinking at least one step ahead.
  • What If You Had to Write a “User Manual” About Your Leadership Style?: After I read Adam Bryant‘s article, I thought the idea of a leadership “user manual” would be pretty cool. Even if there isn’t a single other individual who would benefit from it, at least it would help reveal to myself some of my leadership quirks. That’s useful on it’s own! I’ll be sure to post up my leadership “user manual” when I have it complete… and I imagine I’ll have to keep updating it over time as my style evolves. It’ll be really interesting to see the evolution of my leadership style! Why not consider doing one for yourself?
  • What Bosses Should Never Ask Employees to Do: Jeff Haden‘s article was a little bit controversial in my opinion–and in the opinion of some of the commenters. I think I get the underlying message behind a lot of what Jeff is saying for each of his points, but as one commenter said, it sounds like a bit of a personal complaint the whole way through. Consider the topic of donating to charities at work. The feel I get after reading that segment is that your organization should not attempt to do fundraising through employees. While I don’t actually think that’s what Jeff is saying, that’s how I feel after reading it. I know that we’ve been able to do several charity events at Magnet, and we’ve always said that they are completely voluntary. I think that’s the crucial part. It’s the holiday season and your budget is a bit tight? How could anyone get mad at you for backing out of a completely optional charity donation? Busy with some personal matters or want to focus on finishing up something at work the day we’re doing a charity event? No big deal, it’s optional. Anyway, the point is that perhaps based on the wording in the article, I felt like some of the messaging will be misinterpreted. I think there are some good points buried in there. Check it out and let me know if you agree or not!

Follow Dev Leader on social media outlets to get these updates through the week. Thanks!


Yield! Reconsidering APIs with Collections

Yield! Reconsidering APIs with Collections (Image by http://www.sxc.hu/)

Yield: A Little Background

The yield keyword in C# is pretty cool. Being used within an iterator, yield lets a function return an item as well as control of execution to the caller and upon next iteration resume where it left off. Neat, right? MSDN documentation lists these limitations surrounding the use of the yield keyword:

  • Unsafe blocks are not allowed.
  • Parameters to the method, operator, or accessor cannot be ref or out.
  • A yield return statement cannot be located anywhere inside a try-catch block. It can be located in a try block if the try block is followed by a finally block.
  • A yield break statement may be located in a try block or a catch block but not a finally block.

So what does this have to do with API specifications?

A whole lot really, especially if you’re dealing with collections. I personally haven’t been a big user of the yield keyword, but I’ve never really been forced to use it. After playing around with it for a bit, I saw a lot of potential. I’ve written before about what I think makes a good API. In my article, I was making a point to discuss two perspectives:

  • Who needs to implement your interface. You want it to be easy for them to implement.
  • Who needs to call your interface. You want it to be easy for them to use.

In my opinion, the IEnumerable<T> interface was a tricky thing to work with as a return value. You can essentially only iterate an IEnumerable, and at the time of calling a function, maybe that’s not what you want to do. The flip side is that for the person implementing the interface, IEnumerable<T> is a really easy interface to satisfy. However, the yield keyword has opened up some new doors.

In this article, I’d like to go over a couple of different approaches for an API and then explain why the yield keyword might be something you consider next time around. Disclaimer: I’m not claiming anything I’m about to present is the only way or the best way–I’m just sharing some of my own findings and perspective.

Interface For Returning Collections

The first type of API I’d like to look at is for returning collections. Based on my own API guidelines, I’d ideally choose an interface or class to return that provides a lot of information to the caller that is also easy to create for the implementer of my interface. The List<T> class is a great choice:

  • It’s easy to construct
  • It’s built-in to the .NET framework
  • It provides many handy functions (All of the IList<T> functionality as well as things like AddRange(), or functions that support delegates)

My next choice might be to have a return type of IList<T>, which would provide a little less ease of use to the caller, but make it even easier for the implementer of the interface. They could return arrays of type T, since an array implements the IList<T> interface, or their own custom list implementation that doesn’t inherit from the List<T> class. The differences between using IList<T> and List<T> are arguable pretty small.

A third alternative, which I would have avoided in the past, is to return an IEnumerable<T>. My opinion used to be that this made the life of the interface implementer a bit easier compared to returning an IList<T>, but complicated the life of the caller for a couple of reasons:

  • The caller would have to use the results of the function in a foreach loop.
  • The caller would have to add the items to their own collection to be able to do much more with the items.

My naive implementations of being forced to return an IEnumerable<T> were… well… crap. I would have constructed a collection within the function, fill it up, and then return it as an IEnumerable<T>. Then as the caller of my function, I’d have to re-enumerate the results (or add it to another collection):

public static IEnumerable<T> GetItems()
{
  var collection = new List<T>();
  // add all the items to a collection
  return collection;
}

private static void Main()
{
  var myCollection = new List<T>();
  myCollection.AddRange(GetItems());
  // use myCollection...

  // or.....
  foreach (var item in GetItems())
  {
    // use the items
  }
}

Seems like overkill to me with that implementation. However, we’ll examine how using yield can truly transform this into something… better. So to reiterate, a few potential implementations for an API involving collections might be:

  • Return a List<T> class
  • Return an IList<T> (or even an ICollection<T>) interface
  • Return an IEnumerable<T> interface

Constantly Creating Collections

My design decisions, in the past, were really driven by two guidelines:

  • Make it easier for the person implementing/extending the API
  • Make it easy for the person consuming the API

As I quickly illustrated in the first section, this meant that I would have a method where I would create a collection, fill it with items, and then return it. I could generally pick any concrete collection class and return it since I would usually pick a simple collection as the return type. Easy.

One thing that might be noticeable with this approach is that it looks pretty inefficient to keep creating new collections, fill them, and then return them. I’ll illustrate with a simple example. We’ll create a class that has a method on it called GetItems(). As per my reasoning presented earlier, we’ll have this method return a List<T> instance, and to make this example easier to work with, we’ll pass in an IEnumerable<T> instance. For what it’s worth, the input to this function is really just for demonstration purposes here–We’re really focusing on how we’re creating our return value.

public class CreateNewListApi<T>
{
  public List<T> GetItems(IEnumerable<T> input)
  {
    var newCollection = new List<T>();

    foreach (var item in input)
    {
      newCollection.Add(item);
    }

    return newCollection;
  }
}

And now that we have our simple class we can mock up a little test for performance… Just how inefficient is creating new lists every time?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Creating New Collections");
    var api = new CreateNewListApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 1.73 seconds. The memory printout I get when running is 1615908864 bytes. So is that slow? Is that a lot of memory usage? I think it’s pretty hard to say conclusively without being able to compare it against anything. So let’s keep this number in mind as we continue to investigate the alternatives.

Side Note: At this point, some readers may be saying “Well, if the input to our function was also a list (or if whatever our function has to work with was otherwise equivalent to our return value) then we wouldn’t have to go populate a new collection every time… We can just return the underlying collection”! And I would say you are absolutely correct. If your function has access to an instance of the same type as the return type, then you could always just return that instance. But what implications does this have? You’re now giving people access to your underlying internals, and they can go modify them as they please. So, if you need to control access to items being added or removed, then it might not make sense for you to expose your internal collections like this.

Yield to Incoming API Alternatives

We’ve seen how my past implementations may have looked, so how might we tweak this? If we tweak our API a bit, we can make our method return an IEnumerable<T> instead. Let’s see what that might look like:

public class YieldingApi<T>
{
  public IEnumerable<T> GetItems(IEnumerable<T> input)
  {
    foreach (var item in input)
    {
      yield return item;
    }
  }
}

So in this API implementation, all we’ll be doing is iterating over some type of collection and then yielding each result. If we run it through the same type of test as out previous API implementation, what kind of results do we end up with?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Yielding");
    var api = new YieldingApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 2.80 seconds. The memory printout I get when running is 449409024 bytes. How does this relate back to our first implementation? Well, it’s certainly slower. It takes about 1.62x as long to enumerate using the yield implementation as it did with the first API we created. However, yield also uses less than 1/3 (about 27.8%, actually) of the memory footprint when compared to the first implementation. Pretty cool results!

Site Note: So yield was a bit slower according to our results, but what happens if print the elapsed time before we run that foreach loop? Well, on my machine it averages at about one millisecond. Now that’s fast, right?! The cool thing about using yield with the IEnumerable<T> interface is that the work is deferred. That is, not until the program goes to actually run the enumeration do we get our performance hit. Try it out! Try moving the time printout from after the foreach loop to before the foreach loop. Try sticking breakpoints in on the line that yields. You’ll see what I mean.

Summary

In this article, I’ve explored two different ways of implementing an API (specifically focusing on the return value). We saw a brief performance analysis between the two and I highlighted some differences in both approaches. Let’s recap though:

  • Approach 1: Returning a List<T> and creating the collection ahead of time
    • Appeared to be overall a bit faster then yielding.
    • Consumed much more memory than yielding.
    • Callers can use the results immediately for enumeration, checking count, or as a collection to add more things to
    • The return type of List<T> is a bit more restrictive than an IEnumerable<T> like in the second API implementation
  • Approach 2: Return type of IEnumerable<T> and yielding results
    • Appeared to be overall a bit slower than the List<T> implementation
    • Lazy. We don’t actually execute any enumeration code until the caller actually enumerates
    • Consumed significantly less memory than the first approach using List<T>
    • Callers can enumerate the results immediately, but they need to add the results to a collection class to do much more than enumerate

So next time you’re designing an API for your interfaces and classes, try keeping these things in mind!

EDIT (December 30th, 2013):
As per some comments on Google+ by Dan Nemec, I figured I’d add a bit more here in the summary. IEnumerable<T> on it’s own is certainly not useless, especially if you’re leveraging LINQ or extension methods. My main beef in the past was that the consumer of an API with a IEnumerable<T> return value can only iterate over the results… And that’s just because that’s all that IEnumerable<T> lets you do. Dan made a great point though–If you are leveraging things like extension methods, or LINQ (which introduces tons of handy extension methods for working with IEnumerable<T>) then you get all of that functionality tacked on to IEnumerable<T>.

So if you’re not fortunate enough to be working with LINQ or extension methods (i.e. working with legacy code in old .NET framework versions… and yes I am familiar with the attribute you can add in to allow extension methods provided you have a compiler version high enough to support it), then IEnumerable<T> sometimes just plain sucks. I’d wager the majority of C# developers aren’t in this boat though, so I’d like to thank Dan again for his comments.


Movember Wrap-up – Weekly Article Dump

Movember Wrap-up - Weekly Article Dump

Movember Wrap-up

At the start of December, it’s time for a lot of us to shave off our glorious Movember badges from our upper lips. This year, MoMagnets did an absolutely amazing job raising money for Movember. At the time of writing, we’re sitting at just under $2400! An incredible effort by Magnet Forensics and all of those that helped with their generous contributions.

My ‘stache didn’t quite get to where I wanted to this year. It was close, but it was another connector-less Movember for me. I was almost able to get some twisting done for some not-so-legitimate connectors. Oh well… Here’s what I ended up rocking for most of the month:

Movember Wrap-up - Nick's Final 'Stache

My final Movember creation: The Anti-Connector.

Matt Chang definitely took the lead for raising the most of all the MoMagnets members at over $700! Mica Sadler is sitting in second at just under $400. That’s nearly half the team’s total between these two beauties. We also had a very gracious contribution from our CEO that I wanted to call out. Thanks so much, Adam!

There’s still a bit of time left before donations are closed for the 2013 Movember season. We have until the 9th to get some final contributions in! If you’re feeling generous, please visit our team page and make a contribution. Every little bit helps, and we greatly appreciate it!

Articles

  • Top 5 Reasons People Love Their Jobs and How You Can Love Yours, Too: Some great points on why people love their jobs. Some of these may be pretty obvious, but it’s important to be reminded about what keeps people engaged. Among the top things: the work culture, the amazing people you get to work with, and autonomy. If you’re trying to create an awesome place to work (or if you’re looking for an awesome place to work) then these are probably things you’ll want to focus on!
  • 5 Things Zapping Your Company’s Productivity: Ilya Pozin always has some interesting articles. This article takes the perspective that some of the fancy perks or awesome processes you have in place may actually be hindering productivity. One common theme that was brought up under two separate points in this article is that sometimes people need a spot where they can work in peace. People like having an fun collaborative culture, but many personality types require some quiet time in order to buckle down.
  • Reduce Your Stress in 2 Minutes a Day: I’m not the type of person that truly believes doing one tiny thing for only a moment every day is going to have an enormous positive impact on your life. However, I do think that if you can take the time to try and do a few little things here and there, that overtime, you’re likely to have more a positive outlook. In this article, Greg McKeown shares a few tips on relaxing and trying to regain some focus. I don’t think it’s anything that’s going to be life-changing, but it never hurts to think about different ways to catch your breath.
  • Building a fast-failure-friendly firm: This was a pretty cool series of slides put together by Eric Tachibana that I thought was worth sharing. There are lot’s of articles on failing and why it’s important–especially for innovating. This series of slides provides a high level perspective on how you can approach failing… the right way!
  • Code Smells – Issue Number 3: This is an article I wrote about Code Smells. This entry talks about the use of exception handlers to guide logical flow in your code and alternatives for when your class hierarchy starts to get too many very light weight classes. As always, I’d love to get your feedback. If you have other code smells, or a different perspective on the ones that I’ve posted, please share them in the comments!
  • 5 Bad Thoughts That Will Throw You Off Track: This short little list is worth a quick read through. There are a ton of things that distract us every day, but the distractions you can easily control are the ones that you cause. Examples? Don’t take on too much at once. Don’t try to make every little thing you do perfect. It’s a quick read, but well worth the reminder!
  • Not Crying Over Old Code: Another programming article for this week. As the article says, the common meme for programming is that your old code is always bad code. However, there should be a point in your programming career where old code isn’t bad, it’s just different than how you might have approached it now. If your always experiencing your old code being bad, then maybe you’re not actually that great at programming yet! Or… maybe you’re just too damn picky.
  • Things I Wish Someone Had Told Me When I Was Learning How to Code: This article by Cecily Carver is something I’ve been hoping to come across for a while now. It’s another programming article–a good read for experienced programmers but incredibly important for newbies to check out. Cecily covers some of the roadblocks you experience early on, like code never (almost never) working the first time, or things you experience throughout your programming career, like always being told of a “better” alternative. I highly recommend you read through this if you dabble in programming, or if you’ve ever considered it.

Please visit our team page for MoMagnets and make a Movember contribution if you’re able to! Remember to follow Dev Leader on social media outlets to get these updates through the week. Thanks!

Nick Cosentino – LinkedIn
Nick Cosentino – Twitter
Dev Leader – Facebook
Dev Leader – Google+
Nick’s CodeProject Articles


  • Nick Cosentino

    Nick Cosentino

    I work as a team lead of software engineering at Magnet Forensics (http://www.magnetforensics.com). I'm into powerlifting, bodybuilding, and blogging about leadership/development topics over at http://www.devleader.ca.

    Verified Services

    View Full Profile →

  • Copyright © 1996-2010 Dev Leader. All rights reserved.
    Jarrah theme by Templates Next | Powered by WordPress