Tag: C#

Lambdas: An Example in Refactoring Code

Lambdas: An Example in Refactoring Code

Background: Lambdas and Why This Example is Important

Based on your experience in C# or other programming languages, you may or may not be familiar with what a lambda is. If the word “Lambda” is new and scary to you, don’t worry. Hopefully after reading this you’ll have a better idea of how you can use them. My definition of a lambda expression is a function that you can define in local scope to pass as an argument provided it meets the delegate signature. It’s probably pretty obvious to you that you can pass in object references and value types into all kinds of functions… But what about passing in a whole function as an argument? And what if you just want to declare a simple anonymous method right when you want to provide it to a function? Lambdas.

So now you at least have a basic idea of what a Lambda is. What’s this article all about? I wanted to discuss a real-world coding experience that helped demonstrate the value of lambdas to me. In my honest opinion, I think having real world programming topics to learn from is more beneficial than many of the “ideal” scenario examples/tutorials you end up reading on the Internet. We can argue and debate that certain things are better or worse in an ideal sense, but when you have a real practical example, it really helps to drive the point home.

So for me, I love working with events. I’m very comfortable with the concept of delegation in C#. I can have one object that may notify anyone that’s interested that something is happening, and the other objects that do care are able to handle the event. Thus, actions can get delegated to those objects that care to be notified. One of my weaknesses at this point in my development experience is leveraging the concept of delegation outside of the realm of events. Delegation is powerful, but it’s certainly not limited to hooking onto events with event handlers.

The particular example I want to illustrate is a parallel of a real coding scenario. I was refactoring some code that was leveraging close to zero OOP practices. I wanted to create a nice extensible framework and class hierarchy to replace it. Once I was done, a few colleagues of mine at Magnet Forensics picked up on a bit of a code smell. We all agreed the new framework and class hierarchy was better, but there seemed to be a lot of boiler plate code going on. We got into the discussion of how lambdas could reduce a lot of the light-weight classes I had introduced. After taking their thoughts and refactoring my changes just a little bit more, the benefits of the lambdas were obvious to me.

So obvious, I had to write about it to share with all of you! Feel free to skip ahead to the downloads section to get the code and follow along with it. There are plenty of options for downloading.

The Scenario

I mentioned that this was a real world scenario. I’ve contrived a parallel example that hopefully demonstrates some of the real world issues while illustrating how lambdas are useful. Let’s imagine we have some big chunk of logic that does data processing. In my real-world scenario, this may have existed as one monolithic function. I would have one big function that, based on all the parameters I provide, can figure out how to process the data I feed it.

Problems:

  • Hard to test (You need to test the whole function even if you’re really just wanting to target a small part of it)
  • Error prone (Any small change to one part can potentially break an entire other part of the function as it grows in complexity)
  • Not extensible (As soon as you need to deviate a little bit from the structure that’s existed, suddenly things get really complicated)

By switching to more of an OOP approach, I can start to address all of the above problems. So in this example, I’ll illustrate what my initial refactoring would have looked like by introducing classes. Afterward, I’ll show what my second refactor may have looked like after taking lambdas into account. In order to stay true to some of the real world problems you might encounter when performing a big refactor like this, I’ve opted to include some fictitious dependency. I refer to this at the “mandatory argument” or “important reference”. You’ll notice in the code that I don’t really use it to do any work, but it’s demonstrating having to pass down some other critical information to my classes that the original function may have had easy access to.

Pre-Refactor: No Lambdas Here!

Let’s start with our new OOP layout. I want to have a factory that can create data processor instances for me. So let’s define what those look like.

First, we have the interface for our data processors:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public interface IProcessor
  {
    bool TryProcess(object input);
  }
}

And then a simple interface for a factory that can create the data processor instances for us:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public interface IProcessorFactory
  {
    IProcessor Create(ProcessorType type, object mandatoryArgument, object value);
  }
}

As you may have noticed, the factory interface I’ve provided above takes a ProcessorType enumeration. You may or may not agree that using an enumeration as an argument for the factory is good practice, but I’m using it to make my example simple. Here’s what our enumeration will look like:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing
{
  public enum ProcessorType
  {
    GreaterThan,
    LessThan,
    NumericEqual,
    StringEqual,
    StringNotEqual,
    /* we could add countless more types of processors here. realistically,
     * an enum may not be the best option to accomplish this, but for
     * demonstration purposes it'll make things much easier.
     */
  }
}

And now we have a definition for all of the basic building blocks defined. These will also be used later when we refactor, so I wanted to get them out of the way right in the beginning.

Right. So, let’s create an extensible IProcessor implementation. We can address some of our basic requirements (like our artificial dependency) and create something that can easily be built on top of. All of our child classes will just have to handle validating their constructor input and overriding a single method. Easy!

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PreRefactor
{
  public abstract class Processor : IProcessor
  {
    private readonly object _importantReference;

    public Processor(object mandatoryArgument)
    {
      if (mandatoryArgument == null)
      {
        throw new ArgumentNullException("mandatoryArgument");
      }

      _importantReference = mandatoryArgument;
    }

    public bool TryProcess(object input)
    {
      if (input == null)
      {
        return false;
      }

      return Process(_importantReference, input);
    }

    protected abstract bool Process(object importantReference, object input);
  }
}

And now let’s provide the factory that’s going to be making all of these instances for us. Please not that the factory is left incomplete on purpose. I’ll only be providing two actual processor implementations and I’ll leave it up to you to try and fill out the rest!

using System;
using System.Collections.Generic;
using System.Text;

using LambdaRefactor.Processing.PreRefactor.Numeric;
using LambdaRefactor.Processing.PreRefactor.String;

namespace LambdaRefactor.Processing.PreRefactor
{
  public class ProcessorFactory : IProcessorFactory
  {
    public IProcessor Create(ProcessorType type, object mandatoryArgument, object value)
    {
      switch (type)
      {
        case ProcessorType.GreaterThan:
          return new GreaterProcessor(mandatoryArgument, value);
        case ProcessorType.StringEqual:
          return new StringEqualsProcessor(mandatoryArgument, value);
        /*
         * we still have to go implement all the other classes!
         */
        default:
          throw new NotImplementedException("The processor type '" + type + "' has not been implemented in this factory.");
      }
    }
  }
}

And now that we have a factory that can easily create our processors for us, let’s actually define some of our processor implementations.

We’ll start off with a simple processor for checking if some input is greater than a defined value. It should really only work with numeric values, but one of the challenges we need to work with is that our data is only provided to us as an object. As a result, we’ll have to do some type checking on our own.

using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace LambdaRefactor.Processing.PreRefactor.Numeric
{
  public class GreaterProcessor : Processor
  {
    private readonly decimal _value;

    public GreaterProcessor(object mandatoryArgument, object value)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      _value = Convert.ToDecimal(value, CultureInfo.InvariantCulture); // will throw exception on mismatch
    }

    protected override bool Process(object importantReference, object input)
    {
      decimal numericInput;
      try
      {
        numericInput = Convert.ToDecimal(input, CultureInfo.InvariantCulture);
      }
      catch (Exception)
      {
        return false;
      }

      return numericInput > _value;
    }
  }
}

And to put a spin on things, let’s implement a processor that operates on string values only. We’ll implement the processor that checks if strings are equal. Like the GreaterProcessor, we’re forced to get object references passed in. We’ll need to convert these to strings to work with them.

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PreRefactor.String
{
  public class StringEqualsProcessor : Processor
  {
    private readonly string _value;

    public StringEqualsProcessor(object mandatoryArgument, object value)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      _value = (string)value; // will throw exception on mismatch
    }

    protected override bool Process(object importantReference, object input)
    {
      return Convert.ToString(input, System.Globalization.CultureInfo.InvariantCulture).Equals(_value);
    }
  }
}

Where can we go from here?

  • We can make simple inverse processors by overriding others and inverting the return value on the Process() function. Want a StringDoesNotEqual processor? It’s just as easy as  inheriting from the StringEqualsProcessor and then modifying the return of Process(). Then we add this to our factory.
  • Adding other various types of processors is easy. We just have to extend our base class and add a couple of lines to our factory.
  • This code is much easier to test than one monolithic function that does all types of processing. We can now put a nice testing framework around this, and test each method on each class individually.

Post-Refactor: All of the Lambdas!

So… Why don’t we stop here? Because we can do better.

I mentioned that to make a simple inverse processor, all I had to do was override a class and invert the return value of Process(). That’s pretty easy to do… Except I need an entire new class to do it. If I want to make more types of numeric processing, I need to provide similar type checking and conversion. This code gets duplicated every time I go to add another simple class.

I also have my factory class responsible for creating my processor instances. They’re relatively coupled already, but I want developers to have to use my factory to construct instances of processor interface and not worry about the specific implementations. So what if my factory had a bit more say in the construction if the processors? I could use lambdas to pass in the logic that’s unique to each type of processor, and keep each type of processor pretty bare bones. This would move more logic into the factory, but reduce the number of processor implementations I have to make.

So let’s do better!

Let’s start with our new IProcessor implementation. We’ll provide a delegate signature that will be the basis for the lambda expressions we pass in:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PostRefactor
{
  public abstract class Processor : IProcessor
  {
    private readonly object _importantReference;

    public Processor(object mandatoryArgument)
    {
      if (mandatoryArgument == null)
      {
        throw new ArgumentNullException("mandatoryArgument");
      }

      _importantReference = mandatoryArgument;
    }

    public delegate bool ProcessDelegate<T>(object importantReference, T processorValue, T input);

    public bool TryProcess(object input)
    {
      if (input == null)
      {
        return false;
      }

      return Process(_importantReference, input);
    }

    protected abstract bool Process(object importantReference, object input);
  }
}

From here, we can come up with some child classes that that are generic enough for us to work with using lambas that still provide enough functionality for them to exist on their own. We can break our processors up based on the type of data they’ll be working with. That is, we can have a processor for numeric values and a processor for string values. This will cover a lot of the duplicated functionality that exists in the current state of our refactor if we wanted to keep creating new IProcessor implementations.

Let’s start with our NumericProcessor:

using System;
using System.Collections.Generic;
using System.Text;
using System.Globalization;

namespace LambdaRefactor.Processing.PostRefactor.Numeric
{
  public class NumericProcessor : Processor
  {
    private readonly decimal _value;
    private readonly ProcessDelegate<decimal> _processDelegate;

    public NumericProcessor(object mandatoryArgument, object value, ProcessDelegate<decimal> processDelegate)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      if (processDelegate == null)
      {
        throw new ArgumentNullException("processDelegate");
      }

      _value = Convert.ToDecimal(value, CultureInfo.InvariantCulture); // will throw exception on mismatch
      _processDelegate = processDelegate;
    }

    protected override bool Process(object importantReference, object input)
    {
      decimal numericInput;
      try
      {
        numericInput = Convert.ToDecimal(input, CultureInfo.InvariantCulture);
      }
      catch (Exception)
      {
        return false;
      }

      return _processDelegate(importantReference, _value, numericInput);
    }
  }
}

And similarly, a StringProcessor:

using System;
using System.Collections.Generic;
using System.Text;

namespace LambdaRefactor.Processing.PostRefactor.String
{
  public class StringProcessor : Processor
  {
    private readonly string _value;
    private readonly ProcessDelegate<string> _processDelegate;

    public StringProcessor(object mandatoryArgument, object value, ProcessDelegate<string> processDelegate)
      : base(mandatoryArgument)
    {
      if (value == null)
      {
        throw new ArgumentNullException("value");
      }

      if (processDelegate == null)
      {
        throw new ArgumentNullException("processDelegate");
      }

      _value = (string)value; // will throw exception on mismatch
      _processDelegate = processDelegate;
    }

    protected override bool Process(object importantReference, object input)
    {
      return _processDelegate(importantReference, _value, Convert.ToString(input, System.Globalization.CultureInfo.InvariantCulture));
    }
  }
}

With these two basic child classes built upon our new IProcessor implementation, we can restructure a new IProcessorFactory implementation. As I mentioned, we can leverage lambdas to move some logic back into the factory class and keep the processor implementations relatively basic.

Here’s the new factory:

using System;
using System.Collections.Generic;
using System.Text;

using LambdaRefactor.Processing.PostRefactor.Numeric;
using LambdaRefactor.Processing.PostRefactor.String;

namespace LambdaRefactor.Processing.PostRefactor
{
  public class ProcessorFactory : IProcessorFactory
  {
    public IProcessor Create(ProcessorType type, object mandatoryArgument, object value)
    {
      switch (type)
      {
        case ProcessorType.GreaterThan:
          return new NumericProcessor(mandatoryArgument, value, (_, x, y) => x <; y);
        case ProcessorType.StringEqual:
          return new StringProcessor(mandatoryArgument, value, (_, x, y) => x == y);
        /*
         * Look how easy it is to add new processors! Exercise for you:
         * implement the remaining processors in the enum!
         */
        default:
          throw new NotImplementedException("The processor type '" + type + "' has not been implemented in this factory.");
      }
    }
  }
}

As you can see, our new factory is simple like our first implementation. The major difference? We’re passing very simple lambdas that would have otherwise been functionality defined in a very light-weight child class. This allows us to move away from having many potentially very bare-bones classes and minimizes the amount of boilerplate duplication.

Summary

I didn’t post it here, but the original implementation that this example paralleled  in real life was a pain to deal with. It was hard to test, brittle to modify/extend, and just downright unwieldly. It was obvious to me that switching to a refactored object-oriented implementation was going to make this style of code easy to extend and easy to test.

The initial refactor posted in this example was a great step in the right direction. The code became easy to build upon by relying on simple OOP principals, and granular parts of the functionality became really easy to test. If I just wanted to test certain types of numeric processing, I didn’t have set up a test for my entire massive “process” function. All I’d have to do is make an instance of the processor I want to test, and call the methods I’d like to cover. Incredibly easy.

Lambdas took this to the next level though. By leveraging lambads, I could refactor even more common code into a base class. This meant that  in order to use my processors properly, the final factory class implementation definitely became required to use. It caused a paradigm shift where instead of making lots of light-weight child classes for additional processor implementations, I’d only need to implement some logic in the factory. All of my existing processors could be refactored into a handful of generic processor classes, and the factory would be responsible for passing in the necessary lambdas.

Lambdas let you accomplish some pretty powerful things, and this refactoring example was one case where they made code much easier to manage. Hopefully you can find a good use for lamba expressions in your next up-coming programming task!

Code Downloads


Cameron Sapp – Recognizing The New Guy

 

 

Cameron Sapp (Rocking awesome handlebars for Movember)

Cameron Sapp and a Little Background

A couple weeks ago I mentioned that I wanted to start publicly acknowledging some of my teammates. While this is the first one, it certainly won’t be the last. At Magnet Forensics, I’m surrounded by many individuals that bring a lot to the table. There’s certainly no reason and no way I’d only be able to pick one person to write about. Now there wasn’t a particular reason I picked this individual first, but I think I had some concrete things fresh in my head that I wanted to share. Without too much more rambling, I’d like to introduce Cameron Sapp!

New Kid on the Block

Cameron joined our team earlier this year. I don’t think any of us doubted his technical abilities and we were all excited to bring him on board. After all, we have a ton of stuff to work on and we need more great minds working with us! We were getting pretty impatient waiting for him to start, but it was definitely worth the wait.

Cam fit in to the work culture really well and really quickly too. Heck, he’s one of Team Magnet’s awesome volleyball players! Something people may not pay attention to is how much a work culture fit is important in a small organization. Being able to get along with all of your teammates and share a common vision is absolutely crucial for being successful. Luckily for us, Cameron fits in well with the team and definitely embraces the Magnet culture!

I was recently told by a bright individual, Dan Silivestru of tinyHippos, that there will be a time where someone younger is going to show up and surprise me with what they know. Of course, it’s not that I walk around doubting the ability of people, but unfortunately it’s pretty common for age and/or experience to bring about big assumptions for people’s abilities. I’m still young and early in my career, so I don’t think age is something I’m concerned with–but I might be guilty of thinking highly of my technical abilities. While Cameron isn’t the first, and certainly won’t be the last, he definitely was able to pull some tricks from his sleeves to impress me. For that, I would like to applaud him and recognize him here on The Internetz.

Whatcha Gonna Do With All Them Lambdas?

I’ve been programming in Microsoft’s C# for quite a few years now. I’m certainly not a master by any stretch of the imagination, but I’d say I’m pretty well versed. I’ve also written in the past about how I like to use events a lot when I’m programming (like here, here, and here) and almost always try to find an event-driven approach to things. But what does this have to do with Mr Cameron Sapp?

Well, you see… In C# it’s often the case where you hook up events like this:


someObject.DidSomething += SomeObject_DidSomething;

private void SomeObject_DidSomething(Object sender, EventArgs e)
{
    // do some awesome stuff.
}

That’s not so bad, right? Well, except if you’re making these suckers everywhere… And when you don’t want to have to type out a big ugly signature… Or when the type of your event arguments is obnoxiously long… Well, you get the point. If you’re not a C# programmer, take my word for it: if you use events a lot, having these event handlers all over the place sometimes just sucks to have to look at.

Enter… The lambda.

So once upon a time, Cameron stuck up a code review. Things were looking pretty good (as per usual with Cam’s code), but I noticed something right before I was going to give it the stamp of approval.


// Some code...

someClass.SomeEvent += (s, e) =>
{
    // event handler logic
}

// Some more code...

What the heck is that?! My alarms for event handler memory leaks weren’t going off (since this handler needed to exist for the entire lifetime of the objects in question), but I had no idea what I was looking at. Cameron’s a pretty smart guy, I remember thinking, so this code definitely had to compile on his machine before he pushed it up for me to review. Still… What was I looking at?

This was my first real shocker where someone caught me off guard for something I always felt really comfortable with. I mean… C# and events are my bread and butter. How was this guy showing me something I hadn’t seen before regarding events? How can he know something about them I don’t?! Well, he did it. And I’m sure that he’s got a lot more up there in that head of his that I don’t know. And I can’t wait for him to teach me it!

Summary

So this was pretty quick, and it probably doesn’t do Cameron enough justice, but I think it’s a start. We’re really fortunate to have Cameron as part of our team–both from a culture fit and a technical perspective. He’s a rock solid developer that is not only willing to adapt to our coding environment, but he’s also got lots of insight to bring to the table.

It’s important that we never put ourselves in a position where we think we know it all. As soon as you get comfortable with what you know, you stop learning. When you stop learning, you have people like Cameron show up and send you a wake up call. There isn’t a single person out there who knows everything, and you might be surprised who can teach you a thing or two.

Thanks for being part of our team, Cam. Let’s show ’em how it’s done.

More team member recognition to come! Stay tuned.


Dynamic Programming with Python and C#

Dynamic Coding with C# and Python

Dynamic Code: Background

Previously, I was expressing how excited I was when I discovered Python, C#, and Visual Studio integration. I wanted to save a couple examples regarding dynamic code for a follow up article… and here it is! (And yes… there is code you can copy and paste or download).

What does it mean to be dynamic? As with most things, wikipedia provides a great start. Essentially, much of the work done for type checking and signatures is performed at runtime for a dynamic language. This could mean that you can write code that calls a non-existent method and you wont get any compilation errors. However, once execution hits that line of code, you might get an exception thrown. This Stack Overflow post’s top answer does a great job of explaining it as well, so I’d recommend checking that out if you need a bit more clarification. So we have statically bound and dynamic languages. Great stuff!

So does that mean Python is dynamic? What about C#?

Well Python is certainly dynamic. The code is interpreted and functions and types are verified at run time. You won’t know about type exceptions or missing method exceptions until you go to execute the code. For what it’s worth, this isn’t to be confused with a loosely typed language. Ol’ faithful Stack Overflow has another great answer about this. The type of the variable is determined at runtime, but the variable type doesn’t magically change. If you set a variable to be an integer, it will be an integer. If you set it immediately after to be a string, it will be a string. (Dynamic, but strongly typed!)

As for C#, in C# 4 the dynamic keyword was introduced. By using the dynamic keyword, you can essentially get similar behaviour to Python. If you declare a variable of type dynamic, it will take on the type of whatever you assign to it. If I assign a string value to my dynamic variable, it will be a string. I can’t perform operations like pre/post increment (++) on the variable when it’s been assigned a string value without getting an exception. If I assign an integer value immediately after having assigned a string value, my variable will take on the integer type and my numeric operators become available.

Where does this get us with C# and Python working together then?

Example 1: A Simple Class

After trying to get some functions to execute between C# and Python, I thought I needed to take it to the next level. I know I can declare classes in Python, but how does that look when I want to access it from C#? Am I limited to only calling functions from Python with no concept of classes?

The answer to the last question is no. Most definitely not. You can do some pretty awesome things with IronPython. In this example, I wanted to show how I can instantiate an instance of a class defined within a Python script from C#. This script doesn’t have to be created in code (you can use an external file), so if you need more clarification on this check out my last Python/C# posting, but I chose to do it this way to have all the code in one spot. I figured it might be easier to show for an example.

We’ll be defining a class in Python called “MyClass” (I know, I’m not very creative, am I?). It’s going to have a single method on it called “go” that will take one input parameter and print it to the console. It’s also going to return the input string so that we can consume it in C# and use it to validate that things are actually going as planned. Here’s the code:

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.Scripting.Hosting;

using IronPython.Hosting;

namespace DynamicScript
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Enter the text you would like the script to print!");
            var input = Console.ReadLine();

            var script =
                "class MyClass:\r\n" +
                "    def __init__(self):\r\n" +
                "        pass\r\n" +
                "    def go(self, input):\r\n" +
                "        print('From dynamic python: ' + input)\r\n" +
                "        return input";

            try
            {
                var engine = Python.CreateEngine();
                var scope = engine.CreateScope();
                var ops = engine.Operations;

                engine.Execute(script, scope);
                var pythonType = scope.GetVariable("MyClass");
                dynamic instance = ops.CreateInstance(pythonType);
                var value = instance.go(input);

                if (!input.Equals(value))
                {
                    throw new InvalidOperationException("Odd... The return value wasn't the same as what we input!");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! There was an exception while running the script: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

Not too bad, right? The first block of code just takes some user input. It’s what we’re going to have our Python script output to the console. The next chunk of code is our Python script declaration. As I said, this script can be loaded from an external file and doesn’t necessarily have to exist entirely within our C# code files.

Within our try block, we’re going to setup our Python engine and “execute” our script. From there, we can ask Python for the type definition of “MyClass” and then ask the engine to create a new instance of it. Here’s where the magic happens though! How can we declare our variable type in C# if Python actually has the variable declaration? Well, we don’t have to worry about it! If we make it the dynamic type, then our variable will take on whatever type is assigned to it. In this case, it will be of type “MyClass”.

Afterwards, I use the return value from calling “go” so that we can verify the variable we passed in is the same as what we got back out… and it definitely is! Our C# string was passed into a Python function on a custom Python class and spat back out to C# just as it went in. How cool is that?

Some food for thought:

  • What happens if we change the C# code to call “go1” instead of “go”? Do we expect it to work? If it’s not supposed to work, will it fail at compile time or runtime?
  • Notice how our Python method “go” doesn’t have any type parameters specified for the argument “input”? How and why does all of this work then?!

Example 2: Dynamically Adding Properties

I was pretty excited after getting the first example working. This meant I’d be able to create my own types in Python and then leverage them directly in C#. Pretty fancy stuff. I didn’t want to stop there though. The dynamic keyword is still new to me, and so is integrating Python and C#. What more could I do?

Well, I remembered something from my earlier Python days about dynamically modifying types at run-time. To give you an example, in C# if I declare a class with method X and property Y, instances of this class are always going to have method X and property Y. In Python, I have the ability to dynamically add a property to my class. This means that if I create a Python class that has method X but is missing property Y, at runtime I can go right ahead and add property Y. That’s some pretty powerful stuff right there. Now I don’t know of any situations off the top of my head where this would be really beneficial, but the fact that it’s doable had me really interested.

So if Python lets me modify methods and properties available to instances of my type at runtime, how does C# handle this? Does the dynamic keyword support this kind of stuff?

You bet. Here’s the code for my sample application:

using System;
using System.Collections.Generic;
using System.Text;

using Microsoft.CSharp.RuntimeBinder;

using IronPython.Hosting;

namespace DynamicClass
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Console.WriteLine("Press enter to read the value of 'MyProperty' from a Python object before we actually add the dynamic property.");
            Console.ReadLine();

            // this script was taken from this blog post:
            // http://znasibov.info/blog/html/2010/03/10/python-classes-dynamic-properties.html
            var script =
                "class Properties(object):\r\n" +
                "    def add_property(self, name, value):\r\n" +
                "        # create local fget and fset functions\r\n" +
                "        fget = lambda self: self._get_property(name)\r\n" +
                "        fset = lambda self, value: self._set_property(name, value)\r\n" +
                "\r\n" +
                "        # add property to self\r\n" +
                "        setattr(self.__class__, name, property(fget, fset))\r\n" +
                "        # add corresponding local variable\r\n" +
                "        setattr(self, '_' + name, value)\r\n" +
                "\r\n" +
                "    def _set_property(self, name, value):\r\n" +
                "        setattr(self, '_' + name, value)\r\n" +
                "\r\n" +
                "    def _get_property(self, name):\r\n" +
                "        return getattr(self, '_' + name)\r\n";

            try
            {
                var engine = Python.CreateEngine();
                var scope = engine.CreateScope();
                var ops = engine.Operations;

                engine.Execute(script, scope);
                var pythonType = scope.GetVariable("Properties");
                dynamic instance = ops.CreateInstance(pythonType);

                try
                {
                    Console.WriteLine(instance.MyProperty);
                    throw new InvalidOperationException("This class doesn't have the property we want, so this should be impossible!");
                }
                catch (RuntimeBinderException)
                {
                    Console.WriteLine("We got the exception as expected!");
                }

                Console.WriteLine();
                Console.WriteLine("Press enter to add the property 'MyProperty' to our Python object and then try to read the value.");
                Console.ReadLine();

                instance.add_property("MyProperty", "Expected value of MyProperty!");
                Console.WriteLine(instance.MyProperty);
            }
            catch (Exception ex)
            {
                Console.WriteLine("Oops! There was an exception while running the script: " + ex.Message);
            }

            Console.WriteLine("Press enter to exit...");
            Console.ReadLine();
        }
    }
}

Let’s start by comparing this to the first example, because some parts of the code are similar. We start off my telling  the user what’s going to happen and wait for them to press enter. Nothing special here. Next, we declare our Python script (again, you can have this as an external file) which I pulled form this blog. It was one of the first hits when searching for dynamically adding properties to classes in Python, and despite having limited Python knowledge, it worked exactly as I had hoped. So thank you, Zaur Nasibov.

Inside our try block, we have the Python engine creation just like our first example. We execute our script right after too and create an instance of our type defined in Python. Again, this is all just like the first example so far. At this point, we have a reference in C# to a type declared in Python called “Properties”. I then try to print to the console the value stored inside my instances property called “MyProperty”. If you were paying attention to what’s written in the code, you’ll notice we don’t have a property called “MyProperty”! Doh! Obviously that’s going to throw an exception, so I show that in the code as well.

So where does that leave us then? Well, let’s add the property “MyProperty” ourselves! Once we add it, we should be able to ask our C# instance for the value of “MyProperty”. And… voila!

Some food for thought:

  • When we added our property in Python, we never specified a type. What would happen if we tried to increment “MyProperty” after we added it? What would happen if we tried to assign an integer value of 4 to “MyProperty”?
  • When might it be useful to have types in C# dynamically get new methods or properties?

Summary

With this post, we’re still just scratching the surface of what’s doable when integrating Python and C#. Historically, these languages have been viewed as very different where C# is statically bound and Python is a dynamic language. However, it’s pretty clear with a bit of IronPython magic that we can quite easily marry the two languages together. Using the “dynamic” keyword within C# really lets us get away with a lot!

Source code for these projects is available at the following locations:


Movember Prep – Weekly Article Dump

MoMagnets - Magnet Forensics' Movember Team

Movember Preparation

You might think we’re a bit early on this one, but at Magnet Forensics we’re going to take Movember to a whole new level this year. If you’re not familiar with Movember, you may want to head over here and get a rundown of the history of it. Movember started in Australia between a group of people who wanted to (somewhat jokingly) bring the moustache back into style. The next year they started getting people to grow mo’s for causes. Now people participate in Movember to raise awareness for men’s health, and it’s bigger than ever.

Our team members of MoMagnets have started discussing the various styles of mo’s that they’ll grow this year. It looks like there’s going to be some intra-team competition to grow the best mo. The top contenders? It’s looking like:

Matthew Chang - Movember

Matthew “The Chang” “Changarang” Chang sporting a well-groomed black moustache. Although it’s a standard ‘stache, the care put into keeping this beauty mo in tip-top shape is obvious. Can he do it again for this Movember?

Cameron Sapp - Movember

Cameron Sapp showing off a rock solid handle bar mo. The bars on this ‘stache are so impressive that it almost gives the illusion that this mo is taller than it is wide. Wait… is it?!

Check out the MoMagnets page and keep track of us! Please contribute what you can to help raise awareness for men’s health.

Articles

  • Python, Visual Studio, and C#… So. Sweet.: First one on the list this week is the post I put out on Monday about using Python, C#, and Visual Studio all together. It’s definitely for the developers out there, but for those of you who aren’t programmers, it’s still interesting to see how PyTools and IronPython have bridged a gap between C# and Visual Studio. I was pretty happy with the number of people who responded on social media and thought that it was a good read. The tweets actually led me to find a related post by Scott Hanselman from earlier this year (that I wish I saw sooner). My article has also received some pretty good visibility at Code Project which I’m excited about. Feel free to check it out over there too (people seem more likely to engage in discussion at Code Project versus on my blog)!
  • Want To Build A Business? Lead With Trust: David Hassell wrote an article that really hit home with me. Having a successful business means crafting a team and culture built upon trust. It needs to be the foundation of your team. Having high levels of trust makes everything else in the business come together more easily, but lacking trust can really make everything fall apart. Teams need to trust their leaders, and leaders need to trust their team members–it goes both ways.
  • Amazon CEO Jeff Bezos Had His Top Execs Read These Three Books: John Fortt discusses his interview with Amazon CEO Jeff Bezos. Now while I don’t read as much as I should (and I’m consciously trying to get better at it), I thought this little list of books might be great to keep my eye out for:
  • Confidence ‘boosts pupils’ academic success: I thought this article was a great find. It’s primarily around research that’s shown confidence plays a big role in students’ success, but I believe it applies outside of the realm of formal education. As a leader or mentor, I think it’s incredibly important to instill confidence. You want your team members to know you trust them with what they’re doing. They need to know they can make mistakes and learn without having to be punished for doing so. Having that confidence is going to be what makes them successful.
  • Leadership Lessons From LEGO: What do leadership and Lego have in common? A whole lot according to John Kotter. Consider innovation (get creative with those bricks!), overcoming challenges (can’t find that piece you were looking for?), team work (building things with friends is way more fun), and quality (it’s as good as you make it). It was an unexpected article for me to stumble upon, but I thought the parallels were interesting!
  • The Four Most Powerful Lessons in Management: Joel Peterson has some great points on being a successful leader or manager. Among them, putting actions behind your words, bring the right people on board (noticing a trend with having the right people yet?), and having a meaningful mission.
  • What is a Thought Leader?: I found myself asking this question at one point, which is why I wanted to share Daniel Tunkelang‘s article. It seems straight forward really. It’s important to have an area of expertise in the ideas you want to share, and it’s important that the things you’re sharing have meaning. In my case with Dev Leader, I certainly haven’t mastered leadership and programming, but I’m sharing the ideas that I’m hoping will some day get me there.
  • 17 Things You Should Never Say to Your Boss: This was definitely a great read. At first, I started thinking “How could anyone in their right mind say these things to their boss”? But then I realized I had actually heard some of these things (or similar things) and it really got me thinking. Dave Kerpen has put together a great list, and although it’s humourous, it’s still something important to watch out for. Just in it for the money? Not your role? Some people need to get a grip or find something else to do in their career.
  • Why These Happiness “Boosters” Might Actually Make You Feel Worse: Gretchen Rubin shares some ideas on why certain things we do to make us happier may actually be counter-productive. One interesting one I thought was the idea of your attitude shaping your behaviour may actually be your behaviour shaping your attitude. On weekends I often hang around in a pair of shorts until I have to head out of my condo. If I got in the habit of being prepped to leave the house and be productive from the beginning of the day, would I find that I’m actually more productive? Worth trying!
  • What Makes Developers Really Great: Deane Barker shares his experience with a software developer that was giving off some bad vibes. So what’s a good developer? Is it just someone who can code? Do they need to know all the latest and best languages, dream in code, and have four computer science degrees? It certainly helps (and I don’t think many would dismiss it), but the one thing that’s really important is their attitude and ability to work in their team. Check out the comments on that blog post. If you’re working on a team and you can’t fit in the team, you’ll bring the whole team down. This means if you’re all soft skills and no hard skills, you can’t contribute squat. If you’re all hard skills and no soft skills, you’re going to be a road block to your team. You need to have both to be a really great developer.

Remember to check out the MoMagnets page! We’d really appreciate it. Follow Dev Leader on social media outlets to get these updates through the week.

Nick Cosentino – LinkedIn
Nick Cosentino – Twitter
Dev Leader – Facebook
Dev Leader – Google+

You can also check out Dev Leader on FlipBoard.


Simple Way To Structure Threads For Control

Background

I’ve previously discussed the differences between the BackgroundWorker and Thread classes, but I figured it would be useful to touch on some code. I’d like to share the pattern I commonly use when creating threads in C# and discuss some of the highlights.

The Single Thread

I like to use this design when I have a single thread I need to run and in the context of my object responsible for running the thread, I do mean having a single thread. Of course, you could have your object in control of multiple threads as long as you repeat this design pattern for each of them.

Here’s the interface that I’ll be using for all of the examples:

    internal interface IThreadRunner
    {
        #region Exposed Members

        void Start();

        void Stop();

        #endregion 
    }

Behold!

    internal class SingleThreadRunner : IThreadRunner
    {
        #region Fields

        private readonly object _threadLock;
        private readonly AutoResetEvent _trigger;
        private Thread _theOneThread;

        #endregion

        #region Constructors

        /// 
        /// Prevents a default instance of the class from being created.
        /// 
        private SingleThreadRunner()
        {
            _threadLock = new object();
            _trigger = new AutoResetEvent(false);
        }

        #endregion

        #region Exposed Members

        public static IThreadRunner Create()
        {
            return new SingleThreadRunner();
        }

        public void Start()
        {
            lock (_threadLock)
            {
                // check if already running
                if (_theOneThread != null)
                {
                    return;
                }

                _theOneThread = new Thread(DoWork);
                _theOneThread.Name = "The One Thread";
                _theOneThread.Start(_trigger);
            }
        }

        public void Stop()
        {
            lock (_threadLock)
            {
                // check if not running
                if (_theOneThread == null)
                {
                    return;
                }

                _theOneThread = null;
                _trigger.Set();
            }
        }

        #endregion

        #region Internal Members

        private void DoWork(object parameter)
        {
            var currentThread = Thread.CurrentThread;

            // this was the trigger that we passed in. elesewhere in the 
            // instance, we can use this object to wake up the thread.
            var trigger = (AutoResetEvent)parameter;

            try
            {
                // keep running while we're expected to be running
                while (currentThread == _theOneThread)
                {
                    // DO ALL SORTS OF AWESOME WORK HERE.
                    Console.WriteLine("Awesome work being done.");

                    // put this thread to sleep, but remember it can be woken 
                    // up from other places in this instance.
                    trigger.WaitOne(1000);
                }
            }
            finally
            {
                lock (_threadLock)
                {
                    // if we were still expected to be running, change the 
                    // state to suggest that we're not
                    if (_theOneThread == currentThread)
                    {
                        _theOneThread = null;
                    }
                }
            }
        }

        #endregion
    }

This design was taken from some Java programming I had done in a previous life. Essentially, I have a thread that is responsible for doing some work in a loop. It could be anything… Periodically polling for some data, a work dequeing thread, a random-cursor-moving thread… Anything! The point is, you only want one of these suckers hanging around. How is this accomplished?

Leveraging the instance variable that marks the one expected running thread is key here. Whenever this thread checks if it should still be running, if the current thread doesn’t match what’s assigned to the instance variable then it needs to stop! This means you could potentially spawn off two of these threads, and if you set the instance variable to one of the two, then the other one should kill itself off! Pretty neat.

By using the reset event, we can actually interrupt this thread if it’s sleeping. This is great if we have a thread that periodically wakes up to do some work but we want to stop it and have it stop fast. We simple set our instance variable for the thread to be null and then set this thread’s reset event to ensure it get’s woken up. Presto! It wakes up, checks the condition, and realizes it needs to exit the loop.

Simple.

The Handful of Threads

This design is almost identical to the single thread design above. I use it primarily when I want to have an object responsible for a bunch of threads that are turned on/off under the same conditions. The major difference between the two designs? In the single thread scenario, we check that our current thread is still set to be the one instance. In this design, we need all of our threads to be checking against the same state object which is not going to be a single thread instance.

Let’s have a peek:

    internal class GroupThreadRunner : IThreadRunner
    {
        #region Fields

        private readonly object _threadLock;
        private readonly Dictionary<Thread, AutoResetEvent> _triggers;

        private bool _running;

        #endregion

        #region Constructors

        /// 
        /// Prevents a default instance of the class from being created.
        /// 
        private GroupThreadRunner()
        {
            _threadLock = new object();
            _triggers = new Dictionary<Thread, AutoResetEvent>();
        }

        #endregion

        #region Exposed Members

        public static IThreadRunner Create()
        {
            return new GroupThreadRunner();
        }

        public void Start()
        {
            lock (_threadLock)
            {
                // check if any are already running
                if (_triggers.Count > 0)
                {
                    return;
                }

                _running = true;

                const int NUMBER_OF_THREADS = 4;
                for (int i = 0; i < NUMBER_OF_THREADS; ++i)
                {
                    var thread = new Thread(DoWork);
                    thread.Name = "Thread " + i;

                    var trigger = new AutoResetEvent(false);
                    _triggers[thread] = trigger;

                    thread.Start(trigger);
                }
            }
        }

        public void Stop()
        {
            lock (_threadLock)
            {
                // check if not running
                if (_triggers.Count <= 0)
                {
                    return;
                }

                _running = false;
                foreach (var trigger in _triggers.Values)
                {
                    trigger.Set();
                }
            }
        }

        #endregion

        #region Internal Members

        private void DoWork(object parameter)
        {
            var currentThread = Thread.CurrentThread;

            // this was the trigger that we passed in. elesewhere in the 
            // instance, we can use this object to wake up the thread.
            var trigger = (AutoResetEvent)parameter;

            try
            {
                // keep running while we're expected to be running
                while (_running)
                {
                    // DO ALL SORTS OF AWESOME WORK HERE.
                    Console.WriteLine("Awesome work being done by " + currentThread.Name);

                    // put this thread to sleep, but remember it can be woken 
                    // up from other places in this instance.
                    trigger.WaitOne(1000);
                }
            }
            finally
            {
                lock (_threadLock)
                {
                    _triggers.Remove(currentThread);

                    // if we were still expected to be running, change the 
                    // state to suggest that we're not
                    if (_running && _triggers.Count <= 0)
                    {
                        _running = false;
                    }
                }
            }
        }

        #endregion
    }

Summary

The above patterns I discussed cover my common usage for threads: Instances that have reoccurring work over long periods of time. Both patterns are very similar and only have slight modifications to make them support one instance or many thread instances running. If you have one unique thread or many threads… there’s a pattern for you!

Check out a full working example of this code over here.


What Makes a Good API?

Background

My position at work allows me a bit of freedom in how I code and more importantly, influence how others code. I was recently having a conversation with a colleague about what I think makes a good API, from a high level. The context of our discussion was pertaining to developing a C# based API, but this really applies to any object oriented API.

I had two key points that I wanted to address, and while they’re not the only important things, I believe they’re often overlooked. The first thing is how people will use your API, so how they will call methods and use the results. The second point was about how people will implement your API should they want to extend your work and implement their own classes. Here’s what I was trying to drive home:

 

Usage:

As a programmer, when you use an API, you want it to be simple. If you’re using preexisting concrete classes, you want the methods to be easy to call and you want the results of those methods to be useful. How do you achieve this when making an API? Well, my guidelines are:

  • Make your inputs to the method as generic as possible
  • Make your return values as information-packed as possible

Simple right? If your inputs are generic enough, you can pass all sorts of things in. For example, if your function takes in a ReadOnlyCollection, this function wouldn’t necessarily be as easy to use as one that takes only an IEnumerable. If it’s not obvious, it’s because IEnumerable is a far more generic type. With an IEnumerable, I can pass in an array, a list, a queue, a stack, or any collection. I can pass in anything that implements IEnumerable! Conversely, if I require a ReadOnlyCollection, all of my callers who may have these other various types of collections need to do some conversion to make it a ReadOnlyCollection.

To the second point, you want as much information as you can get when calling a function. It’s almost the exact same argument as for parameters, but it works the opposite way. Consider if I have a function that returns an IEnumerable. That means that for anyone that calls my function, all they’ll have access to is something they can enumerate over to get string values. Okay, that’s not too bad… But what if realistically everyone who calls your method really needs a list of strings? What if the common way to use your method is to get the IEnumerable result of your function, create a list out of it, and then add a few more items. Your API has basically created the additional step of requiring callers to create a list out of your return value. So… Why not just return a list? This is a lot more obvious if you look at your concrete implementation and notice that you likely do use something like a list (or some other concrete collection) when doing work inside the function. Why return something that’s harder to use?

 

Implementation:

The flip side to all of this is how other developers will implement the interfaces (or extend the classes) you provide in your API. And guess what? All of the arguments I just made for simplifying the life of the caller are essentially inverted for people implementing interfaces in your API.

If my interface calls for an IEnumerable to be passed in, then the only thing I can do is enumerate over it. Maybe in my own implementation this works fine… but what if someone else implementing your interface would benefit greatly from having a list? What if they can make an awesome optimization by knowing how many items are in the collection or by checking to see the if the 100th item is a particular value? Well, they can only enumerate, so this becomes difficult.

As for return types, before I argued that for the caller, returning as much information as possible is great. Consider this example. If in my API I managed to create a custom collection class that has all sorts of awesome metadata. Just to make up something completely random, let’s pretend I have a class for collections of integers and I have all these fancy properties on it that tell me the Mean/Median/Mode. The caller would say that’s awesome! Sweet! So much information returned to me just by calling this simple function! However, the implementer of your interface is now thinking, “Oh crap… First you restrict my inputs to something super basic and then I have to somehow return that fancy object?! How the heck am I going to do that?!”

 

Summary:

To summarize what I wrote here, I think a good guideline for API’s comes down to:

  • Making inputs generic enough to ease the life of the caller and provide just enough information to the implementer of the method.
  • Make return values as information-packed as possible without placing the burden of creating complex classes (and adding dependencies) to the implementer of the method.

Simple right? If your API is designed such that others will not be extending it (and it’s really only people calling your methods) then you can completely bias your design in favour of the caller!


  • Nick Cosentino

    Nick Cosentino

    I work as a team lead of software engineering at Magnet Forensics (http://www.magnetforensics.com). I'm into powerlifting, bodybuilding, and blogging about leadership/development topics over at http://www.devleader.ca.

    Verified Services

    View Full Profile →

  • Copyright © 1996-2010 Dev Leader. All rights reserved.
    Jarrah theme by Templates Next | Powered by WordPress