Tag: performance

Yield! Reconsidering APIs with Collections

Yield! Reconsidering APIs with Collections (Image by http://www.sxc.hu/)

Yield: A Little Background

The yield keyword in C# is pretty cool. Being used within an iterator, yield lets a function return an item as well as control of execution to the caller and upon next iteration resume where it left off. Neat, right? MSDN documentation lists these limitations surrounding the use of the yield keyword:

  • Unsafe blocks are not allowed.
  • Parameters to the method, operator, or accessor cannot be ref or out.
  • A yield return statement cannot be located anywhere inside a try-catch block. It can be located in a try block if the try block is followed by a finally block.
  • A yield break statement may be located in a try block or a catch block but not a finally block.

So what does this have to do with API specifications?

A whole lot really, especially if you’re dealing with collections. I personally haven’t been a big user of the yield keyword, but I’ve never really been forced to use it. After playing around with it for a bit, I saw a lot of potential. I’ve written before about what I think makes a good API. In my article, I was making a point to discuss two perspectives:

  • Who needs to implement your interface. You want it to be easy for them to implement.
  • Who needs to call your interface. You want it to be easy for them to use.

In my opinion, the IEnumerable<T> interface was a tricky thing to work with as a return value. You can essentially only iterate an IEnumerable, and at the time of calling a function, maybe that’s not what you want to do. The flip side is that for the person implementing the interface, IEnumerable<T> is a really easy interface to satisfy. However, the yield keyword has opened up some new doors.

In this article, I’d like to go over a couple of different approaches for an API and then explain why the yield keyword might be something you consider next time around. Disclaimer: I’m not claiming anything I’m about to present is the only way or the best way–I’m just sharing some of my own findings and perspective.

Interface For Returning Collections

The first type of API I’d like to look at is for returning collections. Based on my own API guidelines, I’d ideally choose an interface or class to return that provides a lot of information to the caller that is also easy to create for the implementer of my interface. The List<T> class is a great choice:

  • It’s easy to construct
  • It’s built-in to the .NET framework
  • It provides many handy functions (All of the IList<T> functionality as well as things like AddRange(), or functions that support delegates)

My next choice might be to have a return type of IList<T>, which would provide a little less ease of use to the caller, but make it even easier for the implementer of the interface. They could return arrays of type T, since an array implements the IList<T> interface, or their own custom list implementation that doesn’t inherit from the List<T> class. The differences between using IList<T> and List<T> are arguable pretty small.

A third alternative, which I would have avoided in the past, is to return an IEnumerable<T>. My opinion used to be that this made the life of the interface implementer a bit easier compared to returning an IList<T>, but complicated the life of the caller for a couple of reasons:

  • The caller would have to use the results of the function in a foreach loop.
  • The caller would have to add the items to their own collection to be able to do much more with the items.

My naive implementations of being forced to return an IEnumerable<T> were… well… crap. I would have constructed a collection within the function, fill it up, and then return it as an IEnumerable<T>. Then as the caller of my function, I’d have to re-enumerate the results (or add it to another collection):

public static IEnumerable<T> GetItems()
{
  var collection = new List<T>();
  // add all the items to a collection
  return collection;
}

private static void Main()
{
  var myCollection = new List<T>();
  myCollection.AddRange(GetItems());
  // use myCollection...

  // or.....
  foreach (var item in GetItems())
  {
    // use the items
  }
}

Seems like overkill to me with that implementation. However, we’ll examine how using yield can truly transform this into something… better. So to reiterate, a few potential implementations for an API involving collections might be:

  • Return a List<T> class
  • Return an IList<T> (or even an ICollection<T>) interface
  • Return an IEnumerable<T> interface

Constantly Creating Collections

My design decisions, in the past, were really driven by two guidelines:

  • Make it easier for the person implementing/extending the API
  • Make it easy for the person consuming the API

As I quickly illustrated in the first section, this meant that I would have a method where I would create a collection, fill it with items, and then return it. I could generally pick any concrete collection class and return it since I would usually pick a simple collection as the return type. Easy.

One thing that might be noticeable with this approach is that it looks pretty inefficient to keep creating new collections, fill them, and then return them. I’ll illustrate with a simple example. We’ll create a class that has a method on it called GetItems(). As per my reasoning presented earlier, we’ll have this method return a List<T> instance, and to make this example easier to work with, we’ll pass in an IEnumerable<T> instance. For what it’s worth, the input to this function is really just for demonstration purposes here–We’re really focusing on how we’re creating our return value.

public class CreateNewListApi<T>
{
  public List<T> GetItems(IEnumerable<T> input)
  {
    var newCollection = new List<T>();

    foreach (var item in input)
    {
      newCollection.Add(item);
    }

    return newCollection;
  }
}

And now that we have our simple class we can mock up a little test for performance… Just how inefficient is creating new lists every time?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Creating New Collections");
    var api = new CreateNewListApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 1.73 seconds. The memory printout I get when running is 1615908864 bytes. So is that slow? Is that a lot of memory usage? I think it’s pretty hard to say conclusively without being able to compare it against anything. So let’s keep this number in mind as we continue to investigate the alternatives.

Side Note: At this point, some readers may be saying “Well, if the input to our function was also a list (or if whatever our function has to work with was otherwise equivalent to our return value) then we wouldn’t have to go populate a new collection every time… We can just return the underlying collection”! And I would say you are absolutely correct. If your function has access to an instance of the same type as the return type, then you could always just return that instance. But what implications does this have? You’re now giving people access to your underlying internals, and they can go modify them as they please. So, if you need to control access to items being added or removed, then it might not make sense for you to expose your internal collections like this.

Yield to Incoming API Alternatives

We’ve seen how my past implementations may have looked, so how might we tweak this? If we tweak our API a bit, we can make our method return an IEnumerable<T> instead. Let’s see what that might look like:

public class YieldingApi<T>
{
  public IEnumerable<T> GetItems(IEnumerable<T> input)
  {
    foreach (var item in input)
    {
      yield return item;
    }
  }
}

So in this API implementation, all we’ll be doing is iterating over some type of collection and then yielding each result. If we run it through the same type of test as out previous API implementation, what kind of results do we end up with?

internal class Program
{
  private static void Main(string[] args)
  {
    const int NUM_ITEMS = 100000000;
    var inputItems = new int[NUM_ITEMS];

    Console.WriteLine("API Yielding");
    var api = new YieldingApi<int>();

    var watch = Stopwatch.StartNew();
    var results = api.GetItems(inputItems);

    foreach (var item in results)
    {
    }

    Console.WriteLine(watch.Elapsed);
    Console.WriteLine(Process.GetCurrentProcess().PrivateMemorySize64);
    Console.ReadLine();
  }
}

When I run this on my machine, I get an average of about 2.80 seconds. The memory printout I get when running is 449409024 bytes. How does this relate back to our first implementation? Well, it’s certainly slower. It takes about 1.62x as long to enumerate using the yield implementation as it did with the first API we created. However, yield also uses less than 1/3 (about 27.8%, actually) of the memory footprint when compared to the first implementation. Pretty cool results!

Site Note: So yield was a bit slower according to our results, but what happens if print the elapsed time before we run that foreach loop? Well, on my machine it averages at about one millisecond. Now that’s fast, right?! The cool thing about using yield with the IEnumerable<T> interface is that the work is deferred. That is, not until the program goes to actually run the enumeration do we get our performance hit. Try it out! Try moving the time printout from after the foreach loop to before the foreach loop. Try sticking breakpoints in on the line that yields. You’ll see what I mean.

Summary

In this article, I’ve explored two different ways of implementing an API (specifically focusing on the return value). We saw a brief performance analysis between the two and I highlighted some differences in both approaches. Let’s recap though:

  • Approach 1: Returning a List<T> and creating the collection ahead of time
    • Appeared to be overall a bit faster then yielding.
    • Consumed much more memory than yielding.
    • Callers can use the results immediately for enumeration, checking count, or as a collection to add more things to
    • The return type of List<T> is a bit more restrictive than an IEnumerable<T> like in the second API implementation
  • Approach 2: Return type of IEnumerable<T> and yielding results
    • Appeared to be overall a bit slower than the List<T> implementation
    • Lazy. We don’t actually execute any enumeration code until the caller actually enumerates
    • Consumed significantly less memory than the first approach using List<T>
    • Callers can enumerate the results immediately, but they need to add the results to a collection class to do much more than enumerate

So next time you’re designing an API for your interfaces and classes, try keeping these things in mind!

EDIT (December 30th, 2013):
As per some comments on Google+ by Dan Nemec, I figured I’d add a bit more here in the summary. IEnumerable<T> on it’s own is certainly not useless, especially if you’re leveraging LINQ or extension methods. My main beef in the past was that the consumer of an API with a IEnumerable<T> return value can only iterate over the results… And that’s just because that’s all that IEnumerable<T> lets you do. Dan made a great point though–If you are leveraging things like extension methods, or LINQ (which introduces tons of handy extension methods for working with IEnumerable<T>) then you get all of that functionality tacked on to IEnumerable<T>.

So if you’re not fortunate enough to be working with LINQ or extension methods (i.e. working with legacy code in old .NET framework versions… and yes I am familiar with the attribute you can add in to allow extension methods provided you have a compiler version high enough to support it), then IEnumerable<T> sometimes just plain sucks. I’d wager the majority of C# developers aren’t in this boat though, so I’d like to thank Dan again for his comments.


Performance Reviews – Weekly Article Dump

Performance Reviews - Weekly Article Dump (Image by http://www.sxc.hu/)

Performance Reviews

It’s almost the end of the year, and performance reviews for many companies are just around the corner. This will be the first time for me sitting on the other side of a performance review. I’m excited, and to be honest, a little nervous about how it will all play out. I know our HR manager has done an excellent job putting together our initial take on performance reviews, but it’s still going to be up to me to ensure that all aspects of a performance review are communicated properly to my team. It’s definitely going to be an interesting time of year!

I’ve started doing a little bit of reading on performance reviews. From what I can tell, the general consensus is that most performance review systems are flawed and nobody knows the perfect way to do them. That’s kind of scary actually. So, like anything, I started questioning all the aspects of performance reviews that I can think of. So things like: What’s stack ranking? Why do companies stack rank? What are alternatives? What about leveraging teammate-driven reviews? etc… There’s a whole lot for me to learn, so I need to start by questioning everything.

With that said, how do you do performance reviews? Have performance reviews been working at your organization? Do you stick to “the norm”, or do you have your own interesting spin on performance reviews that make them effective for your organization?

Articles

  • Employee retention is not just about pizza lunches and parties: On the surface, things like candy stashes, catered lunch, and all other shiny perks seem like a great way to get and keep employees. However, keeping employees engaged is the sum of what attracts them to the company and what keeps them motivated while they’re working. Recognizing their accomplishments and giving them challenging and meaningful work is an awesome start.
  • 7 Reasons Your Coworkers Hate You: The truth? You probably know at least one person at work who does at least one of the things on this list. The harder truth? You probably do one of these yourself. It’s a pretty cool list put together by Ilya Pozin. I’d suggest a quick look!
  • How To Inspire Your Team on a Daily Basis: In this article by James Caan, he echoes one of the things I wrote about recently. You can’t expect to have a motivated team unless you lead by example. You really shouldn’t expect anything from your tea unless you are going the lengths to demonstrate that your dedication to the team and the team’s goal.
  • humility = high performance and effective leadership: Michelle Smith write about how humility is actually a great leadership characteristic. A couple of the top points in her article include not trying to obtain your own publicity and acknowledge the things you don’t know. The most important, in my opinion, is promoting a spirit of service. You lead because you are trying to provide the team guidance and ensure every team member can work effectively.
  • The Surprising Reason To Set Extremely Short Deadlines: This one might not be the same for all people. I think that anyone that tries to apply this as a blanket statement is probably setting themselves up for failure. How do you feel about short deadlines? Some people tend to work really well under pressure and having short deadlines. For those that do, this article offers a perspective on why. Under pressure, you operate creatively given your restricted set of resources, and you don’t have time to dawdle and let things veer of track. Interesting to read.
  • Eliciting the Truth: Team Culture Surveys: Gary Swart talks about something I think is extremely important for all businesses. Maybe your work culture is established, but where did it come from? It’s easy for people to get together in a room and say “we want to have a culture that looks like X”. It’s harder to actually have the culture you say you want. Gary suggests you do a culture survey to actually see what your work culture is like because… well, who knows better? A few people sitting together in a room, or everyone in the company?
  • You Are Not a Number: With year-end performance reviews and the like coming up, I thought it would be interesting to share this short article by Dara Khosrowshahi. Do you stick to stack ranking? Do you have in-depth conversations with employees about their performance? Have you tried switching things up because the canned approach just wasn’t delivering?
  • Which Leads to More Success, Reward or Encouragement?: Deepak Chopra analyzes the positives and negatives of using rewards and using encouragement as a means of driving success. The takeaway from Deepak’s article is that using rewards is not a sustainable means to motivate your team, and actually tends to create separation within the team. By leveraging encouragement, you can empower your team to work together and self-motivate.

Remember to follow Dev Leader on social media outlets to get these updates through the week.

Nick Cosentino – LinkedIn
Nick Cosentino – Twitter
Dev Leader – Facebook
Dev Leader – Google+
Nick’s CodeProject Articles


Halloween – Weekly Article Dump

Halloween at Magnet Forensics

Happy Halloween

Happy Halloween, everyone! I hope those of you who were out and about with your own little ghouls and ghosts had a safe Halloween this year.

Halloween costumes were pretty creative again this year at Magnet Forensics. I tried going with my own Horse Lime attempt, but it’s difficult when not many people know what the Horse Lime actually is. Regardless, my awesome mother put together the lime portion of my costume, and I was extremely grateful for that (and yes, I’m in my mid 20’s. No judging). I think it turned out pretty damn good.

This year, Saige won our Halloween costume contest. As Old Gregg, it was hard for that to not be a sure-fire win. Complete with Bailey’s in hand, I think the only thing that could have made it better was a set of watercolours to go with it. Absolutely awesome job.

On behalf of Team Magnet, Happy Halloween!

Articles

  • Kenneth Cole’s 10 Keys to Success: In this article by Teresa Rodriguez, we get a list of 10 tips from Kenneth Cole on success. While I don’t think there’s anything groundbreakind about them, I do think they’re all relatively straight forward. My main take aways are being innovative, being passionate about what you do, and create value. This article also has a bit of background on Mr. Cole that I wasn’t even aware of, so that was pretty interesting.
  • Community is Everything: How to Build Your Tribe: This article was kind of unique. It doesn’t necessarily apply directly to startups or business, but I see lots of parallels. Miki Agrawal writes about creating a “tribe” or a community of people around you. It’s really about placing positive people in your life, or go-getters in your business for the parallel. Again, no monumental secret tips in here, but it’s a great topic.
  • Performance Recognition: Cutting the Cost of Disengagement: This one is an infographic (and not really an article) about engagement and performance recognition. There are a lot of stats in there, but regardless of whether or not I trust the accuracy, I think the general points made are sound. Essentially, there are a lot of disengaged employeed in the global work force and it hurts productivity. By creating a culture of recognizing performance, you can help boost engagement which has all sorts of positive effects.
  • Code Review Like You Mean It: The first programming article for this week. Phil Haack discusses how to actually code review effectively. One of the key topics is taking breaks from long code reviews so you can maintain focus. Another is forgetting about the author when reviewiing and focusing solely on the code. Phil even put up his own code review checklist and suggests you have your own. Personally, I think I’ve kept a mental one but it probably would help to have it solidified.
  • Converse, Don’t Complain: This article by Hiroshi Mikitani had the most buzz from the things I shared this week. It really seemed to hit home with people, and I imagine it’s for a couple reasons. First of all, if you’re honest with yourself, you probably complain. You probably chat with at least one colleague you’re really close with and just flat out complain with them. You both don’t like something, so you vent. That’s definitely a comforting activity, and sometimes we need it. The flip side is you have authority or responsibility over something that people have problems with. Nobody is voicing any concerns to you (since they are just complaining among themselves) and if they are, there aren’t solutions being brought forward.The first of this two part solution to this is instead of whining, start coming up with potential solutions. It doesn’t matter how big or small your ideas are, start thinking about what a solution might be. The second part is communication. If you want something to get resolved, you need to bring your concerns with potential solutions forward. If you only complain and vent to one person, your concerns won’t be heard. If you only ever whine about something not being correct, then you’re doing a half-assed job at trying to come to a solution.
  • Lead by Example and Emulate Ideal: This one is a plug for my own article. I decided I’d write about why leading by example is actually more powerful than some people think. You have a lot of eyes on you as a leader, and you may not realize it. By leading by example and emulating the attributes you consider ideal, people will catch on to it.
  • Keys to Productivity: I’ve sort of noticed this through my own experiences so far, but early morning and late at night are great times to be productive. When there are a lot of stresses on you during the day, sometimes it feels like you’re not being productive. It doesn’t necessarily mean that you aren’t, but it’s your own perception. Getting a head start on the ay by getting into the office early, or staying up late for your own creative endeavours can prove to feel really rewarding.
  • Build Trust Through Training, Transparency and Trials: I’ve shared articles from this series by Jake Wood before, but this is another standout one for me. Trust isn’t something you can just put into your company values or your mission statement. Trust is something you have to live out each and every day in your organization. We can all say we value it, but if you aren’t willing to live it out, then it’s not something you truly value. One quote I really like from the article is:

    Transparency cannot happen unless your leadership regularly visits the “front lines,” wherever that may be in your business.

  • Here Is What Smart Companies Get That Others Don’t: The first of the three points offered in this article is that smart companies think differently. They are leaders and not followers when it comes to everything they do. The second is that they sell their culture. Their culture is actually core to their business and their organization, not some after thought. The third is that they help others become smarter. Provide value and become something that other people and business want and need to use.
  • Why Good Strategies Often Fall Apart: Ron Ashkenas highlights a few reasons why strategies that look great sometimes just don’t work. The first two points he makes in his article are the ones I want to highlight. The first is passive aggressive disagreement. Not everyone is going to be on board with all parts of all changes, so you’re going to have people that disagree. If the culture does not actively embrace people being able to voice their concerns, it’s difficult to carry out a successful strategy. Individuals might complain, but they wont end up doing anything about it. The second is something along the lines of “being too nice”. Trying to avoid confrontation because you’re afraid of it is a recipe for disaster. If you actually encourage open communication and trust, then being able to have hard discussions about something can be really powerful.
  • Three Things that Actually Motivate Employees: This probably isn’t new to a lot of people, but money (after a certain point) isn’t the driving force for employee motivation. The three things outline in this article are mastery, membership, and meaning. Employees want to be able to mastery their skill sets, learn, and get better at the things they do. Individuals within the organization want to have a sense of community. They want to feel like they align with the people they work with and their working toward a common goal. Employees also want to work on something that has meaning. Work that has a large positive impact is extremely motivating.

Happy Halloween! Remember to follow Dev Leader on social media outlets to get these updates through the week.

Nick Cosentino – LinkedIn
Nick Cosentino – Twitter
Dev Leader – Facebook
Dev Leader – Google+
Nick’s CodeProject Articles


  • Nick Cosentino

    Nick Cosentino

    I work as a team lead of software engineering at Magnet Forensics (http://www.magnetforensics.com). I'm into powerlifting, bodybuilding, and blogging about leadership/development topics over at http://www.devleader.ca.

    Verified Services

    View Full Profile →

  • Copyright © 1996-2010 Dev Leader. All rights reserved.
    Jarrah theme by Templates Next | Powered by WordPress