Tag: Thalmic

MyoSharp – Update On The Horizon

MyoSharp

If you haven’t checked it out already, my friend Tayfun and I created an open source C# wrapper for Thalmic’s Myo. It’s hosted on GitHub over here, so you can browse and pull down code whenever you want. We’ve had some great feedback from users of our API, so we continue to welcome it (both positive and negative!) in order to improve the usability.

Thalmic has plans to release a firmware update to allow more data to be accessible through their API. Right now, MyoSharp is a bit out of date, but once this big firmware update lands we’ll take some more time to get it up to date again. Remember, it’s open source so you can feel free to contribute!

Troubleshooting

The most common question I receive is “I keep getting an exception about not being able to connect when I run the sample code”. I’ve tried to help a few people through this so I just figured I’d mention it right here for clarity: It’s more than likely that your MyoConnect version and the version we packaged with MyoSharp have become out of date. You probably keep your Myo SDK more up to date than MyoSharp is.

Don’t worry! So far we’ve had reasonably good luck just replacing the Myo DLLs in the x86 and x64 folder of the solution. Provided Thalmic didn’t break any API compatibility, things should actually just work out of the box. If they *DID* break backwards compatibility, it’s likely not that big of a deal either. You can update the PInvokes used to match the signatures they expect, and again, you should be up and running pretty quickly.

With that said, hold tight! We’ll get something updated soon. If you can’t wait, then that’s my suggestion for how to get up and running. Please don’t hesitate to contact Tayfun or myself for troubleshooting. Just post in the comments here and we can try to help out!


Controlling a Myo Armband with C#

Controlling a Myo Armband with C#

Background

Thalmic Labs has started shipping their Myo armband that allows the wearer’s arm movements and gestures to control different pieces of integrated technology. How cool is that? My friend and I decided we wanted to give one a whirl and see what we could come up with. We’re both C# advocates, so we were a bit taken back when we saw the only C# support in the SDK was made for Unity. We decided to take things into our own hands and open source a Myo C# library. We’re excited to introduce the first version of MyoSharp!

The underlying Myo components are written in C++, and there’s only several functions that are exposed from the library that we can access. In order to do this, we need to leverage platform invocation (PInvokes) from C# to tap into this functionality. Once you have the PInvokes set up you can begin to play around!

The Workflow

Getting setup with the Myo is pretty straightforward, but it wasn’t obvious to us right away. We didn’t have anyone to walk us through how the different components were supposed to work together (just some good ol’ fashioned code) so we had to tinker around. Once we had everything mapped out, it was quite simple though.

  1. The first step is opening a communication channel with the Bluetooth module. You don’t need to worry about the implementation here since it’s all done in C++ by the Thalmic devs. Calling the correct methods using PInvokes from C# allows us to tap into a stream of “events” that come through the Bluetooth module.
  2. Now that we can intercept events, we need to be able to identify a Myo. After all, working with Myos is our main goal here! There’s a “pair” event that we can listen to from the Bluetooth module that notifies us of when a Myo has paired and provides us a handle to the device. This handle gets used for identifying events for a particular Myo or sending a particular Myo messages.
  3. There’s a connect event that will fire when a Myo connects after it’s been paired with the Bluetooth module. A Myo can be paired but disconnected.
  4. Now that we can uniquely identify a Myo, the only things we need to do are intercept events for a particular Myo and make sense of the data coming from the devices! Orientation change? Acceleration change? There’s a host of information that the device sends back, so we need to interpret it.
  5. When a Myo disconnects, there’s an event that’s sent back for that as well.

Getting Started with MyoSharp

I’m going to start this off with some simple code that should illustrate just how easy it is to get started with MyoSharp. I’ll describe what’s going on in the code immediately after.


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you the basics for setting up and working with
    /// a Myo using MyoSharp. Primary communication with the device happens
    /// over Bluetooth, but this C# wrapper hooks into the unmanaged Myo SDK to
    /// listen on their "hub". The unmanaged hub feeds us information about
    /// events, so a channel within MyoSharp is responsible for publishing
    /// these events for other C# code to consume. A device listener uses a
    /// channel to listen for pairing events. When a Myo pairs up, a device
    /// listener publishes events for others to listen to. Once we have access
    /// to a channel and a Myo handle (from something like a Pair event), we
    /// can create our own Myo object. With a Myo object, we can do things like
    /// cause it to vibrate or monitor for poses changes.
    /// </summary>
    internal class BasicSetupExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.Vibrate(VibrationType.Short);
                    e.Myo.PoseChanged += Myo_PoseChanged;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.PoseChanged -= Myo_PoseChanged;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_PoseChanged(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo detected {1} pose!", e.Myo.Arm, e.Myo.Pose);
        }
        #endregion
    }
}

In this example, we create a hub instance. A hub will manage a collection of Myos that come online and go offline and notify listeners that are interested. Behind the scenes, the hub creates a channel instance and passes this into a device listener instance. The channel and device listener combination allows for being notified when devices come online and is the core of the hub implementation. You can manage Myos on your own by completely bypassing the Hub class and creating your own channel and device listener if you’d like. It’s totally up to you.

In the code above, we’ve hooked up several event handlers. There’s an event handler to listen for when Myo devices connect, and a similar one for when the devices disconnect. We’ve also hooked up to an instance of a Myo device for when it changes poses. This will simply give us a console message every time the hardware determines that the user is making a different pose.

When devices go offline, the hub actually keeps the instance of the Myo object around. This means that if you have device A and you hook up to it’s PoseChanged event, if it goes offline and comes back online several times, your event will still be hooked up to the object that represents device A. This makes managing Myos much easier compared to trying to re-hook event handlers every time a device goes on and offline. Of course, you’re free to make your own implementation using our building blocks, so there’s no reason to feel forced into this paradigm.

It’s worth mentioning that the UserInputLoop() method is only used to keep the program alive. The sample code on GitHub actually lets you use some debug commands to read some Myo statuses if you’re interested. Otherwise, you could just imagine this line is replaced by Console.ReadLine() to block waiting for the user to press enter.

Pose Sequences

Without even diving into the accelerometer, orientation, and gyroscope readings, we were looking for some quick wins to building up on the basic API that we created. One little improvement we wanted to make was the concept of pose sequences. The Myo will send events when a pose changes, but if you were interested in grouping some of these together there’s no way to do this out of the box. With a pose sequence, you can declare a series of poses and get an event triggered when the user has finished the sequence.

Here’s an example:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user has performed a series of poses. A 
    /// <see cref="PoseSequence"/> can monitor a Myo for a series of poses and
    /// notify you when that sequence has completed.
    /// </summary>
    internal class PoseSequenceExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // for every Myo that connects, listen for special sequences
                    var sequence = PoseSequence.Create(
                        e.Myo, 
                        Pose.WaveOut, 
                        Pose.WaveIn);
                    sequence.PoseSequenceCompleted += Sequence_PoseSequenceCompleted;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Sequence_PoseSequenceCompleted(object sender, PoseSequenceEventArgs e)
        {
            Console.WriteLine("{0} arm Myo has performed a pose sequence!", e.Myo.Arm);
            e.Myo.Vibrate(VibrationType.Medium);
        }
        #endregion
    }
}

The same basic setup occurs as the first example. We create a hub that listens for Myos, and when one connects, we hook a new PoseSequence instance to it. If you recall how the hub class works from the first example, this will hook up a new pose sequence each time the Myo connects (which, in this case, isn’t actually ideal). Just for demonstration purposes, we were opting for this shortcut though.

When creating a pose sequence, we only need to provide the Myo and the poses that create the sequence. In this example, a user will need to wave their hand out and then back in for the pose sequence to complete. There’s an event provided that will fire when the sequence has completed. If the user waves out and in several times, the event will fire for each time the sequence is completed. You’ll also notice in our event handler we actually send a vibrate command to the Myo! Most of the Myo interactions are reading values from Myo events, but in this case this is one of the commands we can actually send to it.

Held Poses

The event stream from the Myo device only sends events for poses when the device detects a change. When we were trying to make a test application with our initial API, we were getting frustrated with the fact that there was no way to trigger some action as long as a pose was being held. Some actions like zooming, panning, or adjusting levels for something are best suited to be linked to a pose being held by the user. Otherwise, if you wanted to make an application that would zoom in when the user makes a fist, the user would have to make a fist, relax, make a fist, relax, etc… until they zoomed in or out far enough. This obviously makes for poor usability, so we set out to make this an easy part of our API.

The code below has a similar setup to the previous examples, but introduces the HeldPose class:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it's useful to know
    /// when a user is still holding a pose and not just that they've 
    /// transitioned from one pose to another. The <see cref="HeldPose"/> class
    /// monitors a Myo and notifies you as long as a particular pose is held.
    /// </summary>
    internal class HeldPoseExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // setup for the pose we want to watch for
                    var pose = HeldPose.Create(e.Myo, Pose.Fist, Pose.FingersSpread);

                    // set the interval for the event to be fired as long as 
                    // the pose is held by the user
                    pose.Interval = TimeSpan.FromSeconds(0.5);

                    pose.Start();
                    pose.Triggered += Pose_Triggered;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Pose_Triggered(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo is holding pose {1}!", e.Myo.Arm, e.Pose);
        }
        #endregion
    }
}

When we create a HeldPose instance, we can pass in one or more poses that we want to monitor for being held. In the above example, we’re watching for when the user makes a fist or when they have their fingers spread. We can hook up to the Triggered event on the held pose instance, and the event arguments that we get in our event handler will tell us which pose the event is actually being triggered for.

If you take my zoom example that I started describing earlier, we could have a single event handler responsible for both zooming in and zooming out based on a pose being held. If we picked two poses, say fist and fingers spread, to mean zoom in and zoom out respectively, then we could check the pose on the event arguments in the event handler and adjust the zoom accordingly. Of course, you could always make two HeldPose instances (one for each pose) and hook up to the events separately if you’d like. This would end up creating two timer threads behind the scenes–one for each HeldPose instance.

The HeldPose class also has an interval setting. This allows the programmer to adjust the frequency that they want the Triggered event to fire, provided that a pose is being held by the user. For example, if the interval is set to be two seconds, as long as the pose is being held the Triggered event will fire every two seconds.

Roll, Pitch, and Yaw

The data that comes off the Myo can become overwhelming unless you’re well versed in vector math and trigonometry. Something that we’d like to build up and improve upon is the usability of data that comes off the Myo. We don’t want each programmer to have to write similar code to get the values from the Myo into a usable form for their application. Instead, if we can build that into MyoSharp, then everyone will benefit.

Roll, pitch, and yaw are values that we decided to bake into the API directly. So… what exactly are these things? Here’s a diagram to help illustrate:

Roll, Pitch, and Yaw - MyoSharp

Roll, pitch, and yaw describe rotation around one of three axes in 3D space.

The following code example shows hooking up to an event handler to get the roll, pitch, and yaw data:


using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you how to hook onto the orientation events on
    /// the Myo and pull roll, pitch and yaw values from it.
    /// </summary>
    internal class OrientationExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.OrientationDataAcquired += Myo_OrientationDataAcquired;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.OrientationDataAcquired -= Myo_OrientationDataAcquired;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_OrientationDataAcquired(object sender, OrientationDataEventArgs e)
        {
            Console.Clear();
            Console.WriteLine(@"Roll: {0}", e.Roll);
            Console.WriteLine(@"Pitch: {0}", e.Pitch);
            Console.WriteLine(@"Yaw: {0}", e.Yaw);
        }
        #endregion
    }
}

Of course, if we know of more common use cases that people will be using the orientation data for, then we’d love to bake this kind of stuff right into MyoSharp to make it easier for everyone.

Closing Comments

That’s just a quick look at how you can leverage MyoSharp to make your own C# application to work with a Myo! As I said, MyoSharp is open source so we’d love to see contributions or ideas for suggestions. We’re aiming to provide as much base functionality as we can into our framework but designing it in a way that developers can extend upon each of the individual building blocks.


Hack The North

Hack The North

Hack The North… What The Hack Is That?

Hack The North is Canada’s largest international hackathon. It’s big. It’s bad. It’s awesome. Okay, but what does that even mean?

The idea is that 1000 people get together from all over the world in an event where they’re given 36 hours to create amazing technology hacks. There’s a lack of sleep but no lack of amazing ideas and hacked together proof of concepts that show incredible innovation. Sponsors are present to hand out prizes for best usage of their product or API to competitors as well as mentor them and provide help with problem solving. This year, Hack The North was hosted at The University of Waterloo.

Bro, Do You Even Hack?!

This was my first time ever at a hackathon. I’ve participated in the Ontario Engineering Competition (OEC) which is much smaller scale event with a similar structure–A bunch of students get together and have to come up with a design to solve a problem. I even ran OEC in 2010 with some schoolmates (you might know them at Thalmic Labs) which was a blast. However, this weekend I was sitting on the other side of the table.

I got to hang out with a group of my colleagues at Hack The North at the Magnet Forensics booth. We had a great time representing our company and meeting tons of incredible students from all over the world. We provided assistance where we could with some of the challenges that students were facing, and spoke to them about the important role that our software plays in the life of a forensic examiner/investigator.

What’d You See There?

When I first showed up at Hack The North, it was around 9:00 on Friday night. I got there just in time for the opening ceremonies, which I thought were great. They had live Q&A with a former Facebook employee who has “made it” in the software industry and intros from some of the major sponsors at the event.

Hack The North - Opening Ceremonies

Everyone getting settled for the opening ceremonies at Hack The North.

From that point on, I spent the rest of the time hanging around the Magnet booth… But that doesn’t mean I didn’t see anything awesome. We had people from all over the place coming by to talk to us and pick up some Magnet swag. We handed out an incredible amount of stress balls and couldn’t seem to keep a sufficient supply of them coming. We had our stickers showing up on everything from fuzzy viking hats to laptops to shirts to megaphones. You name it, our stickers made it onto it.

Hack The North - Magnet Viking

Akshay Joshi decorated his hat with Magnet stickers!

Hack The North - Magnet Megaphone

One of the Hack The North organizers was sporting a Magnet sticker on his megaphone.

On the last day of Hack The North, we had participants coming up to our booth to demonstrate some of their awesome hacks. We got to see how Thalmic’s Myo was being incorporated or how other vendors’ APIs were being leveraged to do some really awesome things. We were really impressed with some of the things we saw.

What’s Next?

I think even after the first night of being at Hack The North I was trying to think of what we could do next time or if we even wanted to come back… The answer to the latter was quickly an “absolutely yes!”, so I’m pretty confident we’ll be making an appearance at Hack The North again. There were so many great people at this even that we spoke with that it would be silly not to go back.

I think next year we’d like to participate even more. We learned a lot about the different ways that we could get involved, so things like speaking sessions or workshops would be awesome to get people involved with. We’ll definitely have more stress balls and unique giveaways to please the masses!

Thanks to everyone who made it out to Hack The North. Stay in touch with us! We’d love to hear more from you.

Hack The North - Kelly and Nick

Kelly and I with our #Truth T-shirts and other Magnet gear on at our booth!


Deloitte Companies to Watch – Weekly Article Dump

Deloitte Companies to Watch - Weekly Article Dump

Deloitte Companies to Watch

Another impressive accolade for Magnet Forensics! Deloitte has placed Magnet on their top 10 companies to watch list! To qualify for the list, the companies need to be operating for less than five years, be based out of Canada, and put a large portion of their revenue to generating intellectual property. Our CEO, Adam Belsher, had this to say about the award:

“We are honoured to be named one of Deloitte’s Companies-to-Watch. This award recognizes the hard work and dedication of our team. We’re thankful for the success we’ve achieved, and we’re incredibly proud to be contributing to the important work done by our customers who use our solutions to fight crime, enhance public safety, protect companies from fraud and theft, and ensure workplace safety and respect for their employees.”

Magnet Forensics Press & Events

The event was put together very well. It was great to be able to interact with individuals from the other companies and share success stories. I even had a chance to meet up with Stephen Lake of Thalmic Labs and have a good chat with him. I’m going to name-drop him everywhere I go because he’s my old university room/house mate! He also happens to be a incredible person that if you have the chance to meet, you absolutely should. Here’s some coverage on twitter of us talking with our founder Jad Saliba:

We enjoyed the whole night, and we were grateful for Deloitte putting on the event. The entire Magnet team is very proud of our achievements.

Articles

  • What comes first: employee engagement or great work?: A short but interesting article on employee engagement. The author claims that most employees probably start of at their position very motivated and engaged. Over time, an employees engagement drops if their leaders are not proactive in keeping their engagement levels up. By proactively acknowledging the success of your employees, you can keep your team engaged and producing great work.
  • Great Leadership Starts and Ends with This: Jeff Haden put together this quick little article about an answer an audience member gave about what the key to leading people is. Caring. Overly simplified? Well, the individual went on to say that regardless of all of your strategies, plans, and experience, if you can’t prove that you truly care about your team then they aren’t going to buy in. I’m never one to buy into something so absolute, but the takeaway for me is that team members cannot be looked at entirely as resources. Sure, the people on your team affect productivity and in that sense are resources, but forgetting to acknowledge the human side of things is a recipe for failure.
  • 9 Ways to Win Employee Trust: In his article, Geoffrey James put together a great list of nine things to help build trust with your team. I wouldn’t say these are groundbreaking things, but it’s important to be reminded about them. Try reflecting on his list and seeing if you actually do the things you think you do. You might be surprised. Some of the top things on the list for me are ensuring employees’ success is number one on your priorities, listen more than you talk, and walk the walk. Great list!
  • Lambdas: An Example in Refactoring Code: I put out this programming article earlier this week and had some great feedback. In this article, I talk about a real world example of how using lambda expressions in C# really helped when refactoring a piece of code. Some people have never heard of lambdas, and some people seem to hate them. In this case for me, it greatly simplified a set of code and reduced a bunch of extra classes. I definitely owe it to myself to start investigating them a little bit more.
  • Executive Coaching: Bringing Out Greater Leadership: This article is all about taking charge with your leadership. Judith Sherven talks about executive coaching for leaders, but the main points I see in here are around confidence. If you aren’t confident in your ability to lead, motivate, and inspire how can you expect anyone else to be? It ends up becoming a tough balance, because you need to listen and take feedback from your team, but when you make decisions you should do so with confidence.
  • Stop Worrying About Making the Right Decision: Ever heard of paralysis by analysis? This article does a great job of explaining why you shouldn’t let that creep in to your leadership approach. It’s important to make good decisions–there’s no doubt about that. But the reality is that no matter what decision you make, there are certain unknowns that can creep in and potentially have a huge effect on the choices you’ve made. So what’s more important: making the perfect decision, or being able to adapt effectively?
  • Appraising Performance Appraisal: Steven Sinofsky‘s article is a bit of a beast, but it’s a great starting point if you’re reconsidering performance appraisals. Even if you’re totally content with your performance review system, it might be worth reading to spark some ideas. Steven does a great job of pointing out some common pitfalls of typical performance appraisal systems and comments on some things you really need to try and understand before sticking to any one system. I’m not well enough versed in the performance review and/or human resources side of things, but this article certainly has enough to get you questioning the common approaches.
  • Tab Fragment Tutorial: Shameless plug for my Android application that I put out on the Google Play store. It’s the end result of the tutorial I wrote up over here. I think it’s going to blow past my legitimate application for converting units!
  • Does a Good Leader Have To Be Tough?: Deepak Chopra has some seriously great articles. In this article, he analyzes the pros and cons of being a “tough” leader. In short, there are positive takeaways from being a tough leader, but there are a lot of negative aspects to it. Deepak suggests you consider a different approach from tough-soft leadership. By focusing on a hierarchy of needs to be a successful leader, toughness is only one aspect of leadership. A pretty solid read, like all of Deepak’s articles.

Remember to follow Dev Leader on social media outlets to get these updates through the week.

Nick Cosentino – LinkedIn
Nick Cosentino – Twitter
Dev Leader – Facebook
Dev Leader – Google+
Nick’s CodeProject Articles


  • Nick Cosentino

    Nick Cosentino

    I work as a team lead of software engineering at Magnet Forensics (http://www.magnetforensics.com). I'm into powerlifting, bodybuilding, and blogging about leadership/development topics over at http://www.devleader.ca.

    Verified Services

    View Full Profile →

  • Copyright © 1996-2010 Dev Leader. All rights reserved.
    Jarrah theme by Templates Next | Powered by WordPress