Controlling a Myo Armband with C#

Controlling a Myo Armband with C#

Background

Thalmic Labs has started shipping their Myo armband that allows the wearer’s arm movements and gestures to control different pieces of integrated technology. How cool is that? My friend and I decided we wanted to give one a whirl and see what we could come up with. We’re both C# advocates, so we were a bit taken back when we saw the only C# support in the SDK was made for Unity. We decided to take things into our own hands and open source a Myo C# library. We’re excited to introduce the first version of MyoSharp!

The underlying Myo components are written in C++, and there’s only several functions that are exposed from the library that we can access. In order to do this, we need to leverage platform invocation (PInvokes) from C# to tap into this functionality. Once you have the PInvokes set up you can begin to play around!

The Workflow

Getting setup with the Myo is pretty straightforward, but it wasn’t obvious to us right away. We didn’t have anyone to walk us through how the different components were supposed to work together (just some good ol’ fashioned code) so we had to tinker around. Once we had everything mapped out, it was quite simple though.

  1. The first step is opening a communication channel with the Bluetooth module. You don’t need to worry about the implementation here since it’s all done in C++ by the Thalmic devs. Calling the correct methods using PInvokes from C# allows us to tap into a stream of “events” that come through the Bluetooth module.
  2. Now that we can intercept events, we need to be able to identify a Myo. After all, working with Myos is our main goal here! There’s a “pair” event that we can listen to from the Bluetooth module that notifies us of when a Myo has paired and provides us a handle to the device. This handle gets used for identifying events for a particular Myo or sending a particular Myo messages.
  3. There’s a connect event that will fire when a Myo connects after it’s been paired with the Bluetooth module. A Myo can be paired but disconnected.
  4. Now that we can uniquely identify a Myo, the only things we need to do are intercept events for a particular Myo and make sense of the data coming from the devices! Orientation change? Acceleration change? There’s a host of information that the device sends back, so we need to interpret it.
  5. When a Myo disconnects, there’s an event that’s sent back for that as well.

Getting Started with MyoSharp

I’m going to start this off with some simple code that should illustrate just how easy it is to get started with MyoSharp. I’ll describe what’s going on in the code immediately after.

[csharp]

using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
/// <summary>
/// This example will show you the basics for setting up and working with
/// a Myo using MyoSharp. Primary communication with the device happens
/// over Bluetooth, but this C# wrapper hooks into the unmanaged Myo SDK to
/// listen on their "hub". The unmanaged hub feeds us information about
/// events, so a channel within MyoSharp is responsible for publishing
/// these events for other C# code to consume. A device listener uses a
/// channel to listen for pairing events. When a Myo pairs up, a device
/// listener publishes events for others to listen to. Once we have access
/// to a channel and a Myo handle (from something like a Pair event), we
/// can create our own Myo object. With a Myo object, we can do things like
/// cause it to vibrate or monitor for poses changes.
/// </summary>
internal class BasicSetupExample
{
Methods
private static void Main(string[] args)
{
// create a hub that will manage Myo devices for us
using (var hub = Hub.Create())
{
// listen for when the Myo connects
hub.MyoConnected += (sender, e) =>
{
Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
e.Myo.Vibrate(VibrationType.Short);
e.Myo.PoseChanged += Myo_PoseChanged;
};

// listen for when the Myo disconnects
hub.MyoDisconnected += (sender, e) =>
{
Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
e.Myo.PoseChanged -= Myo_PoseChanged;
};

// wait on user input
ConsoleHelper.UserInputLoop(hub);
}
}

Event Handlers
private static void Myo_PoseChanged(object sender, PoseEventArgs e)
{
Console.WriteLine("{0} arm Myo detected {1} pose!", e.Myo.Arm, e.Myo.Pose);
}

}
}

[/csharp]

In this example, we create a hub instance. A hub will manage a collection of Myos that come online and go offline and notify listeners that are interested. Behind the scenes, the hub creates a channel instance and passes this into a device listener instance. The channel and device listener combination allows for being notified when devices come online and is the core of the hub implementation. You can manage Myos on your own by completely bypassing the Hub class and creating your own channel and device listener if you’d like. It’s totally up to you.

In the code above, we’ve hooked up several event handlers. There’s an event handler to listen for when Myo devices connect, and a similar one for when the devices disconnect. We’ve also hooked up to an instance of a Myo device for when it changes poses. This will simply give us a console message every time the hardware determines that the user is making a different pose.

When devices go offline, the hub actually keeps the instance of the Myo object around. This means that if you have device A and you hook up to it’s PoseChanged event, if it goes offline and comes back online several times, your event will still be hooked up to the object that represents device A. This makes managing Myos much easier compared to trying to re-hook event handlers every time a device goes on and offline. Of course, you’re free to make your own implementation using our building blocks, so there’s no reason to feel forced into this paradigm.

It’s worth mentioning that the UserInputLoop() method is only used to keep the program alive. The sample code on GitHub actually lets you use some debug commands to read some Myo statuses if you’re interested. Otherwise, you could just imagine this line is replaced by Console.ReadLine() to block waiting for the user to press enter.

Pose Sequences

Without even diving into the accelerometer, orientation, and gyroscope readings, we were looking for some quick wins to building up on the basic API that we created. One little improvement we wanted to make was the concept of pose sequences. The Myo will send events when a pose changes, but if you were interested in grouping some of these together there’s no way to do this out of the box. With a pose sequence, you can declare a series of poses and get an event triggered when the user has finished the sequence.

Here’s an example:

[csharp]

using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it’s useful to know
    /// when a user has performed a series of poses. A 
    /// <see cref="PoseSequence"/> can monitor a Myo for a series of poses and
    /// notify you when that sequence has completed.
    /// </summary>
    internal class PoseSequenceExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // for every Myo that connects, listen for special sequences
                    var sequence = PoseSequence.Create(
                        e.Myo, 
                        Pose.WaveOut, 
                        Pose.WaveIn);
                    sequence.PoseSequenceCompleted += Sequence_PoseSequenceCompleted;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Sequence_PoseSequenceCompleted(object sender, PoseSequenceEventArgs e)
        {
            Console.WriteLine("{0} arm Myo has performed a pose sequence!", e.Myo.Arm);
            e.Myo.Vibrate(VibrationType.Medium);
        }
        #endregion
    }
}
[/csharp]

The same basic setup occurs as the first example. We create a hub that listens for Myos, and when one connects, we hook a new PoseSequence instance to it. If you recall how the hub class works from the first example, this will hook up a new pose sequence each time the Myo connects (which, in this case, isn’t actually ideal). Just for demonstration purposes, we were opting for this shortcut though.

When creating a pose sequence, we only need to provide the Myo and the poses that create the sequence. In this example, a user will need to wave their hand out and then back in for the pose sequence to complete. There’s an event provided that will fire when the sequence has completed. If the user waves out and in several times, the event will fire for each time the sequence is completed. You’ll also notice in our event handler we actually send a vibrate command to the Myo! Most of the Myo interactions are reading values from Myo events, but in this case this is one of the commands we can actually send to it.

Held Poses

The event stream from the Myo device only sends events for poses when the device detects a change. When we were trying to make a test application with our initial API, we were getting frustrated with the fact that there was no way to trigger some action as long as a pose was being held. Some actions like zooming, panning, or adjusting levels for something are best suited to be linked to a pose being held by the user. Otherwise, if you wanted to make an application that would zoom in when the user makes a fist, the user would have to make a fist, relax, make a fist, relax, etc… until they zoomed in or out far enough. This obviously makes for poor usability, so we set out to make this an easy part of our API.

The code below has a similar setup to the previous examples, but introduces the HeldPose class:

[csharp]

using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;
using MyoSharp.Poses;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// Myo devices can notify you every time the device detects that the user 
    /// is performing a different pose. However, sometimes it’s useful to know
    /// when a user is still holding a pose and not just that they’ve 
    /// transitioned from one pose to another. The <see cref="HeldPose"/> class
    /// monitors a Myo and notifies you as long as a particular pose is held.
    /// </summary>
    internal class HeldPoseExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub to manage Myos
            using (var hub = Hub.Create())
            {
                // listen for when a Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);

                    // setup for the pose we want to watch for
                    var pose = HeldPose.Create(e.Myo, Pose.Fist, Pose.FingersSpread);

                    // set the interval for the event to be fired as long as 
                    // the pose is held by the user
                    pose.Interval = TimeSpan.FromSeconds(0.5);

                    pose.Start();
                    pose.Triggered += Pose_Triggered;
                };

                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Pose_Triggered(object sender, PoseEventArgs e)
        {
            Console.WriteLine("{0} arm Myo is holding pose {1}!", e.Myo.Arm, e.Pose);
        }
        #endregion
    }
}
[/csharp]

When we create a HeldPose instance, we can pass in one or more poses that we want to monitor for being held. In the above example, we’re watching for when the user makes a fist or when they have their fingers spread. We can hook up to the Triggered event on the held pose instance, and the event arguments that we get in our event handler will tell us which pose the event is actually being triggered for.

If you take my zoom example that I started describing earlier, we could have a single event handler responsible for both zooming in and zooming out based on a pose being held. If we picked two poses, say fist and fingers spread, to mean zoom in and zoom out respectively, then we could check the pose on the event arguments in the event handler and adjust the zoom accordingly. Of course, you could always make two HeldPose instances (one for each pose) and hook up to the events separately if you’d like. This would end up creating two timer threads behind the scenes–one for each HeldPose instance.

The HeldPose class also has an interval setting. This allows the programmer to adjust the frequency that they want the Triggered event to fire, provided that a pose is being held by the user. For example, if the interval is set to be two seconds, as long as the pose is being held the Triggered event will fire every two seconds.

Roll, Pitch, and Yaw

The data that comes off the Myo can become overwhelming unless you’re well versed in vector math and trigonometry. Something that we’d like to build up and improve upon is the usability of data that comes off the Myo. We don’t want each programmer to have to write similar code to get the values from the Myo into a usable form for their application. Instead, if we can build that into MyoSharp, then everyone will benefit.

Roll, pitch, and yaw are values that we decided to bake into the API directly. So… what exactly are these things? Here’s a diagram to help illustrate:

Roll, Pitch, and Yaw - MyoSharp
Roll, pitch, and yaw describe rotation around one of three axes in 3D space.

The following code example shows hooking up to an event handler to get the roll, pitch, and yaw data:

[csharp]

using System;

using MyoSharp.Device;
using MyoSharp.ConsoleSample.Internal;

namespace MyoSharp.ConsoleSample
{
    /// <summary>
    /// This example will show you how to hook onto the orientation events on
    /// the Myo and pull roll, pitch and yaw values from it.
    /// </summary>
    internal class OrientationExample
    {
        #region Methods
        private static void Main(string[] args)
        {
            // create a hub that will manage Myo devices for us
            using (var hub = Hub.Create())
            {
                // listen for when the Myo connects
                hub.MyoConnected += (sender, e) =>
                {
                    Console.WriteLine("Myo {0} has connected!", e.Myo.Handle);
                    e.Myo.OrientationDataAcquired += Myo_OrientationDataAcquired;
                };

                // listen for when the Myo disconnects
                hub.MyoDisconnected += (sender, e) =>
                {
                    Console.WriteLine("Oh no! It looks like {0} arm Myo has disconnected!", e.Myo.Arm);
                    e.Myo.OrientationDataAcquired -= Myo_OrientationDataAcquired;
                };

                // wait on user input
                ConsoleHelper.UserInputLoop(hub);
            }
        }
        #endregion

        #region Event Handlers
        private static void Myo_OrientationDataAcquired(object sender, OrientationDataEventArgs e)
        {
            Console.Clear();
            Console.WriteLine(@"Roll: {0}", e.Roll);
            Console.WriteLine(@"Pitch: {0}", e.Pitch);
            Console.WriteLine(@"Yaw: {0}", e.Yaw);
        }
        #endregion
    }
}
[/csharp]

Of course, if we know of more common use cases that people will be using the orientation data for, then we’d love to bake this kind of stuff right into MyoSharp to make it easier for everyone.

Closing Comments

That’s just a quick look at how you can leverage MyoSharp to make your own C# application to work with a Myo! As I said, MyoSharp is open source so we’d love to see contributions or ideas for suggestions. We’re aiming to provide as much base functionality as we can into our framework but designing it in a way that developers can extend upon each of the individual building blocks.

author avatar
Nick Cosentino Principal Software Engineering Manager
Principal Software Engineering Manager at Microsoft. Views are my own.

This Post Has 16 Comments

  1. Brian Varley

    Hi Nick,

    I just got my hands on the MYO for a project. My aim is to use the MyoSharp library in conjunction with a WPF application. Although I’m not quite sure how to achieve this yet.

    Say for example I create a WPF project, do I just add an existing item, ie MyoSharp project then add a reference to it?

    Will I then be able to call the methods from the above code in my WPF code?

    Thanks

    1. ncosentino

      Hey Brian,

      That’s exactly how you would make this work. Use git to checkout the MyoSharp repository, and you should end up with a project.

      – Add that project to your solution
      – Add the proper reference to the MyoSharp project to your own project
      – Ensure the x86 and x64 folders get copied into your output directory (they have the C++ Myo code that we wrap)

      Our project is also built to .NET 2.0. If you’re using 4.0 or greater for your application, you can either:

      – Modify MyoSharp to build as .NET 4.0
      – OR create an app config that has support for both .NET 4.0 and 2.0 (which is the “harder” solution, but I’d suggest this)

  2. Roby Ardiansyah

    Hi, nick.

    I want to make windows form aplication using C#, but i’m not fully understand how to start with it.

    1. Roby Ardiansyah

      I dont understand how to move this console sample into windows form aplication.

  3. Fatih

    Hi,

    Firstly, thank you very much to implement C# library. I don’t like to write code C++. I have question for you. How can i use PInvoke.net. Would you give me some details?

    Thank you again.

    1. ncosentino

      My apologies for being so incredibly late to replying to this… Were you able to find out what you need? If not, can you elaborate on what your challenge is?

  4. sameer sapra

    How can i implement myo in my fps game?

    1. ncosentino

      Can you elaborate more on what you’re trying to accomplish? If you’re writing a game in C# and you’d like to leverage the Myo gestures as input, I think you’d be on the right track.

  5. Yusuf

    Hi, thank you and your team very much for implementing this to c#. But I need some help and I really appreciate if you could help me.
    In my project, I need to use 2 Myos at the same time.
    I am having some trouble with determining which data is coming from which myo. Can you give a starting point for me to dig in please? Any clue is appreciated.

    1. ncosentino

      Hey Yusuf,

      I haven’t been active in this library lately. Let me see if I can find something for you.

  6. Todd

    Hi Nick, Nice lib!

    I’m building a very “Tony Stark” level experience for moving data. I have 7 Myo’s in our space. I need to programmatically force Myo to connect and disconnect from bands based on face recognition in the space.

    I don’t see a way in your lib to get a list of bands and force a connect/disconnect.

    Am I missing something?

    1. ncosentino

      Hey Todd,

      I haven’t been very actively developing MyoSharp since I was hoping the community would take a bit more action in making changes to it. Let me see if I can whip something up.

      1. Todd

        I was digging through your code before I pinged you on this. I saw that you’re doing native calls the myos libs. I’d be willing to assist if you can point me to the docs for the libs.

        Perhaps a breif post on that might get the community moving.

        I apologize if you already posted that and I missed it.

        Once again great work.

  7. ๋Kittisak

    Hi, Thank you for library

    my project want to show battery MyO device. Can i get device battery from your library?

  8. Tayyip

    I read the data from the myo armband from the serial port and write it with c # I want to see EMG data coming from a console application but I can not read the port seriously How can I do this? I need your help

Leave a Reply to Todd Cancel reply