Vicon Nexus Realtime Streaming to MAYA Tutorial

June 10, 2013
by admin
    I wrote this tutorial for the dyros blog: http://dyros.snu.ac.kr , but thought some readers here might find it useful as well. Enjoy!
    While Vicon Blade is known for its ease of integration with software such as Motion Builder and Maya, Nexus is most known for its application in Biomechanics as it has the capacity to record numerous sensors such as EMG as well as marker positions.   I wanted to look into art driven by motion capture markers, but I also wanted to use the same software as others in the lab for handling the motion capture, which is nexus.
    My strategy was to use the Vicon realtime SDK which can get markers from the TCP/IP of nexus and then send that to MAYA by the same approach.  As a few people online have demonstrated ways to make MAYA receive information over tcp/ip using python, and since python is much faster for me to demonstrate a proof of concept, I decided to reuse some old code that was made for streaming from arduino to maya.  Although I use nexus, the Vicon SDK is supposed to be for both blade and nexus, so this method should work for either.
    To do this, you must first install the Vicon realtime SDK.  You should be able to download it from their website after creating a user login on the support page.
    Next, load visual studio (in this case i used 2012 express, its free) and make sure to include your .lib and .dll files.    Before editing anything I suggest you compile and test to make sure you have setup everything correctly.
    The example file they give for C++ displays all the outputs from nexus however I just want the markers.  If you go down to line 420 you should see:
    // Count the number of segments
    unsigned int SegmentCount = MyClient.GetSegmentCount( SubjectName ).SegmentCount;
    output_stream << "    Segments (" << SegmentCount << "):" << std::endl;
    Right after this is some stuff on segments which we dont want, so comment it out until around line 576.  The first uncommented line should be:
    unsigned int MarkerCount = MyClient.GetMarkerCount( SubjectName ).MarkerCount;
    output_stream << "    Markers (" << MarkerCount << "):" << std::endl;
    And then the for loop begins.  This is the heart of what we want in Maya.  First Vicon demonstrates how to get the marker name and translation and output it to the console:
    // Get the marker name
    std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;
    
    // Get the marker parent
    std::string MarkerParentName = MyClient.GetMarkerParentName( SubjectName, MarkerName ).SegmentName;
    
    // Get the global marker translation
    Output_GetMarkerGlobalTranslation _Output_GetMarkerGlobalTranslation =
        MyClient.GetMarkerGlobalTranslation( SubjectName, MarkerName );
    
    output_stream << "      Marker #" << MarkerIndex            << ": "
        << MarkerName             << " ("
        << _Output_GetMarkerGlobalTranslation.Translation[ 0 ]  << ", "
        << _Output_GetMarkerGlobalTranslation.Translation[ 1 ]  << ", "
        << _Output_GetMarkerGlobalTranslation.Translation[ 2 ]  << ") "
        << Adapt( _Output_GetMarkerGlobalTranslation.Occluded ) << std::endl;
    As vicon has already shown how to get the values and print to screen, we will just modify this method to store the value instead of printing to the screen.  We still want to get the Marker Name and Marker Number, so keep:
    // Get the marker name
    std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;
    // Get the marker name
    std::string MarkerName = MyClient.GetMarkerName( SubjectName, MarkerIndex ).MarkerName;
    Now to get the translation data we will store the data in the Output_GetMarkerGlobalTranslation type:
    Output_GetMarkerGlobalTranslation valueToSend = MyClient.GetMarkerGlobalTranslation( SubjectName, MarkerName );
    For clarity I created three variables to store the X Y and Z position, then create an array to store the values:
    double markerxPOS = valueToSend.Translation[0];
    double markeryPOS = valueToSend.Translation[1];
    double markerzPOS = valueToSend.Translation[2];
    double markerXYZ[] = {markerxPOS,markeryPOS,markerzPOS};
    To handle some string problems with sending over tcp/ip to maya, we will do some conversions to the marker name:
    char *cstr = new char[MarkerName.length() + 1];
    strcpy(cstr, MarkerName.c_str());
    char *Marker_Name = cstr;
    Just before sending the data to our soon-to-be-written function I want to clarify the MarkerIndex is really the Marker number which I just call:
    int Marker_Num = MarkerIndex;
    Now I will call a function senddata, which we will write next, passing it our marker information:
    senddata(Marker_Num,Marker_Name,markerXYZ);
    Thats it for the marker part. Before going on, there is some extra data vicon is getting that we dont need. A few lines down from our marker function is the unlabeled markers, if you want this for debugging you can keep it but below this is the devices and force plate functions. Right after:
    output_stream << "  Devices (" << DeviceCount << "):" << std::endl;
    Comment out the for loop until you reach the commented // Output eye tracker information.
    Before moving on the the custom function we need to include a header file to let the too functions communicate. At the top, under #include “Client.h” , add #include “myheader.h”
    Create a new header file called myheader.  This will be the link to the custom function. There only needs to be one line:
    int senddata(int MarkerNumber, char *MarkerName,double args[]);
    Save this and create a new CPP file. In my case I call it SendToMaya.CPP. As the goal is to communicate with Maya over tcp/ip, I started with the basic microsoft example of the send function. First include the header file, myheader.h and then some extras for communication:
    #include "myheader.h"
    
    #ifndef UNICODE
    #define UNICODE
    #endif
    
    #define WIN32_LEAN_AND_MEAN
    
    #include 
    #include 
    #include 
    #include 
    //#include 
    
    // Link with ws2_32.lib
    #pragma comment(lib, "Ws2_32.lib")
    
    #define DEFAULT_BUFLEN 512
    #define DEFAULT_PORT 27015
    Next create the function senddata that was used in the vicon sdk file.
    int senddata (int MarkerNumber, char* MarkerName, double args[])
    {
    The first part of the function will define some variables to be used with the socket connections.
        int iResult;
        WSADATA wsaData;
    
        SOCKET ConnectSocket = INVALID_SOCKET;
        struct sockaddr_in clientService; 
    
        int recvbuflen = DEFAULT_BUFLEN;
    Now the marker information needs to be converted to strings in order to send over the socket. Although this is probably not the best way, it seemed to work as a prototype:
     double MarkNum = MarkerNumber;    // number to be converted to a string
     std::string ResultMN;          // string which will contain the result
     std::ostringstream convertMN;   // stream used for the conversion
     convertMN << MarkNum;      // insert the textual representation of 'Number' in the characters in the stream
     ResultMN = convertMN.str(); // set 'Result' to the contents of the stream
     char *cstrMN = new char[ResultMN.length() + 1];
     strcpy(cstrMN, ResultMN.c_str());
     char* s_MarkNum = cstrMN;
    
     double x_pos = args[0];    // number to be converted to a string
     std::string Resultx;          // string which will contain the result
     std::ostringstream convertx;   // stream used for the conversion
     convertx << x_pos;      // insert the textual representation of 'Number' in the characters in the stream
     Resultx = convertx.str(); // set 'Result' to the contents of the stream
     char *cstrx = new char[Resultx.length() + 1];
     strcpy(cstrx, Resultx.c_str());
     char* s_xPOS = cstrx;
    
     double y_pos = args[1];    // number to be converted to a string
     std::string Resulty;          // string which will contain the result
     std::ostringstream converty;   // stream used for the conversion
     converty << y_pos;      // insert the textual representation of 'Number' in the characters in the stream
     Resulty = converty.str(); // set 'Result' to the contents of the stream
     char *cstry = new char[Resulty.length() + 1];
     strcpy(cstry, Resulty.c_str());
     char* s_yPOS = cstry;
    
     double z_pos = args[2];    // number to be converted to a string
     std::string Resultz;          // string which will contain the result
     std::ostringstream convertz;   // stream used for the conversion
     convertz << z_pos;      // insert the textual representation of 'Number' in the characters in the stream
     Resultz = convertz.str(); // set 'Result' to the contents of the stream
     char *cstrz = new char[Resultz.length() + 1];
     strcpy(cstrz, Resultz.c_str());
     char* s_zPOS = cstrz;
    
     //std::cout<<"XYZ: "<
    Set the *sendbuf variable to the newly created xyzString:
     char *sendbuf = xyzString;
    Do some initialization and connections:
     char recvbuf[DEFAULT_BUFLEN] = "";
        //----------------------
        // Initialize Winsock
        iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
        if (iResult != NO_ERROR) {
            wprintf(L"WSAStartup failed with error: %dn", iResult);
            return 1;
        }
    
        //----------------------
        // Create a SOCKET for connecting to server
        ConnectSocket = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP);
        if (ConnectSocket == INVALID_SOCKET) {
            wprintf(L"socket failed with error: %ldn", WSAGetLastError());
            WSACleanup();
            return 1;
        }
    This is the part where you define the IP and port. For local computer it is probably 127.0.0.1 and the port can be anything, pick a high number which will usually not conflict with anything:
        //----------------------
        // The sockaddr_in structure specifies the address family,
        // IP address, and port of the server to be connected to.
        clientService.sin_family = AF_INET;
        clientService.sin_addr.s_addr = inet_addr( "127.0.0.1" );
        clientService.sin_port = htons( 1188 );
    And that is mostly it for the C++ side. The rest is more of the standard communication protocols found in the microsoft examples:
        //----------------------
        // Connect to server.
        iResult = connect( ConnectSocket, (SOCKADDR*) &clientService, sizeof(clientService) );
    
        if (iResult == SOCKET_ERROR) {
            wprintf(L"connect failed with error: %dn", WSAGetLastError() );
            closesocket(ConnectSocket);
            WSACleanup();
            return 1;
      }
    
        //----------------------
        // Send an initial buffer
        iResult = send( ConnectSocket, sendbuf, (int)strlen(sendbuf), 0 );
     //wprintf(L"sending the data that i got: %dn",args);
    
        if (iResult == SOCKET_ERROR) {
            wprintf(L"send failed with error: %dn", WSAGetLastError());
            closesocket(ConnectSocket);
            WSACleanup();
            return 1;
        }
    
        printf("Bytes Sent: %dn", iResult);
    
        // shutdown the connection since no more data will be sent
        iResult = shutdown(ConnectSocket, SD_SEND);
        if (iResult == SOCKET_ERROR) {
            wprintf(L"shutdown failed with error: %dn", WSAGetLastError());
            closesocket(ConnectSocket);
            WSACleanup();
            return 1;
        }
    
        // Receive until the peer closes the connection
        do {
    
            iResult = recv(ConnectSocket, recvbuf, recvbuflen, 0);
            if ( iResult > 0 )
                wprintf(L"Bytes received: %dn", iResult);
            else if ( iResult == 0 )
                wprintf(L"Connection closedn");
            else
                wprintf(L"recv failed with error: %dn", WSAGetLastError());
    
        } while( iResult > 0 );
    
        // close the socket
        iResult = closesocket(ConnectSocket);
        if (iResult == SOCKET_ERROR) {
            wprintf(L"close failed with error: %dn", WSAGetLastError());
            WSACleanup();
            return 1;
        }
    
        WSACleanup();
        return 0;
    }
    Now that the C++ is done the Maya file must be created.   The current method has some issues because of the speed of python vs c++ but it is a good way to start getting the markers in maya.  To solve many of these issues the C++ api can be used instead.
    To get started we need to import the maya commands and mel, then setup a melproc:
    # Take in Data (This case Markers) and Do a Transform
    import maya.cmds as mc
    import maya.mel as mm
    #num = 0
    # Our mel global proc.
    melproc = """
    global proc portData(string $arg){
        python(("portData("" + $arg + "")"));
    }
    """
    mm.eval(melproc)
    Next create a function portData that will take in the port information and do something with it. This will be called later in the script. This example will call another function that will parse the data out, then it will try to call cmds.xform, using the second value of the array as the name of the object to move.c
    def portData(arg):
        """
        Reads streaming data passed from TCP/IP
        """
        #print "Recieved!: ", arg
    
        values = get_vals(arg)    
        try:
            cmds.xform(values[1],t=(values[2],values[3],values[4]))
            cmds.refresh()
        except:
            pass
    The get_vals() function takes in the data string and parses it. Since the length of the string is unknown as marker names and position values could be different lengths, we create some variables to count when a space exists. When each space is found it will separate the variables:
    def get_vals(arg):
        i=0
        marker_num=[]
        marker_name=[]
        marker_XP=[]
        marker_YP=[]
        marker_ZP=[]
    
        while (arg[i]!=":"):
            i=i+1   
        marker_num=arg[0:i]
        marker_num=float(marker_num)
        i=i+1
        j=i
        while (arg[i]!=":"):
            i=i+1
        marker_name=arg[j:i]
        i=i+1
        k=i
        while (arg[i]!=","):
            i=i+1    
        marker_XP=arg[k:i]    
        marker_XP=float(marker_XP)
        i=i+1
        l=i
        while (arg[i]!=","):
            i=i+1   
        marker_YP=arg[l:i]    
        marker_YP=float(marker_YP)          
        i=i+1
        m=i     
        try:
            while (arg[i]):
                i=i+1    
        except:
            pass
        marker_ZP=arg[m:i]    
        marker_ZP=float(marker_ZP) 
    
        values = [marker_num,marker_name,marker_XP,marker_YP,marker_ZP]
    
        return values
    The very last step in the script is opening the command port for maya to listen for data.  The name should be equal to the ip and port used in the cpp file:
    # Open the commandPort.  The 'prefix' argument string is calling to the defined
    # mel script above (which then calls to our Python function of the same name):
    mc.commandPort(name="localhost:1188", echoOutput=False, noreturn=False,
                   prefix="portData", returnNumCommands=True)
    Thats it! Remember that if you are trying to actually move some markers in Maya you need to create some spheres and name them the same as the marker names that are streaming.

    Leave a Comment