Integration of Subsystems on the Software level

This post we will be talking about integrating the all subsystems on software level. As a whole, we are using 3 arduino boards.

                                Board 1 - Robot Arm and Hand

                                Board 2 - IR Sensors on the Robot Hand

                                Board 3 - Glove (Vibrational Motors)

Shown is the overview of how the user control the robot remotely. Setting up the server is discussed in detail here.

Figure 1 - Overview

Inverse Kinematics on MENTOR Arm - Part 2

As mentioned in the previous post we are using IK only for 2dof (on x and y plane). By that we calculated the joint angles on the Top and Bottom arm for the given x,y,z coordinates. 

Using the conversion from cartesian coordinate system (x,y,z) to cylindrical coordinates (ρ,theta2,r) the joint angle for base arm (theta2) was calculated.

Angle - Base arm

To calculate the joint angles the function "jointAngles" is used. The x,y,z coordinates of the palm that we get from the leap motion data are passed as an array "v" into the function and theta2 - base angle, angle2 - Bottom angle and angle3 - Top angle are returned as the output. 

So you think you can install the client?

Following the integration of all the subsystems, one very stark problem came to light.

Firstly, let me recap the server-client relationship:

Server: connected to the Robotic Arm, LBS hand and IR sensors

Client: connected to the Leap Motion and the vibrational motors

Previously, for usability purposes, the client was accessed through a web browser, and only required a web browser and the Leap Motion to be connected. However, it quickly became clear that there was no way that you could control the Arduino through the browser. All browsers are sandboxed by default, and for good reason! If you were able to affect the local computer through a browser, there would be a massive vulnerability with Javascript.

(Pedantic note: there are ways to unlock the sandboxing in Chrome, but that is unsafe and if you forget to lock it up after yourself, it leaves you, your machine and your network vulnerable to attack)

Thus, we decided that a server-client relationship that was instead two servers talking to each other would be much easier, but less user-friendly. It will probably not feature in our demo, but it is an option for others to improve on in the future if they wish to.

In this post, I'll be outlining the general way to get the server and client working. There are multitudes of tutorials on this set-up, but for documentation's sake it's good to have this recorded.

Step 1: Install NodeJS - the easiest step. The setup is quite simple, and will help install the Node Package Manager (npm) for you.

Step 2: Create your  new project directory - for our sakes let's call C:\SecretPlans\WorldDomination.

Step 3: Open Command prompt and navigate to that directory. (use "cd [insert path here]") In our example, it will be cd C:\SecretPlans\WorldDomination.

Step 4: Install package dependencies through npm. In our case here, we haven't done it the best way - you will have to use "npm install johnny-five socket.io socket.io-client leapjs" which installs all the latest versions of those packages.

The best way is to instead wrap it up in a JSON file that includes all the dependencies - usually called package.json. In that case, you can merely write "npm install" and it will read the package.json and download all it's dependencies. The package.json file is used mainly when one is writing a script or package that may be used in the npm directories (ie downloadable by others). There's a lot of information you can put in the package.json so that it can be read by others. Read more about it here!

Step 5: Write the script in the editor of your choice (our choices have been Notepad++ (for Windows) and Sublime Text 2 (for Mac)). In our case for Airgrip, we have files called FINALclient.js or FINALserver.js. Each of these need to certain lines modified in the code as to what port the Arduino boards are attached to. In our fictional case, lets call the files NukeWorld.js and CommandCenter.js.

Step 6: Back to Command prompt - set the program running by using 'node filename.js'. So in our fictional case, we would run 'node CommandCenter.js' (assuming that's the server). On the other computer, we would run 'node NukeWorld.js' (assuming that's the client).

Step 7: ??? World Domination...?

Next up - more magical blogposts :)

Multi-boards and the curse of the vibration motors

Johnny-Five is the key Javascript Node Module that we are using in this project. Essentially, it allows Javascript to talk to the Arduino using a standard protocol called the StandardFirmata. You can check the documentation in the link above - it bundles up all the types of inputs and outputs you would need in a very easy-to-use manner.

However, the one problem you'll find with working with Open Source packages is that they'll change on you without notice, quickly and quietly. Furthermore, these packages do not always have your application in mind, and thus will not necessarily have documentation to support what you want to do! This blog post will detail some problems we had while using this package - but of course I should state that it's still an absolutely AMAZING package and we wouldn't be here without it.

 

Multiple Boards

One of the things Johnny-Five does not have very good documentation about is supporting multiple boards in one Javascript script. Here is the documentation they have; it is good for the application they have in mind, but not for our purposes. It seems that using the Boards object allows you to run the same code on BOTH the Arduino's, but not different code on each - which is what we'd like to do. The circuitry for control on the arm, and the IR sensors were on different boards. One uses an Arduino MEGA, one uses an Arduino UNO. (The vibration motors also use an Arduino UNO, but at this time we were just focussing on the 'server-side' system).

The problem was exacerbated by the fact that the laptops Ashan and I were using were different - a MAC and a Windows laptop. Thus, every time we tried to experiment with the same code, we'd have different results.

See below for the different ways you can go about making multiple boards according to the documentation

The first method is an automatic method of acquiring the devices. However, this doesn't guarantee you'll get the same one each time. We tried this, and it did not always assign the correct board.

The second method assigns the ports correctly, but because of the way the class is created in Johnny-Five, it does not allow you to control the boards separately. The third way outlined above is highly similar to the second way.

The actual solution that is MUCH easier is as follows:
1. Create different board objects
2. Make sure that the objects you create are assigned to the board

Each object that is created is given a 'board' property that relates to which board it is attached to. This feature is not in the actual documentation for each of these things - I just hoped that it would be an exposed property of the object, and it was!

Thus, our problem with the documentation was solved. We could use multiple boards in our Javascript programs!

 

IR Communication Issues

When we decided to try and use the server-client relationship to communicate between Arduino boards (more specifically, the IR sensors and the vibration motors), a problem arose.

Namely, the IR sensors were going WILD! Sometimes they would read a consistent value, but for a majority of the time, they would wildly fluctuate between the actual value, and half the actual value. This is the weirdest behaviour I've ever seen, and I started troubleshooting the damn IR sensors.

- the connections were stable - I tested them individually with a separate Arduino and they were giving the right value
- the program itself was fine - I tested the IR sensors isolated on one computer, and the readings I was getting were fine.
- I tried it on both mine and Ashan's laptops, and it worked! It was only with a CLEAN INSTALL on the lab computer that it would fail! WHY!

This testing stage was the most infuriating part, as I could not isolate the problem. But I finally found the solution...

Checking the documentation of the johnny-five website, I found that only recently had documentation changed in how you would access data from a sensor. The reason it was working on both our laptops but not a CLEAN INSTALL is that we had an older version of the package on our computers.


The top part of this code shows how you read from the sensor in an earlier version. The bottom part of this code shows you the NEW way to read from the sensor. This was not reflected as a change anywhere that I could find, and the documentation did not reflect that there had been any change at all! The code was just different - I thought I was going crazy!

This change was able to fix the bug. The lesson we learnt here is that using Open Source packages has it's detriments - namely, that whoever is maintaining the code may not be the clearest about their changes.

Damn it Johnny-Five...

U mad bro?

Programming Arduino with JS

As we mentioned earlier, we are using Javascript as the main programming language through out the project. The Arduino microcontroller can be controlled through the use of frameworks like Rick Waldron's Johnny-five for Node JS, and the "StandardFirmata"software package for Arduino. 

Yeah we will be most likely utilising the Node.JS framework, which includes packages that allow for ease of communication between hardware and software. This enables us to handshake between Arduino Firmware and the Firmata protocol. Structure of the whole system is shown below.