LBSv2 Hand (movement analysis)

We described the linkage mechanism in an older post. This post will described how the linkages work in LBSv2 hand. Here a video demonstration:

Throwing down the gauntlet to objects

This post will detail more about the IR sensor mounting and the final testing we did with it. As we saw in the last post:

 Hardware

The IR sensors have been mounted and look absolutely wonderful. If you have been following the blog, you will have also seen the fingers that Shen designed that have been mounted so nicely on to the arm. Let's have some applause for Shen!

Knowing in advance how big the finger lengths were going to be was quite useful as I was able to limit down the sizes of the VeroBoard. We discussed different ways to mount the boards, but in the end the easiest solution that we still have right now is hot-glue; the interfaces between the curved surfaces were hard to adhere to as the angle didn't work very well. In the end, it's still quite sturdy and has survived the torture of our tests.

The picture above shows the (somewhat ugly) wiring of the sensors on the hand. As you can see, the ribbon cables are slightly raised up to then split off into Vcc, Ground and Analog wires.

PROBLEM: The raised ribbon wires were causing problems as they curled up and twisted around the hand, interfering with the movement of hte fingers and the wires of the other sensors.

SOLUTION: The Gauntlet - Aluminium plates that would guide the ribbon wires along the fingers. We did this for both the 'bones' of the finger. Funnily enough, the workshop had a stack of aluminium plates that were nearly the size we required that only necessitated small cuts using a guillotine. Where were they from? The metal backing from breadboards that the First-Years were given. What goes around comes around, hey?

Software

Just a final note on the IR sensors that we are using: how it responds to certain different types of material.

Here is a short table on the type of tests we did with the hand and the IR sensors:

Object	Thumb	Index	Middle	Ring	Pinky
All Fingers open	9	5	8	10	5
All Fingers closed	13	8	10	16	7
Red Ball	12	73	71	81	41
Roll of Masking Tape	35	16	21	30	7
White Foam	13	40	44	69	36
Pink Foam	13	63	73	92	52
Yellow Foam Mat	60	49	54	76	44

In most of these tests, it was only the index to pinky fingers that were holding the object - as you can see, the Thumb was not always involved in holding the object. Thus it has lower readings.

We also tried a black piece of foam, but, as with all our previous tests, it did not have a good response at all - it was pretty much the same as All Fingers Closed. Thus we did not include it in our tests - and it will not be part of demos. It is a limitation of the hand and the sensors.

Next post - a quick stroll alonger the shores of software for Vibration Motors

LBSv2 Hand (Integration)

The robot hand is finally made!

Now we have to integrate it. 

There was a list of tasks tasks that we had to perform:

1. Mount the hand on Mentor arm. 
2. Find a way to control the servos via the same Arduino Mega used for the Mentor arm.
3. Finally, the software integration with the other sub systems.

Other tasks:

1. Mitigate overshoot

The extra stuff on the hand will be discussed in a later post

LBSv2 Hand (Solidworks drawings)

Illustrations of LBSv2 robot hand on Solid Works!

A Wrist-y exercise

In this post I'll be detailing the problems that we had with the wrist and integrating it with both the hardware and software.

Without a doubt, this has been one of the most time-consuming problems we've had. Let's get into some of the gory details.

In essential terms, the way that we get the arm moving is to read the potentiometer values from each joint, calculate where the arm should be based on the Leap Motion data, and continue towards that value until we get there. The potentiometers, as we found out, were continuous rotation potentiometers - this is opposed to more typical potentiometers which have maximum and minimum values that you can go. Continuous rotation potentiometers are ones where, as the name states, you can continually rotate them. These potentiometers also have a maximum and minimum point, but instead of stopping you from moving past it, it lets you!

So instead of reading from, say, 0 to 1023 and have the potentiometer physically stop it from continuing on, it just continues on and wraps around back to 0. This is problematic, because if we're at a value of 900, and need to go to 1000, there is a chance that the pot will overshoot (covered in a later post), and will go back to 0. The program will try to  correct for this by continuing the motion until it gets back to a value of 1000. However, this may mean that as it continues to move, it tries to move through itself!

This problem only occurred with the wrists. Other potentiometer values were not affected as we limited the motion so that it was in between 0 and 1023.

However, when the wrist moved up or down (roll), or twisted left or right (pitch), it was possible that it would go through 0 to 1023 or vice versa. So firstly, we had a problem of it moving through the maximum/minimum value of the pot.

Furthermore, we initially used these pots to determine our position.We could map the value of the hand position from the Leap Motion to a potentiometer value. We wanted to use this same idea with the wrists, but we hit upon another problem! When you roll or pitch your hand, the potentiometer values will change differently. What this means is that it is possible for two or more different positions to have the SAME potentiometer values!

Through a graveyard of many possibilities, we hit upon a solution. Our solution for this problem was to instead define certain regions and wrist movements that our robot arm could go to, and use the pitch and roll functions that came with the SDK/API to determine how the hand had moved. A gesture of the hand will determine whether the wrist should move as the pitch and roll characteristics (which I detailed in an earlier post) will change.

It is unfortunate that we could not get the robotic Arm to move in real-time and mirror our wrist movements exacly. It is a limitation of the Arm that we have and the set-up that we were given. However, further work may be able to work around the problem that the wrist creates for us, and an even more functional arm may result!

So you think you can install the client?

Following the integration of all the subsystems, one very stark problem came to light.

Firstly, let me recap the server-client relationship:

Server: connected to the Robotic Arm, LBS hand and IR sensors

Client: connected to the Leap Motion and the vibrational motors

Previously, for usability purposes, the client was accessed through a web browser, and only required a web browser and the Leap Motion to be connected. However, it quickly became clear that there was no way that you could control the Arduino through the browser. All browsers are sandboxed by default, and for good reason! If you were able to affect the local computer through a browser, there would be a massive vulnerability with Javascript.

(Pedantic note: there are ways to unlock the sandboxing in Chrome, but that is unsafe and if you forget to lock it up after yourself, it leaves you, your machine and your network vulnerable to attack)

Thus, we decided that a server-client relationship that was instead two servers talking to each other would be much easier, but less user-friendly. It will probably not feature in our demo, but it is an option for others to improve on in the future if they wish to.

In this post, I'll be outlining the general way to get the server and client working. There are multitudes of tutorials on this set-up, but for documentation's sake it's good to have this recorded.

Step 1: Install NodeJS - the easiest step. The setup is quite simple, and will help install the Node Package Manager (npm) for you.

Step 2: Create your  new project directory - for our sakes let's call C:\SecretPlans\WorldDomination.

Step 3: Open Command prompt and navigate to that directory. (use "cd [insert path here]") In our example, it will be cd C:\SecretPlans\WorldDomination.

Step 4: Install package dependencies through npm. In our case here, we haven't done it the best way - you will have to use "npm install johnny-five socket.io socket.io-client leapjs" which installs all the latest versions of those packages.

The best way is to instead wrap it up in a JSON file that includes all the dependencies - usually called package.json. In that case, you can merely write "npm install" and it will read the package.json and download all it's dependencies. The package.json file is used mainly when one is writing a script or package that may be used in the npm directories (ie downloadable by others). There's a lot of information you can put in the package.json so that it can be read by others. Read more about it here!

Step 5: Write the script in the editor of your choice (our choices have been Notepad++ (for Windows) and Sublime Text 2 (for Mac)). In our case for Airgrip, we have files called FINALclient.js or FINALserver.js. Each of these need to certain lines modified in the code as to what port the Arduino boards are attached to. In our fictional case, lets call the files NukeWorld.js and CommandCenter.js.

Step 6: Back to Command prompt - set the program running by using 'node filename.js'. So in our fictional case, we would run 'node CommandCenter.js' (assuming that's the server). On the other computer, we would run 'node NukeWorld.js' (assuming that's the client).

Step 7: ??? World Domination...?

Next up - more magical blogposts :)

Multi-boards and the curse of the vibration motors

Johnny-Five is the key Javascript Node Module that we are using in this project. Essentially, it allows Javascript to talk to the Arduino using a standard protocol called the StandardFirmata. You can check the documentation in the link above - it bundles up all the types of inputs and outputs you would need in a very easy-to-use manner.

However, the one problem you'll find with working with Open Source packages is that they'll change on you without notice, quickly and quietly. Furthermore, these packages do not always have your application in mind, and thus will not necessarily have documentation to support what you want to do! This blog post will detail some problems we had while using this package - but of course I should state that it's still an absolutely AMAZING package and we wouldn't be here without it.

 

Multiple Boards

One of the things Johnny-Five does not have very good documentation about is supporting multiple boards in one Javascript script. Here is the documentation they have; it is good for the application they have in mind, but not for our purposes. It seems that using the Boards object allows you to run the same code on BOTH the Arduino's, but not different code on each - which is what we'd like to do. The circuitry for control on the arm, and the IR sensors were on different boards. One uses an Arduino MEGA, one uses an Arduino UNO. (The vibration motors also use an Arduino UNO, but at this time we were just focussing on the 'server-side' system).

The problem was exacerbated by the fact that the laptops Ashan and I were using were different - a MAC and a Windows laptop. Thus, every time we tried to experiment with the same code, we'd have different results.

See below for the different ways you can go about making multiple boards according to the documentation

The first method is an automatic method of acquiring the devices. However, this doesn't guarantee you'll get the same one each time. We tried this, and it did not always assign the correct board.

The second method assigns the ports correctly, but because of the way the class is created in Johnny-Five, it does not allow you to control the boards separately. The third way outlined above is highly similar to the second way.

The actual solution that is MUCH easier is as follows:
1. Create different board objects
2. Make sure that the objects you create are assigned to the board

Each object that is created is given a 'board' property that relates to which board it is attached to. This feature is not in the actual documentation for each of these things - I just hoped that it would be an exposed property of the object, and it was!

Thus, our problem with the documentation was solved. We could use multiple boards in our Javascript programs!

 

IR Communication Issues

When we decided to try and use the server-client relationship to communicate between Arduino boards (more specifically, the IR sensors and the vibration motors), a problem arose.

Namely, the IR sensors were going WILD! Sometimes they would read a consistent value, but for a majority of the time, they would wildly fluctuate between the actual value, and half the actual value. This is the weirdest behaviour I've ever seen, and I started troubleshooting the damn IR sensors.

- the connections were stable - I tested them individually with a separate Arduino and they were giving the right value
- the program itself was fine - I tested the IR sensors isolated on one computer, and the readings I was getting were fine.
- I tried it on both mine and Ashan's laptops, and it worked! It was only with a CLEAN INSTALL on the lab computer that it would fail! WHY!

This testing stage was the most infuriating part, as I could not isolate the problem. But I finally found the solution...

Checking the documentation of the johnny-five website, I found that only recently had documentation changed in how you would access data from a sensor. The reason it was working on both our laptops but not a CLEAN INSTALL is that we had an older version of the package on our computers.


The top part of this code shows how you read from the sensor in an earlier version. The bottom part of this code shows you the NEW way to read from the sensor. This was not reflected as a change anywhere that I could find, and the documentation did not reflect that there had been any change at all! The code was just different - I thought I was going crazy!

This change was able to fix the bug. The lesson we learnt here is that using Open Source packages has it's detriments - namely, that whoever is maintaining the code may not be the clearest about their changes.

Damn it Johnny-Five...

U mad bro?