USBNet Feature Demystified! Developmental Driver Available!

51 posts / 0 new
Last post
TikaC
TikaC's picture
USBNet Feature Demystified! Developmental Driver Available!

Thanks bunches to Helibot for his help, we have pretty much demystified the USBNet feature! It apparently is there to allow some Java games to access the internet. I've run across some that tried to do that.

However, we are still working on a way to use it to access the linux console. To that end, I've got a driver (works in Vista but not sure about anything else as I haven't tested it anywhere else), which will load for the robot's USBNet mode. It's only for development purposes though and includes the libusb-win32 sources and utilities, so you could probably adapt it to other Windows versions.

More information on this and the download is here:

http://bytebin.net/robotics/rsmedia/dev/rsmusbnet.html

Hope this helps someone! So far all I can do is get it to detect when the robot is connected to the computer in that mode. But read/write is not working because we need to do that via ppp, which is the protocol the robot is using in that mode (via USB).

Other news, I'm waiting on some electronics parts and once I got that (includes header pin material) I will have the stuff to do the serial port hack to my bot. I have the PropPlug now and some wire. So that is another thing that will soon be looked into.

Helibot also gave me some images and script commands for updating Nocturnal's firmware dump script so that images will display when it starts/stops (so you know when to unplug the bot).

All this means that I will hopefully be soon updating the RSMedia Development Kit. I'll announce here when the new version is ready.

 

 

Helibot
Helibot's picture

Hi all,
Just had a break though with this. My RSMedia just sent the text "RSMEDIA Says Hello via USB!!!!!" to my PC over the USB cable!
I have been working with Tika on this, and I made some changes to the PC test progam to match it to the USBserial driver in the RSMedia and also compiled a new version of the mx1ads_usbserial.o and it seems to have worked.
I will pass the source over to Tika and she can put it on her site.

Soon I hope we can redirect the RSMedia console to the USB port.
Does anyone know how this could be done in an embedded linux system (that is mostly readonly filesystem??)

Nocturnal
Nocturnal's picture

It probably won't work, but you could try

sh > /dev/devname

replace devname with appropriate device.

TikaC
TikaC's picture

I've got Helibot's files and will be trying them out, and then hopefully posting them as an update to the article on the site. It's going to be a busy weekend for me so forgive if I'm a bit slow. LOL!

Nocturnal - If it can send/receive like that via USB Port, I'm thinking it *might* actually work. There's a pppd deamon in there, and I did see a telnet command in there as well! Plus there's an ssh_config (but it's got nothing but comments in it).

Here's some stuff I found on the file system. Note that we have established that I have a *newer* version of RSMedia. The older versions and/or those without the USBNet feature may not have some of these (and thus maybe NOT be possible in the way I'm thinking - more on that in a moment)...

/bin/stty
/bin/hostname
/bin/zmrx
/bin/zmtx

(The last two I equate to Zmodem Receive (rx) and Transmit (tx))

/dev/ppp
/dev/ttyS1
/dev/ttyUSB

/etc/hosts
/etc/nsswitch.conf
/etc/ssh/ssh_conf
/etc/terminfo/a/ansi
/etc/terminfo/a/ansi-m
/etc/terminfo/l/linux
/etc/terminfo/l/linux-m

/lib/libresolv-2.3.2.so
/lib/libnsl.so.1
/lib/libnsl-2.3.2.so
/lib/libnss_compat.so.2
/lib/libnss_compat-2.3.2.so
/lib/libnss_dns.so.2
/lib/libnss_dns-2.3.2.so
/lib/libnss_files.so.2
/lib/libnss_files-2.3.2.so
/lib/libnss_hesiod.so.2
/lib/libnss_hesiod-2.3.2.so
/lib/libnss_nis.so.2
/lib/libnss_nis-2.3.2.so
/lib/libnss_nisplus.so.2
/lib/libnss_nisplus-2.3.2.so
/lib/libnss_winbind.so
/lib/libnss_winbind.so.2
/lib/libnss_wins.so
/lib/libnss_wins.so.2
/lib/libresolv.so.2

/lib/modules/mx1ads_usbd.o

/usr/bin/robot/ppp/route
/usr/bin/robot/ppp/ifconfig
/usr/bin/robot/ppp/mx1ads_usbserial.o
/usr/bin/robot/ppp/ppp.sh
/usr/bin/robot/ppp/pppd

/usr/bin/tty
/usr/bin/telnet
/usr/bin/tftp

/usr/lib/libssl.so.0.9.6
/usr/lib/libssl.so
/usr/lib/libssl.so.0

/usr/local/lib/libopenobex-1.0.so.0.0.0
/usr/local/lib/libopenobex.so
/usr/local/lib/libopenobex-1.0.so.0
(see http://dev.zuckschwerdt.org/openobex/ for more info)

Now for those that do not have USBNet (like Helibot's bot), there may still be a way in via the USB port! I think his breakthrough may be the key.

More info coming up as we research this! :)

TikaC
TikaC's picture

The USBNet Driver Kit is available. Here's my post about it:

http://www.robocommunity.com/forum/thread/15805/USBNet-Driver-Kit-and-So...

You can find it here (link is in the last paragraph of the article. Link is titled "RSMedia USBNet USB driver for Windows Vista":

http://bytebin.net/robotics/rsmedia/dev/rsmusbnet.html

Helibot
Helibot's picture

Hi all,
Good news - I have got a basic set of drivers working!!. In short it works like this - I use Hypertermal (or Teraterm) to connect to the RSMedia via USB and send him characters , I have a program (and a loadable module) that runs on RSMedia that receives the characters and loops them back to the PC.
From RSMedia serial console (or body con script) I can do any linux command (like ls -l) and get the results sent back to the PC via USB!!
The only thing I cant do is have the characters going to RSMedia to be sent to the console (so they are considered as commands). I think I will need some help with this. Will put another post up soon with some more details.

Cheers
Helibot

Helibot
Helibot's picture

Nocturnal said: It probably won't work, but you could try sh > /dev/devname < /dev/devname & replace devname with appropriate device.

Hi Nocturnal and all,

   I tried your idea but it only half works :-(. This is what I get:-

# sh < /tmp/ttyUSB > /tmp/ttyUSB  &
# cannot create /tmp/ttyUSB: Input/output error

I can use 'sh > /tmp/ttyUSB script'  then the script is executed and results sent to the PC via USB.
Similarily from the serial console I can use command 'ls -l > /tmp/ttyUSB' and see the results on the PC.
But If I use 'sh < /dev/devname' the I get error 'cannot create /tmp/ttyUSB: Input/output error'.
Any other ideas? 

Does anyone know if you can add a login/getty daemon to a running linux system? Does it need any Kernal support?  

Is there a way to get the linux root filesystem from RSMedia, unpack it, make some changes, pack it pack up and put it back in the robot?.  If so then I can think of other things to try.

Anyone else got any ideas??

Cheers
  Helibot

TikaC
TikaC's picture

Why not try instead of /tmp using a temp dir on the SD card? Maybe the reason it's not working is because /tmp isn't writable (at that time)? Or that /tmp/ttyUSB doesn't exist? Or else, you need to format the stuff in a format that ttyUSB expects, and it doesn't handle raw commands, but some kind of packet or something?

Helibot
Helibot's picture

Hi Tika,
/tmp is always writable on RSMedia. Also note that /tmp/ttyUSB isnt quite a normal file (its a node or device file). So we are actually opening the device (as provided by mx1ads_usbserial.o) I suspect that the shell maybe trying todo some ioctrl function calls to the device , but mx1ads_usbserial.o doesnt support any!......maybe that needs to be added. I will look into the source for the other USB device drivers with the linux source and see what they support.

Nocturnal
Nocturnal's picture

Nocturnal said: It probably won't work, but you could try sh > /dev/devname < /dev/devname & replace devname with appropriate device.

Hi Nocturnal and all,
   I tried your idea but it only half works :-(

:) I did say it probably wouldn't work.

Does anyone know if you can add a login/getty daemon to a running linux system? Does it need any Kernal support?  

Yes you can add a login/getty daemon to a running linux system (your just spawning a process), I don't think it requires specific kernel support, but I could be wrong.

Is there a way to get the linux root filesystem from RSMedia, unpack it, make some changes, pack it pack up and put it back in the robot? If so then I can think of other things to try.
Anyone else got any ideas??

Yes, but you have to end up with a file the is equal to or smaller than the original, and you'd better make sure you keep a copy of the original file system. I have some scripts that will decompress the filesystem, and then compress it again when you are done, but I don't really have time to dig them out right now. If you google cramfs you should be able to find all the information you need.

Normally I'd be try all this myself (you may recall I have no patience :), unfortunately all my free time is accounted for at the moment.

TikaC
TikaC's picture

Helibot - I was thinking along those lines as well, but don't know anything about tty devices, let alone anything USB. You're way ahead of me on that. :) But keep working at it as you can. You've already done a lot!

daemons don't usually need kernal support. I doubt a login/getty would. I'd be highly surprised if it did. As Nocturnal said, daemons are just processes that run in the background. I think it should be ok. Just keep in mind how much memory RS Media has (working memory) but I don't know his RAM size.

Nocturnal - You're saying we could make a backup of the entire RS Media and if the system should go, we could reload it? Or we could create a new system altogether? I was wondering if the firmware dump script you gave me was all there was to the system but looked like some directories weren't really copied over (they were just links). So I wondered if there was a way to back up everything in case of problems.

Also would it be possible to like, increase the size of the memory/storage with new chips physically replacing the old and put in a bigger, more involved system, an improved system? I know I'm dreaming but just curious! LOL!

Nocturnal
Nocturnal's picture

You're saying we could make a backup of the entire RS Media and if the system should go, we could reload it? Or we could create a new system altogether? I was wondering if the firmware dump script you gave me was all there was to the system but looked like some directories weren't really copied over (they were just links). So I wondered if there was a way to back up everything in case of problems.

No, actually I wasn't saying that (although yes, you could), and yes the script I gave you dumps and mount all there is to the system. I think there are some large gaps in your understanding of whats going on. All those symlinks are nothing to do with me or the scripts, that is an exact copy of what the filesystem on the RS Media looks like. 

Yes there is a way to back everything up in case of problems, you already have it, my backup script.

Also would it be possible to like, increase the size of the memory/storage with new chips physically replacing the old and put in a bigger, more involved system, an improved system? I know I'm dreaming but just curious! LOL!

Yes, its also possible that you could win the lottery, or I could get hit by a car when I step outside to get the paper.
Quite often, asking if something is possible is the wrong question. The right question is what would be required.

In addition to purely hardware challenge of piggybacking extra storage chips, you would of course also need to replace all of the software in order to use any additional storage (bootloader, kernal and filesystem). Adding extra ram is much easier than adding extra storage, and may not require any software modifications (you may need a new kernel depending on how much ram you add, which would probably also mean a new bootloader, but not because of the extra ram).

TikaC
TikaC's picture

Glad to hear I got it all backed up after all. :)

I also was under the impression that if one recompiled the kernel and it was bigger than the existing one, that it would require more changes to the hardware to hold the larger kernel? (As you pointed out in the last post, I do have large gaps missing in my understanding of how these things work... So I hope you don't mind giving a hint here and there. :) )

Nocturnal
Nocturnal's picture

A different sized kernel would require a different bootloader. It would also require "moving" all of the partitions, and shrinking the user partition. "Moving" the partitions and shrinking the user partition is fairly simple though, especially if you've already got copy of all the partitions.

No hardware changes required.

TikaC
TikaC's picture

So in theory one could completely reprogram the system, and create a new kernel without upgrading the hardware. Awesome.

Helibot
Helibot's picture

Nocturnal said:

Does anyone know if you can add a login/getty daemon to a running linux system? Does it need any Kernal support?  

Yes you can add a login/getty daemon to a running linux system (your just spawning a process), I don't think it requires specific kernel support, but I could be wrong.

Is there a way to get the linux root filesystem from RSMedia, unpack it, make some changes, pack it pack up and put it back in the robot? If so then I can think of other things to try. Anyone else got any ideas??

Yes, but you have to end up with a file the is equal to or smaller than the original, and you'd better make sure you keep a copy of the original file system. I have some scripts that will decompress the filesystem, and then compress it again when you are done, but I don't really have time to dig them out right now. If you google cramfs you should be able to find all the information you need. Normally I'd be try all this myself (you may recall I have no patience :), unfortunately all my free time is accounted for at the moment.

Thanks for the info.  After further thoughts, I dont think a getty will help - it is still going to eventually call a new shell and redirct the input/output to the getty port. So I think that I will have the same problem.   So I have set out to fix this...
   After some further investigation, I found that the USB serial module (mx1ads_usbserial.o) was only allowing itself to be opened once - which is a problem since the command line causes it to be opened once for the stdin redirections and once for the stdout redirection.  So I allowed to 2 opens....and I can now send a single command from the PC to the RSMedia and get it executued.   BUT the command must be only one character long! AND the shell exits after it has executed it!

   Think I found a problem - looks like the mx1ads_usbserial.o module is behaving badly - it returns all the available data for a read (but the shell is calling read and only expecting 1 byte!). I think I need to rewite some of the  module read code tomorrow!!

Nocturnal, I would love to find out how to repack the root file system...but it will have to wait for the moment. If you find time to dig out the scripts that would be great , otherwise I will readup on cramfs at some stage.
   Cheers
       Helibot

Helibot
Helibot's picture

Hey all,
I rewrote the read code for the mx1ads_usbserial.o and it works much better, I can now successfully execute a string of commands on the PC and see the results back on the PC :-)
BUT there is still one BIG problem - the string of commands have to be queued up in the RSMedia USB buffer. If there is data in the buffer the shell will read it and execute them. But if there is no data (or it runs out of data), then the shell just exits!! AFAIK its not meant todo that!! It should default to a interactive shell (meaning it should wait for input).
Maybe its due to the mx1ads_usbserial.o command not blocking when it does a read (it just returns immediately with 0 bytes), this might upset the shell? (no error messages are being output at all)
Or Maybe the mx1ads_usbserial.o device is not implementing some required ioctrl functions and this causes the shell to exit.??
I found that busybox has a command for setconsole. But its not implemented on RSMedia busybox :-( - I will get the source for it tomorrow and see if that helps!!
Anyone else have any ideas about the shell exiting or how I can debug it?

It feels so close now!! I can see all the commands and results on the PC screen , but just cant get the blasted shell to stay open!!
Cheers from a frustrated
Helibot

TikaC
TikaC's picture

Have you tried to fork a shell instead? Or am I on the wrong page? It sounds like it is exiting after each command so maybe your terminal program needs a "keep alive" of some kind. Or a "wait for command" program to keep the shell busy listening for another command.

But don't be too frustrated! You already come so far with this and this is exciting news! :)

TikaC
TikaC's picture

Maybe also have the terminal automatically put an "&" at the end of each command so that the shell stays open? Don't know if that'll work.

Are you using SSH by any chance?

One place I find useful for Linux stuff is LinuxQuestions.org. They have bailed me out of a tough spot more than once. :) Might want to ask there about how to keep alive a shell.

Nocturnal
Nocturnal's picture

The shell is probably interpreting it as an EOF, and exiting.

TikaC
TikaC's picture

:: slaps self on head :: Now why didn't I think of that! I bet you're right! :) We'll see in Helibot's next report.

Helibot
Helibot's picture

Hi all,
Well another step forward and then another brick wall!!
Seems that the shell is expecting the read function to block.
So therefore it shouldnt be returning with a len 0 (which could cause the shell to look at the empty data and interpert it as an EOF and exiting - thanks Nocturnal)
So I added code to make the read function wait until the IRQ get more data. I found some referenece to other device drivers doing this and used the same methods (which is read block by calling func interruptible_sleep_on(&usb_serial_wait); and IRQ signals by calling wake_up_interruptible(&usb_serial_wait);)
So the read blocks and the Shell waits pateintly for more input - all looks good!!
But when the input arrives I get a kernal crash "Unable to handle kernel NULL pointer dereference at virtual address 00000005" and RSMEdia needs to be restarted :-(
I dont know kernel programming in detail or about these waiting functions, maybe I have done something wrong....I will read up on it tomorrow.
Does anyone know if there are better/different ways to make the read function block until more data is received?

Cheers
Helibot

TikaC
TikaC's picture

That "NULL pointer" I think is that it got a 0 (as you were mentioning). Or, your data is being sent as NULL. Somehow, the robot is recieving a NULL or EOF in the next data packet and crashing.

Perhaps this might help shine a light on the subject (Sorry for the long URL)

http://books.google.com/books?id=M7RHMACEkg4C&pg=PA94&lpg=PA94&dq=Unable...

Helibot
Helibot's picture

Hi all,
Another update for you - ITS WORKING!!. We can now successfully open a console session over the USB! We can type almost any linux command and see the results. So this is very cool, you can now have almost the same access to the console as with the serial port hack!

How does it work? Heres a short explaination:-
The RSMedia USBport usually runs in mass storage device mode (so the PC can see the SD Card). But we found we could remove this driver (mx1ads_usbd) and load another driver (mx1ads_usbserial). The mx1ads_usbserial driver makes the USB port seem like a basic serial device to the PC.

At the RSMedia end I changed the mx1ads_usbserial code so it could more easily accept data from the /dev/ttyUSB (or /tmp/ttyUSB) device. Then with the command 'sh /tmp/ttyUSB 2

At the PC end we used libusb_win32 (a free windows USB driver framework ) to create a driver that will recognise the RSMedia serial USB mode and exchange data with it. I wrote another program that uses this driver to send and receive the data from the USB and send it to a TCP port.
Finially we use a serial terminal program (like hyperterminal , or teraterm) on the PC to connect to the TCP port like a telnet console.

This allows all the fancy terminal functions (like colours, cursor movement, history, zmodem file transfer ) all to work just like it was connected directly to the serial port!!.

So as you can may imagine there , it wasnt a trivial task and there are still quite a few things not working fully :-(. But its definitely functional enough to release a first version!

TikaC and I are working to put the programs and instructions into another "RSMedia Development Kit" release. So stay tuned - I hope we can release it in a few days time!!.

Cheers
Helibot

TikaC
TikaC's picture

This is great news indeed! MUCH thanks to you Helibot and your talent at coding all this. (Folks, I suspect greatly that he is now sleep-deprived! LOL!)

Just one thing I want to mention about zmodem transfers... I could not get that working. That is not to say it won't work. Or might not work now but it might eventually. I don't know how to use the zmtx and zmrx. The help switch wasn't too much help, either. So that is still being investigated.

There's other things that you'll be made aware of but we'll have all the known issues put in the docs. Always RTFM! ;)

I tried this setup out on my own bot which does *not* have a serial port hack (yet, still) and it's working! Note that Helibot's bot didn't exactly come with a USBNet feature in the menus! So this should work on virtually any RS Media, USBNet enabled or not. The reason we are able to do this is because the sources and drivers for this are GPLed to begin with. So we got lucky there. :)

altwolf
altwolf's picture

Wow! this is really interesting! But...what does it mean? LOl.
What can one do now that you awesome peoples have figured this out? Can you write new programs or A.I. for Rsmedia now?

TikaC
TikaC's picture

It means that yes, you can do that. It means you can debug programs and watch what the results are, and see any error messages. With other items in the Dev kit, it's possible now to create your own programs in C and run them on the robot. You could even control the motors, etc. Once you have access to the main linux system, anything is possible.

Nocturnal
Nocturnal's picture

Helibot said:
At the PC end we used libusb_win32 (a free windows USB driver framework ) to create a driver that will recognise the RSMedia serial USB mode and exchange data with it. I wrote another program that uses this driver to send and receive the data from the USB and send it to a TCP port.

Would it not have been better to actually impliment a serial device, rather than a network port. After all, that would have allowed this to be used for what it was originally intended (ie ppp based networking) as well as anything else.

 

Helibot
Helibot's picture

Hey Nocturnal,
Yep , I did consider implementing a USB to Serial device (instead of USB to TCP). But there were several reasons why I chose USB to TCP. Mainly it would have been harder I think and also I thought it would be easier to make a linux only version (since libusb-win32 is also available for linux).
But now I understand it all a whole lot better, I may have a go at a USB to serial converter sometime in the future.
PPP based networking would be possible over serail, but it may also be possible over the TCP port as well, havent checked into this yet.

Cheers
Helibot.

Helibot
Helibot's picture

TikaC said: Just one thing I want to mention about zmodem transfers... I could not get that working. That is not to say it won't work. Or might not work now but it might eventually. I don't know how to use the zmtx and zmrx.

Tika is right, zmrx and zmtx are not working yet. In fact, there is still a problem with the read function not being a true blocking function call in the RSMedia USB driver. (I posted about the Kernel crash previously - this hasnt been solved yet, I have just worked around it.) So doing things with large amounts of data are not reliable yet. So zmrx/zmtx wont work yet :-(.
(Also means that PPP networking cant be investigated yet!) 
    But this first verion will be quite useable for gereral console access.

Cheers
   Helibot

FreddyA
FreddyA's picture

Very cool guys, this is good work, top quality breakthrough. I am awaiting the release to see if I can finally set the date and time! I was thinking, will it be possible to write a 3D editor that would give real time servo command execution? Am I dream'n

Pages