RSMedia Text-to-Speech finally works!!

35 posts / 0 new
Last post
Helibot
Helibot's picture
RSMedia Text-to-Speech finally works!!

Hi All,

   I have a christmas present for you all... RSMedia can now say anything you type in!!!

 I finally got the 'flite' text to speech synthensier program working for RSMedia.
So you can type flite -t "Hello world, I am now a talking robot"  and he says exactly that!!
The voice is very 'electronic' but understandable (most of the time!).

I have uploaded the program to sourceforge. Just download the zip file from here. (4Mb download)
See the flite_rsmREADME.TXT file in the archive to see how to use it.

In the archive I have included a few wav files of some example output. Here are the command I used to generate them:-
 ./flite_rsm -t "the lazy fox jumped over the brown cow" /tmp/test.wav
 ./flite_rsm -t "Hello I am are ess Media and thank to f lite I can finally talk" /tmp/hello_flite.wav
./flite_rsm -t "Merry christmas and happy New year to you all" /tmp/merryxmas.wav
(If you specify a file name at the end of the command then the data is written to a file. If no file is specified then the wav file is played immediately)

So please download it and have a play....You will need the USB console to be able to type the commands in.

I hope you all have a VERY MERRY (& CHATTY) CHRSTMAS AND A HAPPY NEW YEAR!

I have put some notes and details of the modifications needed to build the working RSMeida version here 

Cheers
    Helibot

Here is a brief background on the Flite text to speech program:-
See the web page here http://www.speech.cs.cmu.edu/flite/
The released version of flite 1.0 for ARM runs on RSMedia , but it runs at double speed, so he talks really quickly.
 I eventually fixed this - by forcing Flite to output stereo data (instead of mono). I also fixed a few other issues - it seems that RSMedia Audio driver is not quite 100% linux compatible, or maybe it is just very old??) 

Note - There are newer flite versions 1.1,1.2 & 1.3 but none of these work when recompiled for RSMedia. But I may be able to get them to work now (since I have learned a bit about how the program works!). These newer versions have more features so its probably worth doing, so its on my todo list!!

Santa Matt
Santa Matt's picture

Great! I'll be trying this tonight.

FreddyA
FreddyA's picture

This is outstanding. I tried it out already and i't works perfectly. I cant wait to have it read my email (somehow).

Two notes on the tts;
1) I cant believe my RSM talks now.
2) ./flite-rsm -t "this is so freek'n awesome that I cant stop playing with it"

FreddyA
FreddyA's picture

This sounds familiar, take a guess from what movie this line is from.
./flite_rsm -t "Do you want to play a game?"

Santa Matt
Santa Matt's picture

FreddyA said:
This sounds familiar, take a guess from what movie this line is from.
./flite_rsm -t "Do you want to play a game?"

War Games!

 

 

FreddyA
FreddyA's picture

Yep, the RSM tts voice tone is similar to WOPR / Joshua.

Helibot
Helibot's picture

The docs on flite state that the voice included is very poor. But that it should be compatible with other (higher quality) 'vox' voices availble for 'Festival'. So it should be possible to get better/different voices.....

Cheers
Helibot

Robostuff
Robostuff's picture

Wow, Christmas came early this year! Thanks for this Helibot, and making it easy for us novice programmers to use!

FreddyA
FreddyA's picture

I can understand the voice pretty well, and it would be cool to have additional choices.

Helibot
Helibot's picture

I found quite a few words thae flite doesnt understand or synthersie well, to get him to say them better you can spell them phonetically (eg 'are ess media' worked well for 'RSMedia').
You can also make it say the words saved in a file by using
./flite_rsm -f /tmp/words.txt
But if you make the sentances too long , I think it will slow down while he is talking or lose some words.

Cheers
Helibot

Robostuff
Robostuff's picture

Could you enter commands in the V2 Alarm config file to run flite speech commands? Love that ini file so easy to add and edit stuff.

Helibot
Helibot's picture

Hi Robostuff,
It should work OK if you use the format flite_rsm -f filename.
If you use flite_rsm -t "hello" then the I think the quotes ('"') may get lost.
So you would need

[alarm01]
action = SPEAK:
...
[actioncmds]
SPEAK = ./flite_rsm -f
or
SPEAK = /mnt/sd/Application/flite_rsm -f

I'll be offline for a Xmas break now , so I hope you all have a great christmas.
So I'll let RSMedia do the talking for now :-)....

Cheers
Helibot

Shai Alyt NEMO
Shai Alyt NEMO's picture

Great work, Helibot!

It's really a dream come true. Fantastic Christmas gift for Community!

So, Merry Chirstmas and Happy New Year to you!!!

(P.S. It's not my cat, just found on the Net)

Robostuff
Robostuff's picture

Thats a cool pic, time to revise the cool cats pictures thread. I think he is sleeping happily now that his RS Media can finally speak lol

FREEEK
FREEEK's picture

This is INCREDIBLE! SUPERB!
It's almost like going from bicycle to motorcycle !!!

Sorry for my not quite fitted comparison.

Helibot
Helibot's picture

Hi All,
   I have now got Flite v1.3 working. I think it sounds a bit better and runs a bit quicker. 

I have also compiled the flite_time program - this is a demo program that comes with flite. It reads out the time in a human friendly format. The voice in the time program is better than the flite_rsm program - it sounds very natural (in comparison). So its interesting to have a listen too. It would be good to use this for a RSMAlarmClock action - it could speak the time for an alarm, or use  recurring alarm to tell you the time each hour.

Download the flite_rsm_1.3 from here.
Download the flite_time_rsm from here.

The source and instructions for the V1.3 changes are also in sourceforge site here.

I have also just read that there is also a version 1.4 of flite that includes three (yes 3) different voices ......So I think this should be tried on RSMedia as well.
Any volunteers to try compiling and running it?
(I have started looking at Networking again, so I want to keep going on that) 

BTW I had an absolute hoot of a time making the robot talk my 4yr old son yesterday.... my son was answering him back and laughing his head off (while I was typing madly and trying to keep a straight face!). In the end my son agreed the robot could take his photo , but he wouldnt believe the robot when he told him it was bedtime! 

 Cheers
    Helibot

Helibot
Helibot's picture

Hi All,
  Just a quick note to say that the Flite program is setup so most of it is in a library, so it should be quite easy to include the library into another C program and have that program speak text, I think you will just need a few lines of C to make it happen.

   Also I summerised some of the commands that the RSMedia audiodevice supports (and doesnt support). Its probably useful to anyone else wanting to write (or port) audio programs for RSMedia.

Here is some info:-
   It only supports Stereo output (if you set mono it plays it as stereo and everything sounds twice as fast).
   It supports at least 16bit signed data.
   It supports at least 16Khz and 32Khz - it does not support 8Khz (if you set 8Khz it plays it really fast (at 32khz I guess?)
  It doesnt support any IOCTRL GETS (but does support some SETS).

Here is a list of what I found was and wasnt supported:-

  • SNDCTL_DSP_RESET - OK
  • SNDCTL_DSP_SYNC - Fails (it does NOT seem to block the caller until all data is processed.)
  • SNDCTL_DSP_SETFMT - Ok (at least AFMT_S16_LE does work)
  • SNDCTL_DSP_CHANNELS - OK (1=Mono , 2 = stereo)
  • SNDCTL_DSP_STEREO -Fails (returns -1)
  • SNDCTL_DSP_SPEED - OK (at least 16000,22050 and 32000 work. Note that 8000 does not work)
  • SNDCTL_DSP_SETFRAGMENT - Unknown? (can be set successfully but the settings does not seem to affect behaviour at all)
  • SNDCTL_DSP_GETFMTS -Fails (returns -1)
  • SNDCTL_DSP_GETISPACE -Fails (returns -1)
  • SNDCTL_DSP_GETOSPACE -Fails (returns -1)

 

FreddyA
FreddyA's picture

Thanks helibot, I can tell the difference and the voice for the time is quite believable. Could you give an example of how to have it say the time every hour by recuring alarm? Or even for it to announce the alarm time? I tried a new actioncmd:
SPEEKTIME = /mnt/sd/Application/flite_time_rsm -t
and
action = SPEEKTIME:flite_time_rsm 06:00
Also, any chance for 12/24 hour option later? I would give the 1.4 a go but it is a huge learning curve beyond me now.
Keep up the work on that ppp, it will be interesting to see the ideas that will fly. I've had a good time with this new stuff you are doing thanks Helibot

Helibot
Helibot's picture

Hi Freddy,

We need a way to get the current time in the format HH:MM. Fortunately the linux date command can do this. Use date +%0H:%0M will output 23:09 at 11:09pm.

   So using this we can make the commands for a recurring alarm.
[alarm1]
#Recuring alarm to speak the time on the hour
enabled = Y
time = 8:00
action = TIME_:
recur = 60m

...
[action]
action = TIME_:./flite_time_rsm "`date +%0H:%0M`"

I think you could also use
[alarm1]
#Recuring alarm to speak the time on the hour
enabled = Y
time = 8:00
action = NOCMD:./flite_time_rsm "`date +%0H:%0M`"
recur = 60m

(By the way I have NOT checked these actually work in an alarm. Let me know if they dont!! I have checked that ./flite_time_rsm `date +%0H:%0M` from the command line does actually say the time correctly.)

BTW Freddy - You tried to use "SPEEKTIME" for the action name, but this wont work - because the length MUST be exactly 5 characters. So SAY__ or SPEAK or TIME_ would all be OK.

Cheers
   Helibot

FreddyA
FreddyA's picture

Helibot, Thanks for clearing that up I did get some results from those examples. The NOCMD has not worked for me yet but by using 5 characters the new actioncmds works using ./flite_time_rsm `date +%0H:%0M` I've got it telling me the time every 30m its great. Thanks again.

FreddyA
FreddyA's picture

Hey Helibot, I've been trying to get a random tts alarm going. Here is what i've been trying.

[alarm19]
enabled = Y
action_rnd = NOCMD:/mnt/sd/Application/Random
recur = 2m
recurcount = 10

I placed a folder nemed Random with several .sh files containing this script SPEEK:./flite_rsm_1.3 -t "random text" (made with santa matt's editor)

Also tried just ./flite_rsm_1.3 -t "random text" but no go

What would I be doing wrong?

Helibot
Helibot's picture

Hi Freddy,
I think if you have the .sh file to contain just the line
/mnt/sd/Application/flite-bin/flite_rsm_1.3 -t "hello world"
(Also I must admit I have not tried random with NOCMD: action. So maybe there is a bug in the alarmclock program?)

So you should also be able to use the -f feature of flite. This is probably the best solution.....
Use action_rnd = SAY_F:/mnt/sd/Application/Random
and define
[action]
action = SAY_F:./flite_time_rsm -f

Finially make files in .../Applictation/Random directory. Each file should contain the text you want the robot to say. Eg if you want to have it say good evening then make a file with just the words "good evening" in it. Call the files anything you like (but I would avoid putting spaces in the file names).
(Once again I havent tried this but it should work!!!)

Cheers
Helibot

FreddyA
FreddyA's picture

Hey Helibot, I got NOCMD to work with your first suggestion but the second example had a problem where it reads additional string info before and after the random text file and i did change from your example the /flite_time_rsm -f to /flite_rsm_1.3 -f. Now the NOCMD works like a charm as random action, says the time every 30m and blurts out random text every 5 minutes. Thanks for your help this is awesome.

Helibot
Helibot's picture

Hi Freddy,
Great, I'm glad all working for you. Could you post the working ini file sections here I and others can see how you got it going?
Cheers
Helibot

FreddyA
FreddyA's picture

ok here it is. I made this alarm

[alarm19]
enabled = Y
action_rnd = NOCMD:/mnt/sd/Application/Random
recur = 40s

this action

SPEEK = ./flite_rsm_1.3 -f

And a folder named Random in the application folder containing several .sh files with this

/mnt/sd/Application/flite_rsm_1.3 -t "your text here"

That works for me and in this case I have 11 files in the Random folder and it says them at randome every 40 seconds.

And on one .sh file I put a take picture action and more text after and it will say the text, take a picture and say the rest of the text. Very cool possibilities just on the verge.

FreddyA
FreddyA's picture

For the Flite time I used this alarm

[alarm16]
time = 7:00am
enabled = Y
action = SPKTM:
recur = 30m
recurcount = 26

and this action

SPKTM = /mnt/sd/Application/flite_time_rsm `date +%0H:%0M`

This one will say the time every half hour from 7am to 8pm I think and stops, starts over the next day at 7 again. works nicely

gerber
gerber's picture

Hi

I was wandering is there away to change the pitch of the voice using flite v1.3 on the RS media or is it only possible to get the robot to say text in in one way?

Are there any other command than -t in (e.g. flite_rsm_1.3 -t "hello world")

Have you managed to compile Flite v1.4 and get it working on the robot?

The reason I am interested in this is because I would like to use flite in conjuction with the multibot software I am developing so that I can make each robot voice sound slightly different when they speak to each other by altering either the voice,  speed or pitch. (I know the speed can be made double by running the recorded sound though the rsm media player)

cheers
Gerber

FreddyA
FreddyA's picture

Hi Gerber, you are right, I don't think theres many options for the 1.3 voice. However, theres some additional vices that may work in 1.3.

Helibot said: "The docs on flite state that the voice included is very poor. But that it should be compatible with other (higher quality) 'vox' voices availble for 'Festival'. So it should be possible to get better/different voices....."

Also, in regards to 1.4, Helibot said: "I have also just read that there is also a version 1.4 of flite that includes three (yes 3) different voices ......So I think this should be tried on RSMedia as well.
Any volunteers to try compiling and running it?"

So I'd say he has handed this one off to anyone who has the skill to take it over. Helibot has a few huge projects in the works and probably hasnt touched flite since then.

And without trying to stray to far from the subject, I visited the flite site and found this link leading to intresting bits about using the software there for speech recognition on iphone and ipad and the kinect sensor while performing object recognition. http://cmusphinx.sourceforge.net/

I sure hope maybe theres something there for voice recognition on linux?.

Freddy

gerber
gerber's picture

Hi

I wish I new how to program and compile C programs as I would have a go at compiling Flite 1.4.

Freddy Said > "But that it should be compatible with other (higher quality) 'vox' voices
availble for 'Festival'. So it should be possible to get better/different voices....."

Question 1 - I tried but couldn't find any VOX voices. Do you have a web link?

Question 2 - How do you install Vox voices for Flite so they canl work on the RSmedia? Do they need to be compile in the Flite software?

 

Helibot
Helibot's picture

Hello Gerber and Freddy,
I haven't tried any other voices for flite. And I havenÂ’t got back to looking at compiling newer versions.

Reading a bit further I found this comments about Flite1.4 (at http://www.speech.cs.cmu.edu/flite/doc/flite_8.html#SEC21)
“As of 1.2 initial scripts have been added to aid the conversion of FestVox voices to Flite. In general the conversion cannot be automatic. For example all specific Scheme code written for a voice needs to be hand converted to C to work in Flite, this can be a major task. “
So it sounds like it will not be trivial todo. I guess you would need to know linux/unix, C ,compiling and some more about how the voices work.

But the last lines in the version1.4 release note say "Though in the mean time, a few higher quality voices will be released with the next version." Indicating that 1.5 will include a variety of voices that are also better quality.

For the moment I think a better option is too see what options are available in flite to vary the current voice. I think that pitch and speed were options in flite??. But not until later versions (than 1.0). So we need to compile a newer version (hopefully 1.4) for RSMedia. If anyone wants to try this I am happy to pull together some notes on how todo the compile in the Ubuntu 10.1 VM and help out if you need some guidance.
If no one wants to try then I will try it mself but currently I am in the middle of something else.....

.....and it happens to be exactly what Freddy mentioned - speech recognition using ‘pocketsphinx’ (which is a variation of cmusphinx for smaller embedded platforms.) I have been working on it for about two weeks and a few nights ago I got a version that DID recognise some speech when running on RSMedia !! (But it took 9 seconds to recognise 4 seconds of speech :-( and the speech was from a wav type file, it was not recorded from the microphone.) So next I am trying to get the library to read data from the microphone input. But (as with Flite software) the crappy implementation of OSS sound drivers in RSMedia mean that the standard software just crashes. So I am in the middle of debugging this now. But i really dont know yet if it will all work on RSMedia - currently I give it a 50/50 chance!!

Cheers
Helibot

Helibot
Helibot's picture

Hi Gerber,
To answer your questions:-
>Question 1 - I tried but couldn't find any VOX voices. Do you have a web link?
See http://festvox.org/index.html. There is a good demo page that you can hear the voices on.

>Question 2 - How do you install Vox voices for Flite so they canl work on the RSmedia?Do they need to be compile in the Flite software?
Yes they need to be converted by scripts and compiled in. But as mentioned in the previous posts it seems to also need some coding to complete the conversion :-(
Cheers
Helibot

Pages