python logo

jarvis python code


Python hosting: Host, run, and code Python in the cloud!

I thought it would be cool to create a personal assistant in Python. If you are into movies you may have heard of Jarvis, an A.I. based character in the Iron Man films. In this tutorial we will create a robot.

In this tutorial you get started with coding your own Jarvis, the voice activated assistant in Iron Man. Jarvis is a voice assistant, similar to Apple’s Siri or Google Now. In this tutorial we use the power of the Python programming language and a text-to-speech service.

I’m going to be using Ubuntu Linux for this project but you should be able to use it in Windows or Mac as well. However, since there’s quite a bit of command line work required, I’d recommend doing this on a Linux machine.

The features I want to have are:

For this tutorial you will need (Ubuntu) Linux, Python and a working microphone.

Related course:

Video


This is what you’ll create (watch the whole video, demo at the end):

Recognize spoken voice


Speech recognition can by done using the Python SpeechRecognition module. We make use of the Google Speech API because of it’s great quality.

Answer in spoken voice (Text To Speech)


Various APIs and programs are available for text to speech applications. Espeak and pyttsx work out of the box but sound very robotic. We decided to go with the Google Text To Speech API, gTTS.


sudo pip install gTTS

Using it is as simple as:


from gtts import gTTS
import os
tts = gTTS(text='Hello World', lang='en')
tts.save("hello.mp3")
os.system("mpg321 hello.mp3")

gtts

Complete program


The program below will answer spoken questions.


#!/usr/bin/env python3
# Requires PyAudio and PySpeech.

import speech_recognition as sr
from time import ctime
import time
import os
from gtts import gTTS

def speak(audioString):
print(audioString)
tts = gTTS(text=audioString, lang='en')
tts.save("audio.mp3")
os.system("mpg321 audio.mp3")

def recordAudio():
# Record Audio
r = sr.Recognizer()
with sr.Microphone() as source:
print("Say something!")
audio = r.listen(source)

# Speech recognition using Google Speech Recognition
data = ""
try:
# Uses the default API key
# To use another API key: `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
data = r.recognize_google(audio)
print("You said: " + data)
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print("Could not request results from Google Speech Recognition service; {0}".format(e))

return data

def jarvis(data):
if "how are you" in data:
speak("I am fine")

if "what time is it" in data:
speak(ctime())

if "where is" in data:
data = data.split(" ")
location = data[2]
speak("Hold on Frank, I will show you where " + location + " is.")
os.system("chromium-browser https://www.google.nl/maps/place/" + location + "/&")

# initialization
time.sleep(2)
speak("Hi Frank, what can I do for you?")
while 1:
data = recordAudio()
jarvis(data)


Related posts:


 

BackNext





Leave a Reply:




Ankit Sun, 22 May 2016

How to compile and run that program

Frank Mon, 23 May 2016

Download python from python.org and run with "python program.py"

satyam Fri, 03 Jun 2016

hello! i am getting some error. Can you help me out . i googled it but couldnot find any solution..

ALSA lib pcm_dsnoop.c:618:(snd_pcm_dsnoop_open) unable to open slave
ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started

Frank Sun, 05 Jun 2016

Do you get this error with the top program or the bottom (complete) program? Are you using Ubuntu or another platform?
I found this which may be helpful: https://askubuntu.com/questions/608480/alsa-problems-with-python2-7-unable-to-open-slave

Brent Wed, 06 Jul 2016

Im using a RaspberryPi with USB webcam. I try the 1st scrip to test the TTS and it works awesome, but when I try the complete program it gives me the above error.

CJ Waller Sun, 10 Jul 2016

I usually use the command in terminal: python [name].py but programs will not run if the filament has a space in it? What should I do?

Frank Fri, 15 Jul 2016

Make sure the indention (4 spaces) is correct.

Frank Sat, 16 Jul 2016

Use:


python test\ program.py


or


python 'test program.py'

Mitchell Williamson Wed, 22 Mar 2017

mine keeps coming up with an error saying ImportError: No module called 'speech_recognition'

Mitchell Williamson Wed, 22 Mar 2017

OH! I didn't configure the microphone on my laptop!

Mitchell Williamson Wed, 22 Mar 2017

don't worry!

Abhinav Prakash Sun, 26 Mar 2017

Hey mate!


def recordAudio():
# Record Audio
r = sr.Recognizer()
with sr.Microphone() as source:
print("Say something!")
audio = r.listen(source)


last piece needs to be inside the with loop

gokul gokul Sat, 15 Apr 2017

hey Frank , i am run the above program .But i did not gat any error .
it shows
"root@gokul-hp-notebook:~/Desktop# python3 jarvis.py
Hi Gokul, what can I do for you?"
and it cannot move to next step.!
what i do?

Frank Sun, 16 Apr 2017

Verify that microphone input is processed by changing to:


data = recordAudio()
print(data)
jarvis(data)


If no microphone data is received, try changing your microphone settings or one of the other speech recognition APIs. A list of speech engines can be found on https://pypi.python.org/pypi/ SpeechRecognition

Mahir Mahbub Sun, 16 Apr 2017

When I run the script,It shows,


"Traceback (most recent call last):
File "C:\Users\mahir\Desktop\say.py", line 51, in
data = recordAudio()
File "C:\Users\mahir\Desktop\say.py", line 18, in recordAudio
audio = r.listen(source)
File "C:\Users\mahir\AppData\Roaming\Python\Python35\site-packages\speech_recognition\__init__.py", line 531, in listen
assert source.stream is not None, "Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"
AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"

Frank Sun, 16 Apr 2017

Try another speech engine, maybe this one is not working. Sometimes the APIs change.

Jeff C. Sat, 22 Apr 2017

Here is the error I got, Frank. Any IDEA?


How may I assist you?
sh: 1: mpg321: not found
ALSA lib pcm_dsnoop.c:606:(snd_pcm_dsnoop_open) unable to open slave
ALSA lib pcm_dmix.c:1029:(snd_pcm_dmix_open) unable to open slave
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm_dmix.c:1029:(snd_pcm_dmix_open) unable to open slave
Cannot lock down 82274202 byte memory area (Cannot allocate memory)
Say something!
Traceback (most recent call last):
File "index.py", line 54, in
data = recordAudio()
File "index.py", line 21, in recordAudio
audio = r.listen(source)
File "/usr/local/lib/python3.5/dist-packages/speech_recognition/__init__.py", line 525, in listen
assert source.stream is not None, "Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"
AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?

Frank Sat, 22 Apr 2017

mpg321 is missing, install it to your system. If that doesn't solve all, change the speech engine too.

Jeff C. Mon, 24 Apr 2017

No problem. Ok. Thanks

Shubham Bhuyan Wed, 03 May 2017

In that try-except block, if i don't say something for a short period of time it says "Google Speech Recognition could not understand audio" and exits my program.(I am using the code to make a voice controlled bot. So after each command I need time to make bot move. Giving delay makes a fixed time for each order,so i don't want to use it.) Is there any way to control the time before the except block starts working??

kumar rx Thu, 04 May 2017

Hi mate, I have downloaded gTTS, now what i want to do and where to save the both py files, whether it should get saved in separate file or in same file... And another doubt is you are saving that hello.mp3 what is that ?

Frank Fri, 05 May 2017

That looks like another type of exception.
It may be another type of exception the try-catch block is getting.
Try adding these two exception handlers:


except sr.UnknownValueError:
speak("I don't understand!")
except sr.RequestError as e:
print("Could not request results")
print("from Google Speech Recognition service; {0}".format(e))


Let me know how that works out.

Frank Fri, 05 May 2017

Save as different py files. The file hello.mp3 is the output file saved automatically. You'll also need to install the program mpg321.

Edward Principe Thu, 27 Jul 2017

Frank, I love the quality and execution of this program. I intend to build an interface to run some scientific equipment. I am not a programmer .... I generally hack my way through what I need to get the job done. I have written several basic programs to control the microscope.

This is a Windows 8.1 system. Is that an issue??
Installed the gTTS and SpeechRecognition. Having trouble getting PyAudio and PySpeech installed .... using python 3.3 and seems to need Visual C++ 10.0. Trying to work around that now. ....

When I try to run your example code (short version), I get a string of errors, the end of which oddly seems tied to a URL related to 'translate.google.com'.... if I interpret the error correctly.


File "C:\Python33\lib\site-packages\requests\adapters.py", line 504, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='translate.google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:547)'),))


I know it is a mess ...... any insights are appreciated!

BTW: It is tts.save that generates the error.

Frank Fri, 28 Jul 2017

Thanks Edward! Windows 8.1 should not be an issue, at the time I had tested it on Ubuntu.
The gTTS module underneath uses the translate.google.com website, see inside the gtts source code. This website returns an audio file, which is played with any sound player (mpg321 as example).

In this case I see a connection error, do you have a firewall? It may also be throttling (too many connections). If you have an offline environment, try ms sapi or espeak. The speech recognition part also needs internet connection though.

Pete Fri, 29 Jan 2020

How do you create an mp3 file from spoken via the Google TTS API?

Frank Sun, 31 Jan 2020

If you just want an mp3, you can save the TTS output .save('hello.mp3'). If you want to save the spoken audio, you can do this:

r = sr.Recognizer()
with sr.Microphone() as source:
audio = r.listen(source)

with open('microphone-results.wav','wb') as f:
f.write(audio.get_wav_data())
You can save as raw, wav, aiff and flac. For mp3, you may need to converse it using another module or it may have been added.

Pete Sun, 14 Feb 2021

How do you install mpg321? Because I keep getting:

ALSA lib pcm_dmix.c:1075:(snd_pcm_dmix_open) unable to open slave ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71 ALSA lib setup.c:547:(add_elem) Cannot obtain info for CTL elem (MIXER,'IEC958 Playback Default',0,0,0): No such file or directory ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port ALSA lib pcm_oss.c:377:(_snd_pcm_oss_open) Unknown field port ALSA lib pcm_a52.c:823:(_snd_pcm_a52_open) a52 is only for playback ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card ALSA lib pcm_usb_stream.c:486:(_snd_pcm_usb_stream_open) Invalid type for card ALSA lib pcm_dmix.c:1075:(snd_pcm_dmix_open) unable to open slave Cannot connect to server socket err = No such file or directory Cannot connect to server request channel jack server is not running or cannot be started JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

Frank Sun, 14 Feb 2021

Instead of mpg321, try mpg123. You can install it with your package manager, but any audio player should do.

Naman Jain Mon, 22 Mar 2021 there is an error coming for where is place!
Say something!
You said: where is Ropar
Hold on Frank, I will show you where Ropar is.
'chromium-browser' is not recognized as an internal or external command, operable program or batch file. 'amp' is not recognized as an internal or external command, operable program or batch file.
Frank Mon, 22 Mar 2021 You should install the chromium browser or specifiy the exact path to the browser on your computer.
Shivam Mon, 15 May 2021 Sir in my pc jarvis is using internet explorer as default browser how can i change it to chrome please suggest.
Frank Mon, 15 May 2021 Change the command in os.system to include path.
bankim barman 2023-01-28T19:08:35.410Z

AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?
PS C:\Users\BANKIM\OneDrive\Desktop\python program> c:; cd 'c:\Users\BANKIM\OneDrive\Desktop\python program'; & 'C:\Users\BANKIM\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\BANKIM\.vscode\extensions\ms-python.python-2022.20.2\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '61547' '--' 'c:\Users\BANKIM\OneDrive\Desktop\python program\convert.py'

Swaraj 2022-07-30T15:24:43.049Z

Hi, how to install pyaudio in android

Frank 2022-07-30T15:24:44.049Z

pyaudio is not available for android, because it wraps PortAudio. PortAudio works on desktop platforms like Windows, Linux, Mac OS but not mobile

UHO 2022-01-17T01:25:22.457Z

data = r.recognize_google(audio)
Is it possible to return data and make the returned data into a text file?
And can I change the STT language to Japanese?

Frank 2022-01-17T01:25:22.457Z

Yes you can store returned data into a file, it is a Python string that you can write to a file like any other Python string. If the API allows it, you can change the language to Japanese

Abdo 2021-11-09T07:17:01.479Z

I tried to write your code but it doesn't seem to work for me, why am I getting this message? can you please help.


Playing MPEG stream from audio.mp3 ...
MPEG 2.0 layer III, 32 kbit/s, 24000 Hz mono

[0:02] Decoding of audio.mp3 finished.
Say something!
Traceback (most recent call last):
File '/Users/abdulrahmanalhashmi/Desktop/import speech_recognition as sr.py', line 54, in
data = recordAudio()
File '/Users/abdulrahmanalhashmi/Desktop/import speech_recognition as sr.py', line 21, in recordAudio
audio = r.listen(source)
File '/Users/abdulrahmanalhashmi/Library/Python/3.8/lib/python/site-packages/speech_recognition/__init__.py', line 594, in listen
assert source.stream is not None, 'Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?'
AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?

Frank 2021-11-09T07:17:02.479Z

Try another speech recognition API, this one may be deprecated

Shivam Shrivastav 2021-05-06T05:44:30.335Z

Sir in my pc jarvis is using internet explorer as default browser how can i change it to chrome please suggest.

Frank 2021-05-06T05:45:30.335Z

Change your default browser or run a custom command in the function, with the os.system function you can start any program