Personal Assistant (Jarvis) in Python

I thought it would be cool to create a personal assistant in Python. If you are into movies you may have heard of Jarvis, an A.I. based character in the Iron Man films. In this tutorial we will create a robot.

The features I want to have are:

For this tutorial you will need (Ubuntu) Linux, Python and a working microphone.

Related courses:


This is what you’ll create (watch the whole video, demo at the end):

Recognize spoken voice

Speech recognition can by done using the Python SpeechRecognition module. We make use of the Google Speech API because of it’s great quality.

Answer in spoken voice (Text To Speech)

Various APIs and programs are available for text to speech applications. Espeak and pyttsx work out of the box but sound very robotic. We decided to go with the Google Text To Speech API, gTTS.

sudo pip install gTTS

Using it is as simple as:

from gtts import gTTS
import os
tts = gTTS(text='Hello World', lang='en')"hello.mp3")
os.system("mpg321 hello.mp3")


Complete program

The program below will answer spoken questions.

#!/usr/bin/env python3
# Requires PyAudio and PySpeech.
import speech_recognition as sr
from time import ctime
import time
import os
from gtts import gTTS
def speak(audioString):
tts = gTTS(text=audioString, lang='en')"audio.mp3")
os.system("mpg321 audio.mp3")
def recordAudio():
# Record Audio
r = sr.Recognizer()
with sr.Microphone() as source:
print("Say something!")
audio = r.listen(source)
# Speech recognition using Google Speech Recognition
data = ""
# Uses the default API key
# To use another API key: `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
data = r.recognize_google(audio)
print("You said: " + data)
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print("Could not request results from Google Speech Recognition service; {0}".format(e))
return data
def jarvis(data):
if "how are you" in data:
speak("I am fine")
if "what time is it" in data:
if "where is" in data:
data = data.split(" ")
location = data[2]
speak("Hold on Frank, I will show you where " + location + " is.")
os.system("chromium-browser" + location + "/&")
# initialization
speak("Hi Frank, what can I do for you?")
while 1:
data = recordAudio()

Related posts:



40 thoughts on “Personal Assistant (Jarvis) in Python

  1. gokul gokul - April 15, 2017

    hey Frank , i am run the above program .But i did not gat any error .
    it shows
    [email protected]:~/Desktop# python3
    Hi Gokul, what can I do for you?”
    and it cannot move to next step.!
    what i do?

    1. Frank - April 16, 2017

      Verify that microphone input is processed by changing to:

      data = recordAudio()

      If no microphone data is received, try changing your microphone settings or one of the other speech recognition APIs. A list of speech engines can be found on

  2. Abhinav Prakash - March 26, 2017

    Hey mate!

    def recordAudio():
        # Record Audio
        r = sr.Recognizer()
        with sr.Microphone() as source:
            print("Say something!")
        audio = r.listen(source)

    last piece needs to be inside the with loop

  3. Mitchell Williamson - March 22, 2017

    mine keeps coming up with an error saying ImportError: No module called ‘speech_recognition’

    1. Mitchell Williamson - March 22, 2017

      OH! I didn’t configure the microphone on my laptop!

      1. Mitchell Williamson - March 22, 2017

        don’t worry!

  4. Brent - July 6, 2016

    Im using a RaspberryPi with USB webcam. I try the 1st scrip to test the TTS and it works awesome, but when I try the complete program it gives me the above error.

    1. Frank - July 15, 2016

      Make sure the indention (4 spaces) is correct.

  5. satyam - June 3, 2016

    hello! i am getting some error. Can you help me out . i googled it but couldnot find any solution..

    ALSA lib pcm_dsnoop.c:618:(snd_pcm_dsnoop_open) unable to open slave
    ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
    ALSA lib pcm_dmix.c:1022:(snd_pcm_dmix_open) unable to open slave
    Cannot connect to server socket err = No such file or directory
    Cannot connect to server request channel
    jack server is not running or cannot be started

    1. Frank - June 5, 2016

      Do you get this error with the top program or the bottom (complete) program? Are you using Ubuntu or another platform?
      I found this which may be helpful:

  6. Ankit - May 22, 2016

    How to compile and run that program

    1. Frank - May 23, 2016

      Download python from and run with “python”

      1. CJ Waller - July 10, 2016

        I usually use the command in terminal: python [name].py but programs will not run if the filament has a space in it? What should I do?

        1. Frank - July 16, 2016


          python test\


          python 'test'

Leave a Reply