Personal Assistant (Jarvis) in Python

I thought it would be cool to create a personal assistant in Python. If you are into movies you may have heard of Jarvis, an A.I. based character in the Iron Man films. In this tutorial we will create a robot.

The features I want to have are:

For this tutorial you will need (Ubuntu) Linux, Python and a working microphone.

Related course:


This is what you’ll create (watch the whole video, demo at the end):

Recognize spoken voice

Speech recognition can by done using the Python SpeechRecognition module. We make use of the Google Speech API because of it’s great quality.

Answer in spoken voice (Text To Speech)

Various APIs and programs are available for text to speech applications. Espeak and pyttsx work out of the box but sound very robotic. We decided to go with the Google Text To Speech API, gTTS.

sudo pip install gTTS

Using it is as simple as:

from gtts import gTTS
import os
tts = gTTS(text='Hello World', lang='en')
os.system("mpg321 hello.mp3")


Complete program

The program below will answer spoken questions.

#!/usr/bin/env python3
# Requires PyAudio and PySpeech.

import speech_recognition as sr
from time import ctime
import time
import os
from gtts import gTTS

def speak(audioString):
tts = gTTS(text=audioString, lang='en')
os.system("mpg321 audio.mp3")

def recordAudio():
# Record Audio
r = sr.Recognizer()
with sr.Microphone() as source:
print("Say something!")
audio = r.listen(source)

# Speech recognition using Google Speech Recognition
data = ""
# Uses the default API key
# To use another API key: `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
data = r.recognize_google(audio)
print("You said: " + data)
except sr.UnknownValueError:
print("Google Speech Recognition could not understand audio")
except sr.RequestError as e:
print("Could not request results from Google Speech Recognition service; {0}".format(e))

return data

def jarvis(data):
if "how are you" in data:
speak("I am fine")

if "what time is it" in data:

if "where is" in data:
data = data.split(" ")
location = data[2]
speak("Hold on Frank, I will show you where " + location + " is.")
os.system("chromium-browser https://www.google.nl/maps/place/" + location + "/&")

# initialization
speak("Hi Frank, what can I do for you?")
while 1:
data = recordAudio()

Related posts:



40 thoughts on “Personal Assistant (Jarvis) in Python

  1. andre violante
    - March 20, 2018

    Hey Frank,

    Cool post. I got the code to run and can speak to Jarvis while receiving a reply. However, I can not hear the audio. The audio file is being saved correctly. I haven’t added anything different to the original code you posted.


    1. Frank
      - March 27, 2018

      Does the audio file play in other players? Verify if mpg321 can play as standalone

  2. Tim Vogt
    - February 12, 2018

    How to use dutch as speak + voice command language?

    1. Frank
      - February 14, 2018

      For speech you could use the module gtts, it supports dutch using the ‘nl’ parameter.
      After saving the file, play it using one of the local audio players (or another module).

      from gtts import gTTS
       tts = gTTS(text='Hoe maakt u het?', lang='nl', slow=False)

      For speech recognition you can use the speech_recognition module and embed a language parameter. Most of them are external APIs that do speech recognition remotely. The implementation will depend on which external server is used, lets say we use the google API, this should do it:

      r.recognize_google(audio, language="nl-NL")

      Again, it will depend on the server endpoints and modules, this is one possible solution.

  3. SIlva Dexter
    - January 11, 2018

    Hi Frank, for the google , do we have to install any google cloud package ??

    1. Frank
      - January 20, 2018

      No, that’s not necessary for this example

  4. Prudhvi Chaitanya
    - December 16, 2017

    i have problem with Text to Speech.
    I have installed both pyttsx and gTTS .
    but i am not able to get the jarvis voice on my PC

    1. Frank
      - December 25, 2017

      It may be a playback problem on your computer, do you have the audio file?

  5. Ryan Holland
    - December 8, 2017

    I have speechrecognition installed but when I try to run the program it says importerror no module named speech_recognition. Help!

    1. Frank
      - December 8, 2017

      The module should work with Python 2.7 up to 3.6. Did you run: pip install SpeechRecognition ?

      1. Ryan Holland
        - December 8, 2017

        Yes I used pip install SpeechRecognition and it says installed but when I run the code it says no module found.

      2. Ryan Holland
        - December 9, 2017

        I got speechrecognition to work but now when i run the program it won’t speak or show where something is like in your video.

  6. Emre aşkan
    - September 5, 2017

    First of all, thank you Frank! In recordAudio function, audio = r.listen(source) line there is an indentation mistake. It wasn’t working for me at first and I thought that there was problem about jackd and alsa. But the problem is just a little indentation mistake. Just include the audio line into the with function by putting a space.

    1. Frank
      - September 6, 2017

      You’re right, thanks Emre!

Leave a Reply

Login disabled