Personal Assistant (Jarvis) in Python


I thought it would be cool to create a personal assistant in Python. If you are into movies you may have heard of Jarvis, an A.I. based character in the Iron Man films. In this tutorial we will create a robot.

The features I want to have are:

For this tutorial you will need (Ubuntu) Linux, Python and a working microphone.

Related courses:

Video

This is what you’ll create:

Recognize spoken voice

Speech recognition can by done using the Python SpeechRecognition module. We make use of the Google Speech API because of it’s great quality.

Answer in spoken voice (Text To Speech)

Various APIs and programs are available for text to speech applications. Espeak and pyttsx work out of the box but sound very robotic. We decided to go with the Google Text To Speech API, gTTS.

sudo pip install gTTS

Using it is as simple as:

from gtts import gTTS
import os
tts = gTTS(text='Hello World', lang='en')
tts.save("hello.mp3")
os.system("mpg321 hello.mp3")

gtts

Complete program

The program below will answer spoken questions.

#!/usr/bin/env python3
# Requires PyAudio and PySpeech.
 
import speech_recognition as sr
from time import ctime
import time
import os
from gtts import gTTS
 
def speak(audioString):
    print(audioString)
    tts = gTTS(text=audioString, lang='en')
    tts.save("audio.mp3")
    os.system("mpg321 audio.mp3")
 
def recordAudio():
    # Record Audio
    r = sr.Recognizer()
    with sr.Microphone() as source:
        print("Say something!")
    audio = r.listen(source)
 
    # Speech recognition using Google Speech Recognition
    data = ""
    try:
        # Uses the default API key
        # To use another API key: `r.recognize_google(audio, key="GOOGLE_SPEECH_RECOGNITION_API_KEY")`
        data = r.recognize_google(audio)
        print("You said: " + data)
    except sr.UnknownValueError:
        print("Google Speech Recognition could not understand audio")
    except sr.RequestError as e:
        print("Could not request results from Google Speech Recognition service; {0}".format(e))
 
    return data
 
def jarvis(data):
    if "how are you" in data:
        speak("I am fine")
 
    if "what time is it" in data:
        speak(ctime())
 
    if "where is" in data:
        data = data.split(" ")
        location = data[2]
        speak("Hold on Frank, I will show you where " + location + " is.")
        os.system("chromium-browser https://www.google.nl/maps/place/" + location + "/&")
 
# initialization
time.sleep(2)
speak("Hi Frank, what can I do for you?")
while 1:
    data = recordAudio()
    jarvis(data)

Related posts:

 


18 thoughts on “Personal Assistant (Jarvis) in Python

  1. Jeff C. - April 22, 2017

    Here is the error I got, Frank. Any IDEA?

    How may I assist you?
    sh: 1: mpg321: not found
    ALSA lib pcm_dsnoop.c:606:(snd_pcm_dsnoop_open) unable to open slave
    ALSA lib pcm_dmix.c:1029:(snd_pcm_dmix_open) unable to open slave
    ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
    ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
    ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
    ALSA lib pcm_dmix.c:1029:(snd_pcm_dmix_open) unable to open slave
    Cannot lock down 82274202 byte memory area (Cannot allocate memory)
    Say something!
    Traceback (most recent call last):
      File "index.py", line 54, in 
        data = recordAudio()
      File "index.py", line 21, in recordAudio
        audio = r.listen(source)
      File "/usr/local/lib/python3.5/dist-packages/speech_recognition/__init__.py", line 525, in listen
        assert source.stream is not None, "Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"
    AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?
    1. Frank - April 22, 2017

      mpg321 is missing, install it to your system. If that doesn’t solve all, change the speech engine too.

  2. Mahir Mahbub - April 16, 2017

    When I run the script,It shows,

    "Traceback (most recent call last):
      File "C:\Users\mahir\Desktop\say.py", line 51, in 
        data = recordAudio()
      File "C:\Users\mahir\Desktop\say.py", line 18, in recordAudio
        audio = r.listen(source)
      File "C:\Users\mahir\AppData\Roaming\Python\Python35\site-packages\speech_recognition\__init__.py", line 531, in listen
        assert source.stream is not None, "Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"
    AssertionError: Audio source must be entered before listening, see documentation for ``AudioSource``; are you using ``source`` outside of a ``with`` statement?"
    1. Frank - April 16, 2017

      Try another speech engine, maybe this one is not working. Sometimes the APIs change.

  3. gokul gokul - April 15, 2017

    hey Frank , i am run the above program .But i did not gat any error .
    it shows
    [email protected]:~/Desktop# python3 jarvis.py
    Hi Gokul, what can I do for you?”
    and it cannot move to next step.!
    what i do?

    1. Frank - April 16, 2017

      Verify that microphone input is processed by changing to:

      data = recordAudio()
      print(data)
      jarvis(data)

      If no microphone data is received, try changing your microphone settings or one of the other speech recognition APIs. A list of speech engines can be found on https://pypi.python.org/pypi/SpeechRecognition