chatGPT on top of Amazon Alexa

The skill is not free for me. Beside the cost of chatGPT, I have poured so many hours over the past months. So if anybody wants to support me, he/she can do through this paypal link:

You can find it also in my youtube channel
Revolutionary Voice - YouTube
It is just a container of videos about these chatgpt based Alexa skills and helps to look for the skills. I’m not using a lot the channel.
I will update in any case the skills.
Thanks
Matteo

2 Likes

Here is some ideas I would like to develop with the community.
If anyone is interested, please reply to an idea by writing the idea number and text above your message. I will be developing the following topics one by one, starting with IDEA #1, by adding functionalities/layers to the skill(s):

IDEA #1: Emotion - adding a personality

IDEA #2: Engineering - making a call to an external skill and building a chain of skills

IDEA #3: Real World - how to exit the virtual world and integrate data from the real world

IDEA #4: Education - the universal teacher

IDEA #5: Eastern - entering the Unknown with a skill in Arabic (a metaphor)

IDEA #6: Cryptocurrency & IoT & Metaverse - how to pay a chatbot for a service (any service)

IDEA #7: The Novella - writing an episode of the TV series Black Mirror

IDEA #8: Reprogramming the Matrix

Let’s break everything down into these components and then rebuild everything from scratch.

This idea is related to add a layer to the chatbot personality to convey an emotion to the user. This should be done in several scenario, such as reading a novel or assisting an angry guest in an hotel or…(I have many use cases) …or just for fun.
I’m going to build a “support skill” because I won’t be able to get through the certification process and then invite some users to try it based on their Amazon account (I never tried, I’ll have to see how it works). I think we are going to do it with GPT4. I got the API access!
BASIC GOAL: this BOT/Orace of Light cousin/(GPT+Alexa) thing shall be able to detect a sentiment in the User’s prompt and reply by detecting that sentiment and producing an emotion. To produce an emotion we are going to use Alexa’s SSML tags. Probably the result will be mediocre because of many reasons, but it is useful to build a prototype.

In the next years I think the LLM going to win will be also capable of superior sentiment analysis, being able to understand the user’s emotion and personality and being capable to manipulate the user in calming him/her down in stressful scenario or persuading him (doesn’t need to persuade her…) in buying things or other stuff. Also the company able to integrate this chatbot inside a real product (like Astro or Roomba or Hollywood Actress…) is going to be very successful, creating an emotional bind with its user/customer.


Asking chatGPT3.5 how many emotions is able to detect, it will reply:
<<As an AI language model, I don’t have emotions, but I can recognize and classify several human emotions based on text inputs, such as joy, sadness, anger, fear, surprise, and disgust. The number of emotions that can be detected varies depending on the specific algorithm or model used for emotion detection.>>
If I ask chatGPT if it knows Alexa SSML tags, it replies:
<<Yes, Alexa supports SSML (Speech Synthesis Markup Language) tags to convey emotions in speech output. Here are some examples of SSML tags for conveying emotions with Alexa:
Emphasis: The <emphasis> tag can be used to emphasize certain words or phrases, which can convey a sense of excitement or urgency. For example, <speak> I am <emphasis level="strong">so</emphasis> excited to see you!</speak> .

Prosody: The <prosody> tag can be used to adjust the pitch, rate, and volume of speech to convey emotions such as excitement, anger, or sadness. For example, <speak> I'm really <prosody pitch="high">excited</prosody> to see you!</speak> .

Break: The <break> tag can be used to insert pauses in speech to convey emotions such as hesitation or uncertainty. For example, <speak> I don't know...<break strength="medium"/> maybe we should try something else?</speak> .

Say-as: The <say-as> tag can be used to change the way words are pronounced, which can convey emotions such as sarcasm or playfulness. For example, <speak> Oh, <say-as interpret-as="interjection">great</say-as>! Now my phone is broken.</speak> .
These are just a few examples of SSML tags that can be used to convey emotions with Alexa. There are many other tags and techniques that can be used as well.>>

So here I think it is better to restrict the domain to a subset.
ChatGPT says that As an AI language model, he doesn’t have emotions.
But it can have a personality, built inside the system variable. The one that you see in the playground at Playground - OpenAI API. Personality could be managed later as an embeddings.
You can help to identify this system variable by playing with the Playground (use the settings provided). Here is an example:




You can help me to develop this personality. Basically I will load and unload a few personality for the users that are going to help me. I will build another skill to test this personalities. You load the personality and then input some test vectors (that you have later to provide me) and you check the results and grade them (if you got the emotion). Then we select a couple of best personality and we will force a positive behavior, like mapping user anger to AI disgust or else. The goal is not tricking the AI or trying to achive something unconvinient. The goal is to analyze current SSML tag and see if we can propose an improvement. We always want to provide the chatBOT with a positive behavior and attitude, being able to calm down a user. If anybody has some advice, please let me know.
I’ll write more in the next days. Bye.

1 Like

I find Amazon terrible when it comes to customers not living in their country of origin and not using the default language. Local settings are scattered everywhere, some are common across amazon.* domains, some are not.

In that case, I had my Alexa (Alexa → Settings → General → About) set up to alexa.amazon.fr from a very old setting. I changed that with:

  • Go to Amazon “Manage my devices”, from Amazon Sign-In or whichever URL you have for your local account
  • Open “Preferences” and change the country, from FR to US, even with a fake address. It will say that something that your Kindle catalog may change content.
  • Sign out of the Alexa app and Sign in again, set it up again (may be cumbersome if you have already spent lots of time configuring it with routines etc)
  • Now Alexa host name shows up as alexa.amazon.com, it finds my settings, get localized to my home language and address for the weather forecast (not the US), and it seems it didn’t break my Amazon accounts and Prime on other amazon.* domains.
  • Alexa is still as dumb and annoying, but you can now find the skill from @snuguru_maestro :slight_smile:
  • I’m now at the step where I try to create a routine like “Alexa, open A I” to open the skill faster, but she will stubbornly play music or open her own Prize bots :person_facepalming:
1 Like

Very welcome on the video, I’ve been using your skill for a few weeks and it’s really great. Definitely wanted to make others aware of it. Awesome to hear that you’ve already implemented the UI aspect, very much looking forward to it - mostly on Astro! Very interested in your ideas, I saw your recent post but haven’t read it fully yet. Keep up the awesome work!

Just found the skill, it’s great! Apologies if I missed it elsewhere in thread but is there any way to attach the skill to my openai subscription?

Thanks again. Very exciting to see this skill born. I hope you succeed in navigating Amazon (groan;))

Thank you @jbd , it is very well detailed. In the next days I will have to use it because I cannot use my skill in my primary language! ? I’ll have to switch to English. How can I appreciate the nuances of a sound in a different language?

@brad.miele Thanks for reviewing the skill. At the moment I’m not planning to provide this feature for individuals. I might customize it for a company. If you need more details, send me an email (my mail is in the first section of this thread).

That’s nothing @sono_matteo , thank you for your hard work. I had given up on my Google Nest and Echo devices as doing anything with them, besides turning on/off lights, is more time-consuming than helpful.

To continue on my update:

  • I can’t make a routine that detects the words “Open A.I” or “Chat G P T”, as it will make Alexa look into its Skills repository instead of matching with my routine rule
  • A routine to detect the phrase “Open Oracle” works from time to time. It gets it wrong “Open Rocket”
  • A routine to detect “Alexa, Be smart” works from time to time, it also looks for “Bismark”.

Eventually I’ll have to find out what keyword has the highest precision with Alexa. Once the Oracle is on, I find it way easier: Chat GPT is quite robust to a few wrong words, just like a prompt full of typos and swapped letters on chat.openai.com is actually likely to still generate a good output.

And I had a quick idea to avoid your bill going too high without limiting the reach of the skill:

  • Provided your skill receives a unique Alexa identifier
  • Then, could it be possible to:
  • Count the number of requests to your Skill per AlexaID
  • Create a webpage/form/etc where anyone can send you a pair of {alexaID, openaiAPIkey}
  • Set a threshold of skill request per AlexaID, absolute or per given time period, above which your skill would reply "Maximum number of requests reached. Submit your Open AI API key on or activate Oracle Premium at 1.99E/m
  • If the AlexaID is associated with a user-submitted OpenAIkey, then you make your request to OpenAI with the user OpenAI key instead of your personal one.

Hello @jbd thanks for your suggestions about the monetization. I have an update for you about the French skill. Write me here
revolutionary.voice.2020 @ gmail.com
Matteo

1 Like

Hello, is there a way for it to stop saying …would you like me to continue?also yesterday i was getting sorry im having trouble accessing your oracle of light skill a lot although its much better today any ideas?, its a really cool & awesome skill you’ve created, also will chatgpt 4 be available soon on alexa?
Thankyou :slightly_smiling_face:

Hi @mysticoz thanks for reviewing the skill. Unfortunately there is no way to remove “Shall I continue”. I’m placing it purpousely into the skill, not GPT.
Yesterday you had problems as all regular users of chatgpt because there was a problem with OpenAI server. When you have problems you can check the link
OpenAI Status
from OpenAI and subscribe to their info. When you see a red bar, that means congestion. The skill detects it and reports that “there are problems with the site…”. Today it was way better. I have already tried the GPT4 API. They integrate with the current skill with few lines of code. The GPT4 works perfectly! It is so beautiful. Right now I’ll stay with GPT3.5 because of price. I got an idea for a skill that could specifically leverage on the GPT4 capability. I’ll work on it in the next month.

I have finished a first release of the emotional skill. It’s a prototype, nothing exceptional. By the way, something happens. Alexa can change the pitch and rithm of its voice when GPT detect one of the six emotions. I asked Amazon if there is a way to distribute the skill to some testers before certifying it. I’m waiting for a response. The skill logo is this one:


I used DALL E. In the end this skill will be integrated in a more general skill as a personality layer and loaded upon a trigger.

1 Like

Thanks, and I don’t need a French skill now that my device is un-frenched :slight_smile:

Thanks sono, thats great ill check that out if servers get busy, look forward to chatgpt4 both amazing.

1 Like

The German link does not work and I can’t find it in the store.

My friend is in Germany and his language is German…
I live in Austria and my language is German.

In both cases it did not work

:smile: @jbd now that you have gone through all the troubles to ‘americanize’ your device, I have a feeling that the French skill might be able to get through certification. I’ll let you know of any progress. If you have the chance in the next days, write me an email. I want to check with you a sentence in French that could help the French skill.

Actually your comment @mysticoz has trigggered in me a couple of ideas:

  1. Should I add a visual indication on the skill’s GUI that the OpenAI site is experimenting heavy load? I would report QoS (Quality of Service) LOW or MEDIUM or HIGH.

Tell me in this survey here (2 days)

Hello @rolandschuetz , thanks for reviewing the skill. All my skills but the English one have been removed by the Amazon store because they allowed to search for adult or inappropriate content. It is a comment that I can accept, even if I have my filters on top of a moderated chatBot such as chatGPT. When a skill is removed, they remove it also from the store. Nothing remains. So, right now, you can only open the english skill (locale US, Canada, UK; you cannot with Australia and India). I might be able to update the German skill and add more filters, even if they have a side effect with the conversation. If you have time, please write me at revolutionary.voice.2020 @ gmail.com . I need to ask you to check a sentence in German with a real person (not chatGPT) and you could help me. Then I can try to re-submit the skill in German. Regards, Matteo

I have refreshed the Italian skill. It entered certification almost two weeks ago. Results should be available by 3/24/2023, 5:36 PM. (I think here time is in C.E.T.). Something will come up!