Yeah you don’t need React or any sort of web framework for it to work. SMS or Email is great for back and forth communication.
Like you mentioned, you could do it through Zapier.
Me, I used the Twilio API, AWS (Lambda + DynamoDB + API Gateway), and of course the OpenAI API.
From my phone, I can even change the tone or personality of the bot by issuing the commands over SMS, and this would set a different prompt for all future responses until I changed it to something else.
It’s a fun project and very useful! And still relevant, even with ChatGPT on smartphones, because you can utilize embeddings (semantic search, RAG, etc) to really customize your personal assistant.
2 Likes
So, I’ve got the SMS working now. Only needed the Vonage API and a number.
I don’t need to change the personality, but I was curious if there was a way to change the libraries. On the website query screen, I have several libraries a user can select to narrow his search.
These are metadata options used to filter the cosine similarity vector search.
Now, with the SMS interface, I basically only have the text message itself to work with. Can you think of a strategy for being able to select specific libraries for the query? Maybe functions?
You could use cosine similarity on the input and against your categories (and further details of your categories) to dynamically put you in a category each time, or have it integrate and determine the category after a few initial turns.
You could try a fine-tune to categorize what the topic is, but that could get weird with all the categories you have.
Or you can do progressive keyword correlations … using something like BM25. Working on my own keyword correlation algorithm now, so it’s on my brain.
1 Like
alden
10
Yeah - good idea.
PS: Our Zapier app just got approved – so anyone with zero-technical skills can build RAG agents and connect them to SMS/Email/whatever.
alden
11
Use a separate metadata filter for each library. (e.g. in Pinecone, you can add { project_id = 1 } and then filter by that during the query.
By the way, I hope this is for your personal use, right? 
Good idea …
Thinking this through, I came up with another alternative – good old regex. I instruct users to say a specific phrase (you know, like with Google Assistance “AI” – lol) like “select libraries:” or “include libraries:” then use regex to capture that input.
However, the next problem is maintaining that library “state” through the next query in the same session. Fortunately, I already have a model for this as I’ve been using good ol’ SQL to maintain chat history in API calls.
When I think about all the legacy technology that is necessary to efficiently utilize this new technology, I’m struck by all the YouTubers out there saying “AI is going to replace developers.” Yeah, right.
1 Like
Nice. I imagine that could get somewhat pricy though if you were a super heavy user.
It might be a bit more work up front, but did you consider using the WhatsApp API or even building something like a Discord or Slack bot?
I ask because,
- They are much cheaper, and
- They all offer varying levels of markdown rendering support
You could also try interfacing with signal.
As a long-term dream project you could think about linkinh whisper, gpt-4, and a voice from elevenlabs to an asterisk box so you can call and talk to gpt-4 from anywhere. But, the real power though would come from being able to have it make calls on your behalf.
Anyway, great work!
1 Like
I’ve started working with Twilio in addition to Vonage. Twilio has a Whatsapp API, so I may look into that. I personally don’t use Whatsapp, but I guess I might have to take the plunge as everyone else in the world apparently does use it! At least everyone under 45!
A lot cheaper
https://dashboard.nexmo.com/api/v1/pricing
You probably could build your tool with their free credits at Vontage
Access to GPT via sms is a great idea
Say give the user like 100 free then hit them with a link to get more access or less 
Using vontage is fun and easy I did a mock up prototype using it and used up all my free credit 
I might need to give them a credit card this would be a great addition to my site
Who knows it could become a thing 
I can’t guess what this generation thinks
But it would be handy if you wanted a quick answer in the middle of a conversation on text
Email? Yeah it could be helpful there too include GPT as a cc and join the email thread hell yeah . Going to be a lot more challenging than sms however. But it could be hacked pretty sure, could dust off the old text based mail server.
1 Like
Apt install sendmail and a new sendinblue account about to head down port 25 wish me luck
I already had a Zapier account, so I used their Email Parser + Webhook combo: Connect your Email-Parser to Webhook integration in 2 minutes | Zapier
That was the easy part. The hard part was figuring out how to authenticate the emails since anyone can spoof anyone else’s email address. The solution I came up with was to use email verification on the first API call, then the user can send emails for X minutes until the certification times out.
Only way I see to do it right now as leaving it open is asking for trouble.
Good luck!
1 Like
I don’t see where the problem with making calls would be.
Let a software make a call. And then send an initial request to gpt-4 like “hey, there is a guy on the phone you just called him to promote…”
I was more like thinking what happens if “a” bot calls someone records a little bit of his voice, does some internetresearch and finds relatives of that guy and then calls them with his voice, and also records their voice to call him… taking everyone out of their reality and causing any kind of bad stuff… like traffic jams in the lowest (let’s all meet right now there and there!!!). to financial chaos or even I mean what would happen If a bot breaks up with all partners in the world?
Could be taken care of long time ago. e.g. by two factor auth before you can use your phone.
1 Like
It is a real world scenario there are guys who do that with Elon Musk audio then pretend to be him, for nefarious intent
Email can be spoofed or it used to be I will see if sendmail has made any changes soon
Waiting on GoDaddy to update my dns this five minutes with SendGrid cnames
The “two factor auth” was just a cynical joke about security.
Now imagine a “small” drone army with grenades on that flies into your open window in the middle of the night and “asks” you to hand over your phone and pin. Then drops the grenade and flies away with it.
Would be possible to collect thousands of phones in no time (imagine some secret service would want to take over a country and has the capital and man power). And the bot takes over the identity of that person… this all happens in a couple of hours.
And then some cloud administrators, politicians and generals call in sick…
Joke or not, this remains the most reliable way to authenticate the sender of an email.
Exactly the most reliable.
In the same way as when you jump into a lake pressing a hand on your eyes is the most reliable way to keep your body dry.
I mean there must be a reason why worldcoin scans your eyes…(other than making it possible to send migrants back to where they came from, when they “lost” their passport after illegal immigration)
Well Digital Ocean seems to have locked down any ports other than 22, 443, 80
Can’t even get there from sendgrid anymore
Support is stalling
Seems like spam has destroyed inbound mail for the outdoor dogs
What is your program logic?
This is what is working for me:
user sends email —> email is forwarded to Zapier —> Zapier Email Parser —> Zapier Webhook —> My AI Webhook —> Question submitted to LLM (API) —> LLM returns response —> My AI Webhook sends response back —> sender email
Works perfectly.
Or ask chatgpt to write you a short php script and run it with a cron job. Low code is dead haha…
https://chat.openai.com/share/fd5e3530-66da-492d-ae09-1f2f46956303
I mean… 10 minutes more should be invested…
1 Like
I have my 
reading all my mail now i forwarded to root very nice
Now to reply
So it is possible now to CC: GPT and get a email reply
Sendgrid has a webhook call I listen for and get all the mail for talkeetna.info in one file I decode and feed it into the and ask him for a summary and a email
2 Likes
