Would love some feedback/suggestions on techniques and tooling others may use that prevent and catch spiders/crawlers/spammers from interact with a production live bot created with the AssistantAPI.
So, my organization has a public facing site which is seeing over 50% of interactions with the assistant to be nonsensical spam. I believe we have protections in place, CloudArmor and we are looking to implement reCATCHA but I’m curious if there is possibly anything I can use on an API level in my AssistantAPI backend.
Ultimately, IMO, we’ll also have to deal with bad actors/spammers etc but if anyone has any ideas they’d like to throw my way that they may have used for these circumstances, I’d appreciate it!
Note: ICYWW - we are using FLASK Websockets for communication between frontend and backend.
With the assistants API still in beta, i’m not sure if it really should be used for forward facing in-production use, IMHO. Not-to-mention that your company might be liable for hallucinations that your assistant creates.
Anyway, if you notice that the first spam-bot message is nonsense you could (for new chats) pass the first message to a chat completion that basically vets the content and returns a value of some kind saying it is nonsense, it is not, or it is maybe - and proceed based on that.
Alternatively you might try using Google recapcha V3 to deal with bots before the chat even begins.
You can directly integrate your assistant API endpoint with chat data and get protected with the rate limiter. It’s free if your monthly messages is under 400.
Ultimately a layered approach works well. First layer of defenses are always the tried and tested…cloud armor, reCaptcha etc…second layer is more canned responses through a vector db (depends of course on your use cases) as opposed to even touching chatgpt…recently we have making use of timed-threads (they expire after a while, then reCaptcha back)…and then monitoring conversations to be purposeful…if not, deliver canned responses based on ip addresses
Building on what icdev2dev said, I would recommend saving all request/responses in a simple KV cache and then using that as a first defense in front of the vector DB. I’m assuming bots will likely try the same combinations of requests and you could serve up the responses without ever sending it through to the actual API.
That said, bot fighting is likely going to be an ongoing war that nothing will ever completely work for … which writing that makes me think it’s a perfect market for some cheap/fast AI filtering. In my opinion, the only defense that will truly work is to require the user to be logged in rather than dangling an advanced chatbot out there for arbitrary anonymous traffic to delight in.
