Can anyone let me know if we can actually use the LinkReader Plugin of OpenAI in the OpenAI API ? I want to be able to send a request to OpenAI API saying that “Crawl through google for this article and give me the response in a html format”. I was able to do that using the OpenAI Web application but no luck with the API.
I get that both of them are different and the plugin functionality is not actually integrated with the API yet but are there are any workarounds for this? I checked out Langchain but I dont exactly see how i can do it.
So, If I am essentially am able to use LangChain Google Search. Will Chatgpt be able to provide me the response in html format? I am actually confused a little over this. I am going to use Langchain google search instead of OpenAI API , so how will chatgpt give me the response?
I tried it out already. I want this to be purely automated, so I want a crawler which will go through Google or Bing and search for the articles I needed and only scrape the text i want and convert it to HTML. Writing that code will surely be large and I will also need to check out ratelimits and captcha defenses and more.
Its not only about HTML actually. I want Chatgpt also read some articles and generate YAML code depending upon the articles found from crawling. If its only simple HTML, I would have surely used some scraping with awk , sed and wget.
So, I am a security guy and I am trying to play around with Chatgpt to essentially find any new exploits/CVE’s released as the first step. Once it finds the newly released CVE’s , I want it to use google, github and more sources to find the exploit code and build a YAML Template from the exploit code which can then be sent over to some scanners in my testing. But the issue here is, OpenAI API does not have Plugins functionality and I am unable to get the YAML back I need.
I am successful in making chatgpt retrieve the CVE IDs I need. But using the API to perform the crawling is not possible and hence the failure. I played around with scrapers to get the exploit codes but I feel its not an efficient process in anyway.
Can you share link to an example chat?
It all seems doable with the API, but you’ll need to cobble together several pieces. Generally URL reading plugins just fetch the page content and include it in prompt.
I dont exactly get how Chatgpt created the YAML too because I get a different YAML every single time i generate the response and it seems like it uses different link for it then. I am doubtful if LinkReader is the correct one for this task but it seems to work in some cases but not all.