How about webapplication testing with OpenAI?

Hey Forum,

I don’t even know if I’m posting this idea to the right place, if not, please redirect me.
I was hoping I could use OpenAI to test a web application developed by a team at my company. Currently I can do the workflow alone, manually. Tried to automate this 6-7 years ago but with no luck. What I was thinking about is to tell GPT the same as I was told in the past.

  • Visit the website
  • Log in
  • Try to do this and that (around 3 dozen things)
  • Give a report about how it went and where did you encounter any issues

I know there are some 3rd party services who provide this but honestly, I would trust my own stuff made with OpenAI. Any ideas how could I start? I mean, I know I could start at web-to-text, but I don’t know how to tell him “Push this button”, or “Fill this textbox with information”. Can you help?

1 Like

Not OpenAI related, but check out the Cypress automated testing tool. You have to write your own tests. I don’t think ChatGPT can do this for you, but it can probably help a lot with the Cypress code.

1 Like

The integration of Large Language Models (LLMs) in automating or augmenting tasks such as penetration testing for web applications and adversary emulation presents a fascinating opportunity. These models could serve effectively in roles that range from oversight and coordination to the direct generation of code. Although GPT-4 possesses the capabilities required to excel in both areas, current restrictions around the API use make full utilization infeasible. The API does provide a measure of flexibility, such as the ability to outline potential attack strategies, interpret and analyze threat vectors, and deliberate on specific vulnerability scenarios. However, users are strictly prohibited from progressing towards any actual execution phase, sometimes even from discussing it in hypothetical terms.

I’m actively developing a project in this domain and am considering the use of locally hosted open-source models. For instance, employing a 70-billion-parameter chat model could act as the central coordinator, complemented by LORA or similar, smaller models that have been fine-tuned for specific code generation tasks, and possibly utilizing Mixture of Experts (MoE) techniques. This setup would be in conjunction with some of the more promising agent-based frameworks currently available, such as MetaGPT or CrewAI.

Yes, and to add to this suggestion, of course you can also use the open sourced browser automation tools, Selenium, Playwright and Puppeteer.

For first two I can confirm that ChatGPT can write sufficiently good code.

1 Like