When will it be possible to create physics and or biology images that is for illustrating text books and exams? Anyone knows? Or anyone knows how to do that already?
1 Like
The AI creates images of fantastic fantasy.
It is not suitable for education or documentation by content.
You might be able to make simple well-described human-supervised diagrams such as levers in physics.
Outside of the DALL-E image maker, I ask the AI about what “advanced data analysis”, a python code interpreter, can do for diagrams itself…
The following vector graphics libraries are available in the Python environment:
matplotlibplotlygraphvizsvgwriteGiven these options,
matplotlibis the most appropriate for creating a free body diagram of an airfoil with force vectors. It’s widely used for scientific and technical visualizations. I’ll now create a demonstration of such a diagram usingmatplotlib.
And having AI produce such a diagram…its abilities are underwhelming.
Metplotlib is definitely my go-to when it comes to math graphs. It’s da bomb.
But since I started doing physics and biology related topics I started to cry… lol
Anyway, any updates on this would be super helpful. I will keep searching on hugging face and other platforms for any updates too. Much appreciated you guys!
1 Like
That is an excellent question and one I have an interest.
Have you read the paper on how DALL-E 3 creates images?
“Improving Image Generation with Better Captions” (PDF) by James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, Wesam Manassra,Prafulla Dhariwal, Casey Chu, Yunxin Jiao, Aditya Ramesh
The paper elucidates the constraints and opportunities in generating these images.
Short answer: No.
My thoughts on this are that with an image generation prompt one should do something similar to a multishot text prompt but include images, I have not looked for such but it is on my list. I also think that the technical images are distinct enough from the types of images normally created with current image generators that that the training set needs to be entirely different or very selective.
Please let me know if you have any progress on technical, physics or biology images/illustrations for exams and text book. I’ll also share on this forum if I found anything, much appreciated!
First off creating images is not high on my list but it is on my list so I am currently not putting a lot of effort into it.
However yesterday I was using Kapp.ai on the Discord forum to see what it could note and that lead to this
https://help.openai.com/en/articles/6516417-dall-e-editor-guide#h_04ec3d706f
which might be of value. However I don’t plan to try it at present but noting for others as I would here to hear if that is of value for the goal noted in the first post.
I have code that fairly reliably can take a prompt → GPT → high school physics diagrams for particular chapters/topics (mostly kinematics, because its just boxes). I will say that my use case is largely to make writing MCQ tests easier for teachers, not publishing textbooks. I am trying to expand this to biology (theoretically it should work for simple diagrams), but have other conflicting obligations. I would love advice and/or help if anyone is interested. Bonus points if you have friends at BioRender.
Use matplotlib to draw and label a heart