I’ve just been thinking about what GPT-3 is capable of in broader terms.
One clear way it is an asset is creativity. It already feels indispensable and second nature, in any context whatsoever where I would usually wrack my brains to think of ideas (say, writing ideas), GPT-3 comes up with innumerable original, and quite zany, wonky ideas.
Another is obviously just rote information processing, like a variety of NLP tasks. Here, it’s very useful, although this is one of the most obvious applications.
One I am getting very interested in now is “knowledge generation”. Based on the cleverness of the prompt, it’s interesting to try to use GPT-3 to generate genuinely useful collections of facts, based on how it has internally synthesized all the information it has consumed.
I’ve been trying to think of ways that GPT-3 could do things like write an entire encyclopedia or dictionary or generate systematic and complex tables on any variety of topics.
I think it’s interesting to think of GPT-3 not as a knowledgeable tool that applies its knowledge to “smart” tasks but as a repository of knowledge itself to be mined.
Just one teeny example is trying to get it to list all the words it knows.
This is a list of words I got it to produce:
“The 100 most common words in the English language are:
the of and to a in that is was for on it with as his but at from this they with her i an be this by have not or if so up out only what time when who will no more all day long some back again many over after first here these two now then where why how which do new make one come other into get well off down there them our about us like may could must would did see go away back home through under than before men women children thing place way while because ever since always even though never once still yet just right left above below near far between inside outside high low open close next behind last front middle east west north south top bottom end beginning end middle side same different big small young old black white red yellow green orange purple brown gray pink blue red green yellow orange purple brown gray pink blue violet rose gold silver copper iron lead tin zinc aluminum bronze silver gold copper iron lead tin zinc aluminum bronze platinum palladium iridium rhodium osmium tungsten rhenium cobalt nickel chromium molybdenum technetium ruthenium cadmIUM indIUM thallIUM yttrium scandIUM titanium vanadium chromide niobate tantalate manganese silicate boron phosphate silicon carbide germania antimony tellurite selenite sulfur dioxide phosphorous sulfide arsenic oxide chlorine peroxide iodine fluoride carbon monoxide cyanogen chloride hydrogen sulfide hydroxyl radical nitric acid nitrous acid nitrogen dioxide oxygen dioxygen ozone water vapor argon krypton xenon radon helium neon proton neutron electron positron neutrino mu mesons kaons pions pi mesons mu mesons kaons pions pi mesons quarks gluinos leptoquarks gravitinos photinos higgs bosun squark sneutrino sbottom stau sleptron sneutrino sbottom stau sleptron neutralino axion dark matter WIMP zeno photon neutrinoless double beta decay superpartner supersymmetric particle graviton gluaton photograviton phlogiston quantum gravity scalar field vector field tensor field spinor fields pseudoscalar fields pseudovector fields axial vector bosonic string fermionic string Kaluza Klein monopole magnetic monopole gravitational waves cosmic strings cosmic inflation brane cosmology extra dimensions parallel universes wormholes time travel faster than light speed of light event horizon CMBR background radiation Hubble constant deceleration parameter acceleration parameter expansion rate density mass energy temperature entropy pressure force electric charge magnetism nuclear forces strong interaction weak interaction electromagnetic force gravitational attraction Newton’s law of universal gravitation Coulomb’s law Ohm’s Law Ampere’s Law Gauss’ law Faraday’s laws Lenz’s law Kirchhoff’s circuit laws Maxwell equations Lorentz transformations Einstein equation relativistic energy momentum relation special relativity general relativity equivalence principle equivalence principle postulate Mach number Euler characteristic Bernoulli number Biot–Savart formula Stokes’ theorem Navier–Stokes equations Rayleigh friction viscosity Prandtl number Reynolds number Knudsen layer Mie scattering Thomson scattering Compton effect photoelectric effect Compton scattering.”
I’m trying to think of ways I could get it to be exhaustive. Maybe I could subdivide all words into categories small enough to fit into a prompt; then query GPT-3 on all categories.
I’ll be exploring this more, anyway.