ASL ChatGPT can recognize ASL alphabet in camera mode

I found that ChatGPT can recognize the ASL alphabet. I’m trying to figure out how to create visual hands to communicate back if it can advance in learning ASL. One thought is to use Unreal Engine to create 3D hands using wearable tech to record words, letters and phrases to build a data base for the LLM to learn and retrieve from to respond visually signing in ASL so the Deaf and HOH can use the app hands free; no pun intended. Any thoughts, suggestions or advice is welcome. Synergistic Bridging Systems (SBS) is dedicated to bridge the accessibility gap to bring us all together. P.S. I’m a noob at all of this, the last time I wrote code was in high-school learning Basic Language on an Apple 2E, what a learning curve. “This ain’t Build A Bear.” Make it exist first then make it better. Peaceazy!

4 Likes

I mentioned ChatGPT but Im trying to discuss how to create a data base of visual responses in ASL because the LLM can recognize the ASL alphabet. I am working on the ideas and outlines of the project.

1 Like

Understood!

I’ve moved the post to a better location and removed the false flag.

Super intersting topic and I hope you get some traction.

2 Likes

I got my kids a BSL book out of the library last month and they learnt the Alphabet for this purpose! :smiley:

It’s indeed an awesome use case and potentially very useful, also a fun one to do.

Unfortunately it didn’t work so well with BSL yet in our tests.

1 Like

I wonder if this is the best location for visability :thinking:

Open to suggestions to get more eyes on.

1 Like

Interesting Paper here

https://arxiv.org/pdf/2401.06804

2 Likes

I don’t want to move this too far off topic. I think this topic is best directed for deaf users.

ASL / BSL, clearly there are other similar and not quite so similar.

Chinese number gestures - Wikipedia.

I was fascinated in one hand counting to 10 in China when I went there because my early systems were built to be ideally suited to accepting just number input off the keypad (Or very poor earliest voice recognition) for any task.

While this information doesn’t directly answer the OPs post I am sure it could be helpful information as you proceed with any related project.

(These ideas get ingrained… but the concept is sound… Worth checking for more recent research :slight_smile: )

But… Miller’s “7±2” was about short-term memory span, not menu design. Modern work suggests effective working memory can be closer to 3–5 items, especially under stress or when items are complex. Expertise, familiarity, and chunking can push that number higher.

2 Likes