NPC.TO: An Experimental Project Combining XR, GPT, Blockchain, AI, and Web3

[kaww]

“I have been thinking about whether I can do something interesting for others using GPT. This experiment allows everyone to participate in every aspect, including designing prompts, creating image and video models, sharing prompts, and receiving support and encouragement throughout the process.”

This is the way.

Since the emergence of GPT, there has been a desire to explore its potential applications, but finding the right direction has been a challenge.

However, the team at NPC.TO has gradually realized that combining GPT with XR (extended reality) could provide a unique experience for users.

With their expertise in XR, they have been able to create NPCs (non-playable characters) that can exist in the real world, bringing the gaming experience to life.

Artificial intelligence (AI) is used to generate all the images and models necessary for this project.

The team has integrated a PHP program that interfaces with GPT API to facilitate chat with NPCs. By using prompt words, users can interact with the NPCs just as they would in a game. This project is fully open-source, allowing users to access it both locally and online.

One major issue has been the prompt words used for NPC conversations. While ChatGPT can help generate these prompts, everyone has their own ideas. To allow users to permanently save and share their ideas, the team has turned to blockchain technology. Kevacoin, a key-value data type blockchain, has been used to store prompt words. Each data space can be accessed using a block number, and prompts can be identified by their file names.

With these elements in place, NPC.TO has been able to bring all these ideas together, allowing users to upload digital files such as images and videos with a numeric file name. The platform then automatically retrieves the prompts from the blockchain and sends them to the GPT API for answers. This results in an NPC that can chat with users just like a game NPC.

For those with AR glasses, the experience can be taken to the next level, as the NPC appears in mid-air. Overall, NPC.TO is an exciting experimental project that combines XR, GPT, blockchain, AI, and Web3, showcasing the potential of these technologies when combined.

GPT4

NPC.TO

GPT3.5

NPC.TO/35

GITHUB

https://github.com/kevacointoolkit/npcto

DEMO PROMPT FILE NUMBER

62108412 (e.g. 62108412.png, 62108412.jpg, 62108412.mp4)

[kaww]

Scenario 1: AR+AI Usage

Things you need:

1. AR glasses (such as nreal air)
2. Smartphone or PC
3. Nebula or GingerXR software
4. Any picture or video files

This is the simplest aerial projection scenario that can make any picture or video hover in the air. It can be achieved even without using npc.to, but npc.to makes it easier to open up space and interact with GPT at any time.

How to use:

First, download Nebula (for MacOS) or GingerXR (for Windows) software on your computer. Here, we will take GingerXR as an example. This software can simulate three display screens for AR glasses.

Then, open any image and view it through the AR glasses. The image will appear to be floating in the air. Due to the transparent property of black background in the AR glasses, using black background images and videos makes it look like it is in the real world.

When using npc.to, the viewing direction can be changed, and the image will remain fixed in the specified position, just like in VR.

The role of AI here is to create something in one’s virtual space. If there are no existing files, just tell the AI what is needed, and it can create images and videos.

I think that in the future, if we are able to converse with AI through AR glasses, we should be able to create, manipulate, interact with and engage in real-time with anything in the physical world, effectively making AI the ultimate creator of our reality.

[kaww]

Update 0407: It is now possible to give an NPC a name using the key “NAME”, and using both “NPC” and “PROMPT” creates a point-to-point chat effect with the NPC.