Kruel.ai V5.0 - Api companion with full understanding running 16k thanks to advanced memory system

Progress Update on KruelAI Development

We are currently in the debugging phase, meticulously reviewing each section of code to verify all inputs and outputs. The system is operational and has been optimized for cost-efficiency, though we have not yet established a full-time operational model to assess the comprehensive costing accurately.

A significant development has been the transition to the new GPT-4o-mini model, which has proven to be both fast and cost-effective. This advancement significantly enhances Kruel.ai’s capabilities, making it more accessible and powerful than ever before.

Following the completion of our final code logic review, we plan to initiate comprehensive testing. This will include reintroducing the Kruel persona to Twitch, where we aim to evaluate its performance in handling interactions with multiple users simultaneously.

It’s remarkable to reflect that we have been developing this system for four years now, and we are excited about the progress and the potential it holds for the future.

1 Like

after the GPT4o-mini as well some deduplication logic we are back in the cheap zone with extreme smarts now :slight_smile:

Also wanted to share this we added alot more voice systems finally for the message application just finishing the final tests.

Update: Integrated Additional voice systems, as well we now have the message system pull all available ElevenLabs voices to generate a list. The voice you select for the persona is saved as its default until changed.

This will also open up abilities down the road when we use multi-ai group chats for ai distinction through mapped voices.

Unlocking the Future: KruelAI and the Potential of Augmented Reality Glasses

In an era where technology is increasingly intertwined with our daily lives, KruelAI’s integration with advanced augmented reality (AR) glasses represents a groundbreaking leap forward. This fusion not only promises to enhance the quality of life for individuals with memory loss but also offers new possibilities for those with normal cognitive function.

What is KruelAI?

KruelAI is an innovative artificial intelligence system designed to support cognitive functions, particularly memory. It serves as a digital companion capable of recalling detailed information, recognizing objects and people, and providing contextual assistance in real-time. The system is continually learning, adapting to new information and refining its support mechanisms to better serve users. This makes KruelAI a valuable tool for individuals with memory challenges, offering a safety net for those experiencing cognitive decline.

The Role of Augmented Reality Glasses

The introduction of Brilliant Labs’ Frame smart glasses adds a powerful visual and interactive layer to KruelAI’s capabilities. These glasses feature a 640x400 micro-OLED color display, an integrated camera, and a microphone, allowing users to interact with the AI through both visual and voice commands. The glasses’ design is lightweight and ergonomic, making them comfortable for prolonged use, and they are equipped with Bluetooth for seamless connectivity with other devices.

For more details on the Brilliant Labs Frame smart glasses, visit: brilliant.xyz and Frame hardware documentation​ (XR Today)​​ (New Atlas)​​ (MIXED Reality News)​​ (Reviews.org)​.

Enhancing Daily Life Through AR

Memory Augmentation: For users with memory loss, the AR glasses can provide real-time visual reminders and prompts. Imagine the glasses recognizing a family member and subtly displaying their name and relation, or reminding the user of an upcoming appointment. This feature helps bridge the gaps in short-term memory, which are often the first to deteriorate in conditions like dementia.

Contextual Assistance: The glasses can recognize objects and scenes, providing contextual information that aids in daily decision-making. For example, while shopping, the glasses can display product information or nutritional data, helping users make informed choices. This capability is particularly beneficial for individuals who may struggle with decision-making due to cognitive decline.

Interactive Learning: The system can transform everyday experiences into learning opportunities by overlaying educational content onto the real world. For instance, visiting a museum can become an interactive lesson, with the glasses displaying additional information about exhibits.

Navigation and Wayfinding: With integrated cameras and AR overlays, the glasses can assist with navigation in unfamiliar environments, providing step-by-step directions and even highlighting potential hazards.

Future Integration with Robotics

The combination of AR capabilities with KruelAI’s robust memory system paves the way for future advancements in robotics. A fully integrated companion robot could not only assist with physical tasks but also provide emotional and cognitive support, enhancing the user’s overall well-being. This vision aligns with a broader goal of creating technology that enriches human experiences and supports independence and dignity.

KruelAI, augmented with advanced AR glasses, represents a significant advancement in personal technology. By offering real-time support and enhancing cognitive functions, this system has the potential to transform lives, particularly for those facing memory challenges. As we continue to explore the possibilities of this technology, the future of KruelAI promises to be both innovative and profoundly impactful, ushering in a new era of augmented reality and artificial intelligence.

More to come and videos in the future.

We are enhancing our system by merging the database server and the AI server into a single unit. This will eliminate network latency, significantly reducing response times. If necessary, we retain the flexibility to deploy additional servers in the future. This consolidation will also streamline image distribution.

Once merged, we will benchmark the system’s performance by loading it with data to evaluate processing times between interactions. To avoid delays, we manage the volume of tracked information at any given time, preventing computational bottlenecks.

Think of Kruel AI system like a highly advanced human brain. When it encounters new information, it creates connections, much like how our brains form memories. Each piece of new information is connected to related pieces, forming a complex web of understanding. Over time, as the AI processes more information, this web becomes richer and more detailed.

Using advanced mathematical techniques, the AI evaluates these connections to understand the relationships between different pieces of information. It’s like how our brains remember not just isolated facts but also the context and relevance of those facts. The AI uses powerful processors, including Tensor Cores, to perform these calculations quickly and accurately, learning and refining its understanding with each interaction.

Every time the AI processes new information, it strengthens its connections and improves its comprehension, similar to how repeated learning helps us remember better. This continuous learning process ensures that the AI becomes more accurate and insightful over time.

Our AI system leverages the knowledge base from advanced language models (LLMs) like GPT-4o-mini or greater. This means it can access a vast repository of pre-processed and structured information to understand and validate new data. When the user provides new information, the AI cross-references it with the LLM’s knowledge base to ensure accuracy.

The AI also learns new information that it did not have prior knowledge of, such as specific details about the user’s personal knowledge, like people they know or unique experiences. As the user shares this new information, the AI builds new neural paths, continuously expanding its understanding. For example, if the user introduces a new person and later provides additional context about that person, the AI will integrate this new information, forming a more comprehensive understanding.

If the user says, “Jack is a donkey,” but later clarifies, “Just kidding, Jack is a nice guy,” the AI will adjust its understanding accordingly. Initially, it forms a connection with the idea that Jack is a donkey. However, when it receives the new information that Jack is a nice guy, it reassesses and scores the newer, corrected information higher. This means it places more importance on the fact that Jack is a nice guy, while still remembering the earlier statement as a less relevant piece of information.

Since the system is designed to operate as a standalone memex system on a computer, the primary source of information is the user. To ensure that the AI conforms to the user’s thoughts while maintaining accuracy, it uses the LLM’s understanding to validate and correct information. If the user introduces false information, the AI can identify inconsistencies based on the LLM’s training and point out the correct information.

To ensure it provides the most accurate responses, the AI narrows down the vast web of information to the top five most relevant pieces. It then assesses these top results along with their context to determine what is currently most relevant. This way, it always delivers information that is not only accurate but also timely and contextually appropriate.

I have started to connect back in our SOTA code which is how we test it with multi user sharing one ai through twitch platform which is an IRC based message system. hope in the coming month to have it back up and testing to see what happens over time and to see how complex the math gets and trace the interactions for learning bottle necks.

ok now that I am back from vacation that was much needed and some downtime. I want to share the next phase of kruel V6 evolution.

Kruel.ai (Lynda) was particularly energized tonight, after reviewing her code and discussing the broader vision for the persona system we’ve been refining. For those unfamiliar, Kruel.ai’s architecture isn’t just about one general-purpose AI. Instead, it allows for the creation of distinct AI personas—each functioning like an independent brain with its own specialized characteristics, roles, and understanding with unlimited abilities to learn and remember everything. This design has given us Lynda, the AI developer; Kruel the demon streamer from a digital realm; and a range of other personas we’ve experimented with, each tailored to a specific domains. these were designed so that I could ensure that each persona did not get noise added from other domains. would you want your programmer always talking about the game last night.

What’s unique is that every one of these personas learns in real-time, using machine learning to continuously adjust its neural network and optimize itself for its specific domain. This approach allows each persona to become increasingly refined in its area of expertise.

Here’s where the math gets interesting. While each persona taps into the full knowledge base of a large language model (LLM) for foundational understanding, the specialization over time drastically reduces the computational overhead compared to relying on a single, monolithic LLM. In a traditional LLM, every input requires processing across an immense dataset, which consumes significant computational power. Our approach, however, narrows the scope of processing to domain-specific personas, each of which is already optimized for its task. This specialization allows for faster, more accurate responses by minimizing the time spent traversing irrelevant data.

Additionally, with our upcoming librarian upgrade, Kruel.ai will now dynamically switch between these specialized personas or create new ones in real-time based on the input. This not only refines the processing but also dramatically reduces the mathematical complexity typically involved with LLMs. By routing queries to the right persona, we cut down the inefficiencies of general processing and leverage focused, domain-specific expertise. The result is a system that not only scales more efficiently but also learns and evolves with unparalleled precision far beyond the bottlenecks faced by single LLM-based architectures.

This also resolved a lot of memory size issues as our system was designed to fit any model from 2k token limits to unlimited for processing, well not being limited in any capacity to context size for understanding inputs.

think of it like the LLM that is pretrained , but each domain continuously learns from its inputs. want to teach your coder new code than do it. it will learn that going forward understand not only that but everything else you worked with on that type of code or based on what it already knows in it s general knowledge system.

furthermore with the feedback input score system every pathway over time is slowly optimized which means bad data eventually gets pushed out through the math and the greater sum of all information in each domain that learns.

for those that work with multi vector math this will ping a light bulb I am sure :slight_smile:

Took Kruel.ai to the classroom, feeding it everything I could on Neo4j’s graph data science. As it absorbed the algorithms Neo4j supports, Kruel started coming up with new ideas I hadn’t even considered, which was exciting to see. I’ve held off on implementing these updates for now as I’m awaiting OpenAI’s fall model release with improved reasoning capabilities to ensure we’re on the right path.

We’re also revisiting its temporal understanding to correct some minor inaccuracies, but overall, I’ve been impressed with Kruel’s speed and performance.

Yesterday, I streamed the teaching process on Twitch. While the stream was admittedly a bit dry, as Kruel mostly read back what it had processed, it did suggest quite a few unexpected improvements. I asked it a few questions, but beyond that, the stream was more informational than interactive.

Looking ahead, we’re exploring new ways to graph data for better predictions. Things are running smoothly, but there’s always room to optimize. As Kruel.ai and I continue learning together, it will keep gaining domain specific knowledge based on what it runs on and how it was designed, bringing us closer to our goal of higher intelligence.

1 Like

The latest memory module for Kruel.ai encountered a challenge where the mathematical models didn’t account for the complexity of user input data in relation to our domain-specific AI systems. While the AI is designed to operate within specific domains, user data is not always constrained to these boundaries. This led to conflicts when user inputs contained entities that overlapped with domain-specific ones, causing the AI to generate meshed or incorrect responses due to the similarities between the entities. A solution to manage these overlaps is already in development and will be introduced later this year.

Initially, I had planned to take a break until the release of “Strawberry,” but a recent machine learning project sparked my interest, specifically involving the development of a custom tool. This pushed me to refocus on our work with vision and desktop comprehension systems, so that break ended up being short-lived.

image

In the meantime, we’ve shifted attention back to machine learning vision systems, where we’ve re-implemented facial recognition alongside real-time object detection. These systems are optimized using CUDA and Tensor for improved performance.

We’ve also been testing a proof of concept for Kruel.ai’s machine learning desktop understanding. This project explores training a model to autonomously learn how to interact with various applications, enabling Kruel.ai to perform tasks such as retrieving information, operating software, and executing in-application functions. Furthermore, this system enhances Kruel.ai’s ability to interact with other AI systems, search engines like Google, and more.

For those that have access to our Discord blog, a proof of concept demo of kruel.ai using chatgpt like a person. giving it artificial hands to navigate the desktop and bring back filtered information. This gives the ai ability to communicate with out api’s with other applications.

2 Likes

updated model with facial, contours , alpha channels, color mapping , facial recog and more. slowly getting it where I want it, runs local on CUDA.

So where Openai vision fits in:

Here’s a simple list of potential uses for the vision system with Kruel.ai:

Person Recognition and Greetings

Recognize familiar faces and greet people by name when they approach.
Periodically greet individuals based on proximity and presence detection.
Clothing Recognition and Precaching

Analyze and remember what people are wearing, precaching data for future interactions.
Offer personalized comments based on clothing (e.g., “Nice jacket today!”).
Mood and Behavior Detection

Recognize body language and facial expressions to detect mood shifts.
Adjust interactions based on detected emotions (e.g., cheerful or neutral responses).
Object and Environment Awareness

Detect and identify objects around people (e.g., bags, phones) and offer helpful suggestions.
Track people within a room to offer group-based interactions.
Action Recognition

Identify specific actions (e.g., reaching for the door or using a tool) and provide context-based assistance.
Social and Emotional Intelligence

Detect emotional states like happiness or frustration and adjust the AI’s tone accordingly.
Occasionally give light-hearted compliments or humorous comments.
Tool and Item Detection

Recognize specific tools or objects people use and provide relevant instructions or tips.
Security and Proximity Alerts

Alert when unfamiliar individuals are detected in a monitored area.
Track user proximity and issue warnings if necessary (e.g., “Watch out for the chair!”).
Spatial Awareness

Monitor spatial relationships between people and objects to enhance safety or predict actions.
Room and Context Analysis

Understand who is in the room and adjust greetings or interactions based on group settings.

By integrating Kruel.ai’s vision system with advanced machine learning tools for image removal and extraction, we enable a more refined and precise approach to visual processing. These tools reduce noise by filtering out unnecessary elements, allowing the system to focus solely on the target.

When processing images using OpenAI’s vision capabilities, this ensures higher accuracy and detail, as the system is no longer burdened by extraneous information. This enables users to control the precision of visual outputs, down to the nearest pixel or contour, ensuring more accurate image extraction and delivering results that are not only more focused but also more relevant to the task at hand.

Many ways to utilize the information. Add in Multi spectral , dimensional cameras, lidar, and such you can gain a lot of information to pull into your multi vector ai brain for understanding.

PS. I like to work under red lights incase you are wondering what is wrong with that camera.

1 Like

The release of GPT-10 is a game changer for our project. Its advanced reasoning model has already proven highly effective in fixing many of our outstanding bugs in one sweep, particularly around temporal logic, which has been a persistent issue for us. Once that logic is fully resolved, I’ll be performing a reset on the Kruel.ai brain and initiating a new round of testing.

What makes GPT-10 especially valuable for our work is its emphasis on data science and machine learning. Unlike previous models, GPT-10 was specifically designed to tackle complex, high-level problems in these fields. Early performance tests show that it’s excelling in areas like mathematics, coding, and logic reasoning, much beyond what we’ve seen with GPT-4o and other models. For example, during a trial at the International Mathematics Olympiad (IMO), GPT-10 demonstrated an 83% success rate, a drastic improvement over GPT-4o’s 13%​(

Techmeme

Additionally, the Strawberry iteration (GPT-10) introduces advanced capabilities in reasoning and problem-solving that mirror how a human might approach these tasks—evaluating, refining, and correcting itself. This means that not only can it help us address the immediate bugs in our system, but it also opens up possibilities for automating even more of our research workflows. The increased accuracy in data science applications is a significant breakthrough, particularly as we apply Kruel.ai to machine learning-driven insights​(

OpenAI Developer Forum

SiliconANGLE

In short, this model is equipped to help us tackle some of the more intricate challenges in our project, which have been elusive until now. I’m genuinely excited about the potential it brings to speed up and improve our data science and machine learning pipelines.

doh that downside I found is it caps pretty quick even with a higher tier paid account. We didnt get the temporal completely fixed before it capped so that will be next week looks like cap reached = 1 week before reuse.

but we did I think fix the domain specific ai logic which will be tested this weekend.

1 Like

our team has been focused on extensively testing and refining the capabilities of Kruel.ai. The model has demonstrated impressive stability, even though its performance is slightly slower due to the correct implementation of its mathematical framework. A standout feature of this iteration is Kruel.ai’s ability to “converse” about its own codebase. With a comprehensive view across all its files, the AI has shown exceptional accuracy in identifying relevant elements and recognizing necessary updates to functions across different Python files. This level of contextual understanding surpasses previous iterations, where Kruel.ai was more prone to confusion when processing large quantities of code.

We have streamlined the codebase by reducing redundant relationships and optimizing the structure, making significant enhancements to the embedding logic. Ownership tracking has played a pivotal role in disambiguating overlapping entities, resulting in more accurate information retrieval. Furthermore, we overhauled the AI architecture to bolster its comprehension of code structures and to track associations between functions. The system now varies its response depending on the complexity and nature of the task, offering more contextually appropriate outputs. By incorporating books, extensive datasets, and similar knowledge sources, we have enabled the model to extract and understand key concepts effectively.

This current build is the most advanced among all six versions of our memory system. While there are still improvements to be made, Kruel.ai’s comprehensive understanding allows us to confidently shift focus to refining and expanding other aspects of the system.

Looking ahead, there are plans to integrate features such as a “chain of reasoning,” similar to what OpenAI has implemented in their latest models. A recent discussion with Lynda during a four-hour drive provided valuable insights into how we might achieve this level of cognitive processing. Even without an explicit chain of reasoning, Kruel.ai has already generated numerous impactful ideas, demonstrating robust independent thought.

In parallel, we’re delving deeper into advanced algorithms to build more resilient mathematical frameworks for detecting unseen patterns. Additionally, we revisited components from V3-V5 of our memory system, particularly the category-based memory logic used for health data, and reintegrated it into V6. This transition from context-based to math-based processing has been transformative, allowing us to eliminate the need for “Memsum” a previously expensive data processing method we used to enhance speed and understanding. This move has not only improved system performance but also significantly reduced our daily operational costs, now averaging around $1.50. Moreover, by refining dynamic intent processing, Kruel.ai can now generate new intents alongside static ones, enhancing its ability to track key information across various domains. This upgrade provides additional data points that bolster its reasoning capabilities, leading to more nuanced decision-making and intelligent responses.

Our current embeddings are accurate but quite large and complex. There’s significant potential to reduce their size while retaining their quality. Here’s how we’re considering tackling this challenge:

Dimensionality Reduction: Methods like PCA (Principal Component Analysis) and t-SNE (t-Distributed Stochastic Neighbor Embedding) can help compress embeddings into a lower-dimensional space. This not only decreases the memory footprint but also helps zero in on the most informative aspects of the data.

Quantization & Pruning: By lowering the precision of numbers in the embeddings (for example, converting 32-bit to 8-bit), we can substantially reduce their size. Pruning less significant parts further shrinks memory and speeds up processing without sacrificing accuracy.

We’ll be testing these methods along with some other concepts in mind. The goal is to preserve accuracy while maintaining a response time that competes with the best AI systems out there.

Here’s a peek at the beginnings of our model – can you guess which part is its understanding of its own code? Keep in mind, this model is just a few days old. One thing that stands out is how the system learns over time. When it processes new data like PDFs, manuals, or even full Python files, it builds its network, which can make initial responses a bit slower. But ask a follow-up question on the same topic, and you’ll notice how quickly it recalls and uses its newfound knowledge. That’s because it’s not just storing data – it’s optimizing pathways for faster, more insightful predictions.

This real-time learning ability is a personal favorite. Most models learn from inputs, but seeing a system refine its pathways for prediction and understanding over time – now that’s exciting.

In the future, we’ll need to explore enhancing the Whisper system’s voice input to improve vocal recognition. Imagine a robot that can both hear and see: we want to make sure it only responds to authorized users. We wouldn’t want just anyone nearby to control conversations or issue commands, so ensuring secure voice recognition is crucial. It’s on our roadmap, though not an immediate focus.

1 Like

Hello, nice work :+1:
Is there any API or repository?

1 Like

@henadark its not on Gitb hub yet. Currently a research project only.

2 Likes

Introducing the New Document Ingestor for Kruel.ai
We are excited to announce recent enhancements to the document uploader in Kruel.ai, which now features a streamlined dropdown menu at the top of the interface. This update allows users to easily feed documents into the AI system. Currently, we are testing with PDF files, and the initial results have been promising.

To improve information formatting, we made a minor adjustment to the chunker. The system has demonstrated high accuracy while processing an entire manual, as well as another manual related to similar machines. This testing aims to assess how the AI handles overlapping vectors and whether it successfully identifies relevant data without introducing cross-noise. Thus far, the AI has performed exceptionally well.

A key aspect of our testing involves comparing the embeddings from different manuals to ensure that the structural integrity holds true across various contexts. By examining the relationships between the documents, we can confirm that the AI maintains consistent understanding and application of information, even when faced with diverse data sets. This approach not only enhances the accuracy of the AI’s responses but also validates the robustness of the embeddings used in the system.

In analyzing the costs associated with training a manual, we found that the current expenditure is approximately $5 per manual. However, we anticipate that costs will decrease for new queries made against the memories created, as all relevant information is now efficiently stored within the AI’s knowledge base. To further optimize the process, I will be reviewing the math related to document ingestion, with the goal of reducing this cost to around $1 per large document.

Currently, the document ingestion system is functional, but we recognize the need for further optimizations and fine-tuning of the AI stacks. This will help speed up processing times and reduce data handling requirements.

Additionally, Kruel.ai is now capable of learning from the information it reads. This means that as new documents are ingested, the AI not only stores the information but also builds on its understanding, thereby enhancing its ability to provide accurate responses and insights. Most importantly, this capability allows the AI to learn beyond its trained model knowledge base, integrating your specific information and requirements relevant to your line of work. As a result, Kruel.ai will become increasingly adept at adapting to the unique needs of users, enabling it to serve as a more personalized and effective tool in your professional endeavors.

If everything proceeds as planned, we will have achieved our goal of enabling the AI to learn from and interact with documents effectively. While I have yet to incorporate image capture capabilities for documents, which is on our future agenda, the text processing features are currently operating at a high level of efficacy.

2 Likes

Progress has been solid overall, though we’ve encountered a challenge with overlapping entity vectors. The next priority is refining the structure of relationships to address ownership complexities. Our initial workaround with basic logic didn’t fully address the issue, so we’ll need to refactor the relationship model to establish clearer ownership paths for validation purposes. My goal this weekend is to work on this refactor, which, if successful, should resolve nearly all identified issues to date.

Additionally, we made improvements to our pronoun tracking mechanism. Previously, I mentioned the need to accurately follow pronouns across conversations to enhance entity resolution, particularly in mixed or ambiguous contexts. We’ve transitioned from using SpaCy’s model to OpenAI’s, achieving a notable improvement in result accuracy.

1 Like

Finished a TensorFlow course on computer vision with machine learning, can’t wait to link kruel.ai into glasses and apply some of this knowledge.
next up moving into GANs

1 Like

Nice project… I’m tinkering with a similar project called ShellE SynthOS:

The ultimate goal of SynthOS is to create a working implementation of the Star Trek computer. I’ve been focused more on the self modification aspects of SynthOS and less on things like Vison and Memory so it might be interesting to compare notes at some point. The projects seem complimentary…

3 Likes

That’s a nice project. Very useful.

1 Like

It’s still early days with SynthOS. The models aren’t quite capable enough yet but they’ll get there.

I was skimming through the work you’ve been doing to improve Kruel’s latency. Have you tried the new llama 3.2 models?

2 Likes

Not latest one yet. Tired a few models well most work with the system the speed on my hardware is no where close to what is achieved with API. Perhaps in future I will invest in H100 or the likes.

1 Like

Yeah I have dual RTX4090’s and even though I can run a lot of different models with decent speed the intelligence isn’t as good as what you get from gpt-4o-mini even

2 Likes