What i'm working on -- Self Control Reservoir Network -- Can this take us a step closer to AGI?

Community
, ,
1

Some months ago i started researching about a new type of neural network that I invented.

Briefly, this network can be thought as more similar to the human brain for his architecture, inspired by reservoir networks, and can perform arbitrarially complex calculations, is turing complete and has no context limit.

I think this technology can change LLMs and Deep Learning in general. But i am a student and startup founder, i haven’t many contacts with the high levels of research. hope someone here can watch and diffuse it.

I leave here the link for the paper i published:
Giacomo Bocchese. (2023). Self Control Reservoir Network : enhancing intelligence for Neural Networks. https://doi.org/10.5281/zenodo.7637563

Let me know what you think about, and thanks everyone.

4 Likes
2

I have found this kind of architecture can be experimented with if one thinks of prompts as programs. For example, even gpt-4 can’t solve

find an arithmetic expression over the integers 4 and 6 that evaluates to 24. You can only use each integer once

but, if I add a loop that seems very similar to your idea, but at the prompt level, gpt-3.5 has no problem. Steps 3/4 seem similar to your control-gate, I think?

find an arithmetic expression over the integers 2, 3, and 6 that evaluates to 12. You must use each integer exactly once. use the following problem-solving method:

  1. generate a trial expression
  2. evaluate the trial expression.
  3. test if the result of step 2 equals 12
  4. if test is positive, respond with trial expression and STOP.
  5. Otherwise, add trial expression to the list of expressions tried
    Loop until test is positive
2 Likes
3

ok the idea is interesting. If I understood well, what you are proposing it’s a method that uses gpt4 in an iterative way to perform complex stuff that requires recursion (this can be used for mathematical stuff and other things too), it can be also a way.

What i proposed has some similarities, but it has to be thought as totally a different approach if compared to prompt engineering and model outputs, this processes things in a fundamental way like neurons do (with flow of information seen as raw numerical values or values in time, that can propagate not only forward but flow in and out from every node to every node).

The aritmetic application i proposed is only an example but this network would be able to do every logical and mathematical operation, and thus model any function representable in a way made of recursions and mathematical operations.

This can be applied to every field: generative nlp, regression, classification, logics, math, reinforcement learning for interacting in an environment , etc

1 Like
4

How do you make training tractable?

Previous attempts to make Turing complete trained models I am aware of, end up dying on the hill of not being able to express reasonable gradients, not being able to do back propagation, and seeing too much combinatoric explosion in the solution space.

1 Like
5

Interesting proposal, are you studying maths or computer science? :laughing:

May I ask how far you are in your studies?

1 Like
6

Have you tried training it with something like ZORB?

It’s derivative-free and uses the Moore-Penrose pseudoinverse.

It’s been awhile since I read this paper, but it might be helpful.

1 Like
7

I skimmed the paper and to be honest I’m not an ML guy so can’t weigh in too much as to the math behind your proposal… I would like more information as to the role of the input gate and output gate. I think I get conceptually what you’re going for but what prevents the network from getting stuck in an infinite loop due to the fact the output gate never allows an output through? Also when the model loops internally, what prevents it from just predicting the same output from the previous loop? Again I just skimmed the paper and not an ML guy…

I ask these questions not to discourage you, quite the contrary, because you may be onto something. This gating and looping you’re proposing is what many of us are doing but as a post processing step. We’re applying gates to the output and if we don’t like the models response we’re feeding the output back into the model, along with some tokens expressing our dissatisfaction, and then asking the model to try again.

I’m trying to understand if what you’re proposing is to do this as a function of the neural network.

1 Like
8

Here’s a challenge for you. I have a husband with a head injury and does really odd things and is very vocal and opinionated… Perhaps make a functional brain for him.

1 Like
9

Yeah you say right.
The self-control reservoir (SCRN) is untrainable with gradient-based methods. We need to use a zeroth-order optimization algorithm. Obiouvsly, this requires a longer training time, because you need more flops to converge to the optimum. I believe nowadays we must leave back backpropagation. It’s an algorithm unsuitable for a lot of more complex approaches on deep learning. I believe that to get AGI, we need to get rid of backprop. Nowadays computers are much more powerful and i think we can afford to build an LLM with a zeroth-order trained model.

The advantage is that, even if SCRNs are harder to train with a setted number of parameters, we would need much less parameters to model complex functions. I believe we can make an LLM with just some million parameters or even less.

I worked quite a lot with genetic algorithm, and variations of genetic algorithms. Maybe can be the right candidate for training this networks.

10

Ok.
Im actually studying Energy engineering, im at the third year of the bachelor degree.
I am CTO & CO-founder of Robotics.it LLC , a newborn startup that provides services to companies by using LLMs (chatbots and other stuff).

I started studying Deep Learning by my own at the age of 17, because i liked programming and i have seen some interesting videos on youtube, and i am the person who isn’t able to just watch things and leave them there, i like to practice. So i started programming deep neural networks in processing (a programming language for visual applications), developing my own training algorithms based on genetic algorithms (this because i liked it more than backprop, and i found it easier to implement). I built agents on a simulated environment, to test multi-agent approaches in deep learning.

When i started engineering at the university i got a great boost, because calculus and algebra gave me a strong mathematical background that i could apply to ML. A year ago i started working in the NLP field as a freelancer.

Now im 22, i will finish the university on the next year and i will use all my time for Deep Learning and related business. Im waiting for that time :rofl:

The story became too long , im sorry :rofl:

11

Wow that seems really cool. The abstract amazed me. I will try to figure out if it’s suitable for my applications.

> Gradient descent and backpropagation have enabled neural networks to achieve remarkable results in many real-world applications. Despite ongoing success, training a neural network with gradient descent can be a slow and strenuous affair. We present a simple yet faster training algorithm called Zeroth-Order Relaxed Backpropagation (ZORB). Instead of calculating gradients, ZORB uses the pseudoinverse of targets to backpropagate information. ZORB is designed to reduce the time required to train deep neural networks without penalizing performance. To illustrate the speed up, we trained a feed-forward neural network with 11 layers on MNIST and observed that ZORB converged 300 times faster than Adam while achieving a comparable error rate, without any hyperparameter tuning. We also broaden the scope of ZORB to convolutional neural networks, and apply it to subsamples of the CIFAR-10 dataset. Experiments on standard classification and regression benchmarks demonstrate ZORB’s advantage over traditional backpropagation with Gradient Descent.

The only thing i need to understand is if this approach can be used also in non feed-forward networks. thank you!

1 Like
12

Thanks for your answer.
No problem it’s normal to find hard the math behind, i will try to explain it better here.

Self Control Reservoir Network (SCRN):

  1. Is a reservoir network (meaning that neurons are not organized in layers, and information flows in every direction, both backward and forward (this is similar to what happens in our brain), all neurons inside the network are connected to each other
  2. There are some “special” neurons that control if gates are open or closed, but they work exactly as all the other neurons in the “circuit”, the only difference is that with their activations they control the gates
  3. There is an input gate and looping gate (also named output gate)
  4. The network itself decides at a time to stop the recursion processes, when the activation of the looping control neuron goes over a threshold, the control neuron is the one that controls the output gate (if the activation of the node is higher than for example 0.5)
  5. If the output gate is “opened”, the network stops looping and returns an output, if it’s “closed” it continues looping
  6. If the input gate (controlled by the input control neuron) is opened, the network receives input, otherwise receives a vector of zeros as the input. (this function “input control” it’s likely that i will remove it in the future because i find it not necessary)

About the other questions, yes we can assume that this network embeds inside itself functions of re-modeling and recursive improvement. But this “self controlled recursion” allows not only to gradually improve the outputs, it’s also a way to model more complex things by using only a few neurons, and theoretically allowing to be turing complete and perform arbitrary complex operations , getting to AGI (this is already to prove).

To prevent getting stuck in infinite loops we can pose a limit on iterations (on tests i put 10-100 iterations, even because my laptop is not so fast :rofl:).

What prevents to iterate the same result is that i put a penalty on recursion, so that the network learns that if an output is ready it’s better to loop as less as possible

1 Like
13

I think we would need a very, very strong SCRN to substitute a human’s brain, and still not sure it can

14

When I looked at the paper a few years ago, I was going to use it to train RNN’s. Think of each cycle or loop of an RNN as a single layer of an FFN. Since the paper mentions training one layer at a time, you would translate this to training one cycle at a time for the RNN case.

But when I skimmed your paper, the first thing I thought of was this zeroth order backprop or ZORB. Especially with it returning random values sometimes, which lined up with the concept of a “big vat of random neurons”, kinda like our brains sometimes :rofl:

15

Ok, compelling on the surface. What is the next step? What would it take to get a proof of concept, compute power? Funding?

16

Then we get into the “how is this tractable” problem.
Sure, computers are a bit faster and you get a few more cores on the GPU, but switching to genetic models gives you exponentially more expensive training.
These kinds of models have been proposed before, but historically haven’t manages to actually perform, because they couldn’t be trained.
We need an algorithmic improvement in training for these kinds of models. That’s where the breakthrough will come, not in the model architecture itself, IMO.

What complexity analysis are you basing this on? With a vocabulary of 50000 words (tokens) I don’t see how this would even have enough space to encode the input and output text, much less the behavior of the program.

Or, to put it another way: if this model is Turing equivalent, could you write that same program in a few megabytes of compiled code?

17

that’s a bit different. my model learns how to solve problems, like all neural models. a compiled model is static, can’t learn.

yes for sure we need an algorithmic improvement, but i think this model is worth a try.

I haven’t made analysis yet. Based on theory, if you can model things with recursion you can build complex nonlinear function without the need of many neurons but only with iterating the same small net many times.

18

yes exactly. the fact is that my network can’t “backpropagate” anything since there isn’t a forward direction nor a backward.

i will see if i can apply it, thanks

1 Like
19

I have two plans that i can apply alternatively:
1- receive some funding and private studying (i can study it more if i have some money to rent powerful CPUs / GPUs for intensive testing)
2- Come in contact with someone that can study this in collaboration with me, a PHD researcher in Deep Learning, a college professor, a research institute.

I put it here in openai because i thought maybe someone can continue this work. I have only a little time and can’t dedicate much on this project since i need to work for money for my life.

I would really appreciate if some university teacher or researcher would continue this work.

20

I’m always happy to see people interested in science, doing extra studies outside the curriculum always gets a big gold start from me, I’m sure you got a bright future ahead of you :star:

Reservoir Computing (RC) is a framework for training Recurrent Neural Networks (RNNs) that was developed to overcome some of the difficulties associated with training traditional RNNs. In a traditional RNN, all weights, including those for connections between hidden nodes (internal weights), are adjusted during training.

As I understand it, the new features of the network you propose include the unique features:

  1. All internal weights of the reservoir are trained and not left random.

  2. Control nodes can control the information flow through the network and perform decisions on the reasoning process, like accepting or not the input or looping many times the state update until an output is accepted.

These features seem very similar to stuff that has been explored in previous research. It’s very possible that you were unaware of the previous works, at the time of writing this.

The concept of training internal weights in Reservoir Computing (RC) is not entirely new. In traditional RC, only the output weights are trained, and the internal weights are typically fixed and randomly initialized. Here are just a few studies that have explored the idea of training internal weights in RC.

The paper titled “An approach to reservoir computing design and training” discusses the dynamics of reservoir computing, where the reservoir dynamic is not only determined by its fixed weights [1]. Another paper, “Reservoir computing approaches to recurrent neural network training”, also mentions training both reservoir-internal and output weights [2].

The concept of control nodes in neural networks is also not entirely new. The paper “Recognition with self-control in neural networks” discusses different kinds of self-control mechanisms of recognition in neural networks [3].

With that in mind let’s go over some of key points of reviewing your paper:

Originality: The concept of a Self-Control Reservoir Network doesn’t seem entirely new, you could benefit from pointing out key differences. instead of just stating that the concept is new.

Justification: The paper justifies the need for such a network by comparing it with the functioning of the human brain and existing neural networks. it could benefit from a more detailed explanation of why these features are necessary, why do these features make the functioning more similar to the human brain and how do they improve upon existing models, include empirical evidence to support your claims.

Evidence: The paper states that all what is written has not been proved yet and would need more studies to be confirmed. There’s no empirical evidence or experimental results to support the proposed concept. What methods will you use to test claims? How do you quantify the improvements? Are the claims falsifiable?

Practical Implications: The paper suggests that the proposed network can perform any calculation and can be more efficient than traditional RNNs. it does not provide any practical examples or applications to demonstrate this.

Conclusion: The paper concludes by stating the potential of the proposed network. This is discussion material. a conclusion is for summarizing the key points and suggesting future research directions.

the paper lacks empirical evidence to support its claims. Your future research should focus on testing the proposed network, providing empirical evidence of its effectiveness, and remember to properly cite relevant work.

That last point is important, It’s not just about text, but also about using someone else’s ideas or unique methodology without acknowledgment, even if the words are changed. it’s generally considered good academic practice to cite previous works that have discussed or used similar features. This not only gives credit where it’s due but also helps to situate the new work within the context of the existing literature. It helps substantiate your claims and helps the reader find resources for more knowledge.

Remember that science is the systematic pursuit of understanding and explaining the world and its phenomena through observation and experimentation, whereas engineering is the application of scientific knowledge to design and build practical solutions to problems.

Where to go from here:

Your paper is marked as a journal article, it is not. It’s a pre-print, I assume this is an honest mistake, but it may be viewed as attempted academic dishonesty, as it suggests that the paper has been peer reviewed. This is especially true because you uploaded it to CERN’s archive servers, this may get you blacklisted from posting in an actual journal in the future. This could seriously hurt your career.
My honest advice to you is to pull the paper down, fix the issues and get a “mentor” aka a person within the scientific community with a background in machine learning or similar that can help you determine when your paper is ready for publishing. I’m not say this to discourage you, remember this is 1% idea and 99% work, keep at it and I’m sure you’ll make great progress :heart:

  1. An approach to reservoir computing design and training ↩︎

  2. Reservoir computing approaches to recurrent neural network training ↩︎

  3. Recognition with self-control in neural networks ↩︎

2 Likes