flag
flag
demo

A single bee possesses a vision system that is decades ahead of our algorithms. Similarly, ants can travel the most complicated fields, create bridges as well as boats while our hexapod robots still have issues with difficult terrains. Why can't our algorithms behave as well as these relatively simple living beings? What are we missing?

There is no concluding answer to the above questions but Nature seems to have arrived in such organisms using two important development mechanisms: evolution and learning. In the same way, my research focus on using evolution and learning to solve a wide range of problems from scheduling to vision and robotics.

The long term objective of my research is to create a general learner: a fully automatic algorithm that is versatile enough to learn and adapt to any circunstance. Yes, I want to plug it into a robot and see the robot walking better and better. I want to plug it into an airplane simulation and see it controlling it as well as pilots. Then I want to plug it into a real airplane and let it adapt to the new system while using the previous knowledge from the simulation. Last but not least, the general learner would find new ways of doing things when one of its components do not work anymore or when the problem has changed. In summary, I want to create a new generation of artificial intelligence systems that are as intelligent as a bee.


Research Achievements


Unified Neural Model and Spectrum-Diverse Neuroevolution

Current learning systems behave sometimes erratically when facing new problems or require a lot of trial and error to set its parameters as well as prior knowledge to work. In [6], we showed how it is possible to learn in 5 completely different control problems (continuous input continuous output partially observable Markov decision processes, learning logic gates, function approximation, etc) without changing any parameters. This achievement was made possible by two contributions:

  • Unified Neural Model - To learn a wide variety of problem classes, learning systems need a greater representation power. Therefore, a novel neural model called Unified Neural Model is proposed that unifies most neural network features from the literature into one representation.
  • Spectrum-Diverse Neuroevolution - To develop such complex representations, neuroevolution is used with a new concept that calculates the spectrum of candidate solutions based on their characteristics and use it to keep the diversity. This enable high dimensional structures to be compared efficiently and scale well with their size.
The code for the Spectrum-diverse Unified Neuroevolution Architecture (SUNA) is available here.



Self-Organizing Classifiers
In this research, there are two main achievements:
  • The Creation of Self-Organizing Classifiers (SOC) - SOC [2] (download draft), [3] is a new class of machine learning algorithms. It is one of the few methods that can solve hard reinforcement learning problems such as big continuous input continuous output control problems such as mazes where the agent starts in random positions. It surpasses actor-critic based algorithms in both performance and robustness [4]. In fact, it is the only algorithm that can adapt on the fly when the problem changes (e.g., maze changes in shape [2], mountain car problem changes its characteristics [5]). This is possible because SOC automatically creates states as well as updates them, adapting with experience. For each of these automatically created states, classifiers are learned.
  • Unified Paradigms in Reinforcement Learning - In [5], the two paradigms of reinforcement learning (value function approaches and direct policy search) were joined for the first time. This was made possible by using policies composed of sub-parts. The sub-parts can be developed based on value function approaches while the policy itself can be learned using a combination of previous developed sub-parts and new sub-parts with policy search. In this manner, both paradigms can be combined.



General Subpopulation Framework


In this work published in [1] (download draft), there are three main contributions:
  • Created the concept of Optimization Forces - This concept was used to analyze and demonstrate that there is a deleterious competition inside sets of candidate solutions in population-based optimization algorithms;
  • General Subpopulation Framework - Formalized and generalized structured optimization algorithms (algorithms that have a structure of candidate solutions). This theoretical contribution can aid in the development and analysis of structured optimization algorithms by formalizing them as subsets with interactions;
  • Developed the current state-of-the-art in multi-objective optimization – By using the General Subpopulation Framework an algorithm was created that surpassed all other methods in one of the hardest benchmarks for multi-objective optimization. In fact, the proposed algorithm excelled specially in challenging problems (see Figure on the left). Thus, further application to real world problems looks promising.