Nowadays, Artificial Intelligence, notably Advanced Machine Learning (ML) drives scientific and economic growth worldwide. They are essentially massive ‘learning by experience/examples’ systems. However, as our tasks and the world change, such systems should adapt to new domains/tasks and continue learning. Knowledge should be transferred from one DNN systems to other ones. Distributed DNN training should be performed though Federated Learning, e.g., for privacy protection. New Learning modes should be explored, by reward maximation, as it is done in Deep Reinforcement Learning and Imitation Learning.