The acquisition and self-improvement of novel motor skills is among the most important problems in robotics. I will discuss various learning techniques we developed that enable robots to have complex interactions with their environment and humans. Complexity arises from dealing with high-dimensional input data, non-linear dynamics in general and contacts in particular, multiple reference frames, and variability in objects, environments and tasks. A human teacher is always involved in the learning process, either directly (providing data) or indirectly (designing the optimization criterion), which raises the question: How to best make use of the interactions with the human teacher to render the learning process efficient and effective? I will discuss various methods we have developed in the fields of supervised learning, imitation learning, reinforcement learning, and interactive learning. All these concepts will be illustrated with benchmark tasks and real robot experiments ranging from fun (ball-in-a-cup) to more applied (sorting products).