Adaptation is a property of intelligent machines to update its knowledge according to actual situation. Self-learning machines (SLM) as defined in this paper are those learning by observation under limited supervision, and continuously adapt by observing the surrounding environment. The aim is to mimic the behavior of human brain learning from surroundings with limited supervision, and adapting its learning according to input sensory observations. Recently, Deep Belief Networks have made good use of unsupervised learning as pre-training stage, which is equivalent to the observation stage in humans. However, they still need supervised training set to adjust the network parameters, as well as being non-adaptive to real world examples. In this paper, SLM is proposed based on deep belief networks and deep auto encoders to adapt to real world unsupervised data flowing in to the learning machine during operation. As a proof of concept, the new system is tested on two AI tasks; number recognition on MNIST dataset, and E-mail classification on Enron dataset.