Rapid and accurate recognition of human action and road condition is a foundation and precondition of implementing self-control of intelligent prosthesis. In this paper, a Gaussian mixture model and hidden Markov model are used to recognize the road condition and human motion modes based on the inertial sensor in artificial limb (lower limb). Firstly, the inertial sensor is used to collect the acceleration, angle and angular velocity signals in the direction of x, y and z axes of lower limbs. Then we intercept the signal segment with the time window and eliminate the noise by wavelet packet transform, and the fast Fourier transform is used to extract the features of motion. Then the principal component analysis (PCA) is carried out to remove redundant information of the features. Finally, Gaussian mixture model and hidden Markov model are used to identify the human motion modes and road condition. The experimental results show that the recognition rate of routine movement (walking, running, riding, uphill, downhill, up stairs and down stairs) is 96.25%, 92.5%, 96.25%, 91.25%, 93.75%, 88.75% and 90% respectively. Compared with the support vector machine (SVM) method, the results show that the recognition rate of our proposed method is obviously higher, and it can provide a new way for the monitoring and control of the intelligent prosthesis in the future.