The use of automated systems describing human motion has become possible in various domains. Most of the proposed systems are designed to work with people moving around in a standing position. Because such a system could be interesting in a medical environment, we propose in this work a pipeline that can effectively predict human motion from people lying on beds. The proposed pipeline is tested with a data set composed of 41 participants executing 7 predefined tasks in a bed. The motion of the participants is measured with video cameras, accelerometers and a pressure mat. Various experiments are carried out with the information retrieved from the data set. Two approaches combining the data from the different measurement technologies are explored. The performance of the different carried experiments is measured, and the proposed pipeline is composed of components providing the best results. Later on, we show that the proposed pipeline only needs to use video cameras, which makes the proposed environment easier to implement in real-life situations.