It seems that every second person wears an activity tracker, smart watch or GPS-enabled sports watch these days. And professional sportspeople are often ‘sensored up’, letting us see how many kilometres our favourite cricketer covers during a test match, or how high a rugby player’s heart rate gets when they break and sprint for the try line. Coaches routinely analyse the GPS and heart rate data of their squads after training. But how useful is the data?
It provides some information on how much a person has moved: distance covered, speed, heart rate, but no insight into the nature or quality of that movement. It gives a rough measure of ‘training volume’, but if you’re looking to minimise injury or train a new skill further information is needed.
It’s a problem being tackled by PhD student Danica Hendry, from Curtin’s School of Physiotherapy and Exercise Science, as she studies the contributing factors towards pain and disability in dancers. Professional and pre-professional ballet dancers have an intense physical training regime, which can over time lead to fatigue and overload injuries. Recording and managing their physical workload is completely subjective, and usually limited to personal activity diaries and training schedules. These may document hours spent in training, but don’t record the frequency of specific movements and musculoskeletal loads which may lead to injury.
Her project has turned into a collaboration between physiotherapists Professor Leon Straker and Professor Peter O’Sullivan, biomechanists Dr Amity Campbell and Dr Luke Hopper (ECU), and computer scientists Professor Tele Tan and Dr Kevin Chai. It spans the Curtin Institute for Data Science and the Schools of Physiotherapy and Exercise Science and Civil and Mechanical Engineering at Curtin, along with Edith Cowan University’s Western Australian Academy of Performing Arts (WAAPA).
Hendry aims to better measure the training volume and specific musculoskeletal loads in a cohort of female pre-professional ballet dancers at the WAAPA, but existing activity trackers can’t tell a jeté (jump) or an arabesque (leg lift) from a plié (bending at the knees), and don’t record much when dancers train on one spot at the barre. So to record specific movements, the research team are building an automated human activity recognition system.
“You can put sensors on dancers and measure their movement, but if you don’t know if that dancer was running, jumping or spinning it’s basically useless data,” explains Hendry. “So as a first step we also videoed the dancers, and correlated what we saw against the sensor data.”
She used six sensors per dancer, each incorporating an accelerometer, a gyroscope and a magnetometer, placed on the left and right shins, left and right thighs, sacrum and thoracic spine to document movement as each dancer worked through specific movements. These continuous signals were then segmented and manually cross-referenced against the video footage, so specific signal segments could be connected to individual dance movements.
Each dancer is different, so the research team recorded 23 dancers as they worked through a sequence of dance movements, both as isolated movements with a clearly defined beginning and end, and ‘buried’ within choreographed sequences of dance. Forty minutes of annotated video and correlated sensor data from each dancer built up the 106 GB base ‘library’ of data on specific movements. The movements included in the study were jumps, as the forces exerted on the body on landing are implicated in lower limb injury, and leg lifts as they are implicated in hip and lower back pain.
Collecting a meaningful data set and making sense of it is slow and laborious work, so for larger sensor data sets, Hendry turned to machine learning. CIC specialist Dr Kevin Chai led the team to build a convolutional neural network, which was trained using Hendry’s library of manually-classified movement data. Training let the network identify patterns and diagnostic features in the mass of sensor data that had been correlated via video with different jumps and leg lifts.
They then tested the network on data from two dancers kept out of the training set, to see how accurate it was in assessing patterns of movement in an unknown dancer. Using data from all six sensors, the network could identify target movements with 80 per cent accuracy or better, which was accurate enough to assess training load. But the team was excited to realise that if the network was restricted to data from only one sensor, the one on the sacrum, the neural network still had over 75 per cent accuracy.
“The great thing about that is we can consider using this approach in performance, as well as training,” explains Hendry. “You obviously want to impose as little as possible on the dancers, especially when they’re performing. Having only one sensor, which can be hidden under a costume, opens up avenues to studying performance, not just training.”
Hendry is now recording sensor-only data from 52 dancers over an entire day of training, four times across a semester, and using the trained neural network to convert that into a quantitative measure of jumping and leg-lifting training volume for each. The dancers are also completing a survey each data collection day, self-assessing a range of emotional, cognitive and lifestyle factors, any pain experienced, and any limitations that has on their training. The data will then be used to look at the trajectory of each dancer across the semester and explore the various factors that correlate with pain and disability.
The ramifications of her PhD study are much larger though. If you can create the appropriate correlated training data you can then measure almost anything from just the sensor data. The team are now considering quantifying jumps better – not just when and how often they happen, but how high the dancer jumps, and how much force is exerted on landing.
It doesn’t just have injury implications, and Dr Luke Hopper at WAAPA agrees. “We want to assist our pre-professional dancers to reach the challenging heights of being professional dancers. To replicate the aesthetics and artistry achieved by elite dancers. So a tool like this can be calibrated to focus training, and measure outputs comparable to live performances. We’re really excited about what this approach can offer to our knowledge base.”
Hendry’s supervisor Dr Amity Campbell also concurs. “Field-based analysis is the new way of doing biomechanical research. With this approach we can capture the athlete, the dancer, in their normal environment. If you bring them into the laboratory with cameras and sensors and treadmills and pressure plates it’s not their normal day, and not their normal performance pressures. Capturing their activity in real conditions will be so much more useful for injury prevention, performance development, and high-performance training.”
