The video is not the diagnostic
From the outside, movement analysis can look surprisingly simple. Record a few seconds of someone performing a Deep Squat, wait for the analysis, and receive feedback.
But the video itself is not the diagnostic.
Between those few seconds of movement and the result lies a series of engineering decisions. The position of the phone can alter the apparent geometry. The system has to establish where the person is within the squat, decide which detected body landmarks are reliable, and identify the moment or phase relevant to each measurement. It also has to account for differences in camera distance, framing, and image dimensions so that these differences are not mistaken for differences in movement. Only then can we determine what the geometry actually tells us.
Some information may not come from automatically detected body landmarks at all. For movement characteristics where the computer-vision model does not provide the geometry we need, we can add a short, structured manual annotation instead of pretending that automation can reliably infer something it cannot see.
The goal is not to automate every step. It is to make every step explicit enough that we know where a measurement came from and what we are allowed to conclude from it.
We built Glance as a sequence of stages with separate responsibilities. The first begins before the movement is recorded.
Control what should not change
Recording starts by controlling the conditions that should remain consistent.
If the phone is tilted, angles measured relative to image vertical can change even though the person’s movement did not. If someone stands substantially off-center, perspective can distort the apparent geometry, particularly in the frontal view.
Rather than leaving setup to written instructions alone, Glance provides visual and audio guidance before recording begins. The interface indicates whether the phone is level, while audio cues help the participant move into the center of the image during self-recording.
This prevents an avoidable difference in camera setup from becoming a difference in movement data.
Positioning feedback
Ready to record
Not everything can be controlled perfectly. Participants are currently instructed to position the phone approximately two meters away, but in practice they may only approximate this distance. Measurements that depend on scale therefore require another strategy. Instead of interpreting image-space displacement on its own, Glance relates the movement to dimensions estimated from the participant’s own body. This becomes important when measuring Squat Depth.
Finding the phases of a squat
Once the recording conditions are under control, the squat must be treated as a movement through time rather than a single pose.
The person starts standing, descends into the squat, reaches the bottom, rises again and returns to standing. A diagnostic needs to know where it is within that sequence. Measuring trunk inclination during quiet standing, for example, answers a very different question from measuring it near the bottom of the squat.
Glance therefore first establishes the structure of the movement: standing, descent, bottom, ascent and return to standing. Each diagnostic can then select the phase or a specific frame within that phase that is relevant to the movement characteristic it measures.
The movement context is shared. This prevents every diagnostic from having its own definition of where the squat begins, where the bottom is or when the person starts rising. Glance analyzes the movement once, and the diagnostics work from that shared reference.
With those phases established, we can define what each diagnostic measures from the detected body positions.
Turning landmarks into measurements
Knowing the phase is not enough; each diagnostic still needs an explicit measurement definition. A computer-vision model can locate points such as the hip, knee and ankle in the image. Those landmarks are inputs. They are not diagnostics.
Squat Depth is a good example. At the bottom of the squat, Glance compares the vertical position of the hip with the knee. But the distance between two points in an image depends on how large the person appears. Instead of reporting that image-space distance on its own, we relate it to the participant’s own body dimensions, measured during standing. If the hip remains above the knee, the vertical distance between them is expressed relative to thigh length. If the hip passes below the knee, the distance beyond knee level is expressed relative to shank length.
Tibia Inclination requires a different approach. The ankle and knee landmarks define the shin, but calculating its angle correctly requires a defined coordinate system and reference direction. Glance measures the shin relative to image vertical. For the current diagnostic, the primary value is taken when the knee reaches 90 degrees of flexion during descent. Because that moment will often fall between two video frames, the system interpolates the shin orientation between them rather than pretending that the nearest frame occurred at exactly 90 degrees.
Engineering view
User-facing view
These diagnostics use some of the same body landmarks and both depend on a clearly defined point in the movement, but their measurements are different. Squat Depth uses body-relative scaling at the bottom of the squat, while Tibia Inclination uses angular geometry at the point during descent when the knee reaches 90 degrees of flexion.
Automatically detected landmarks do not always provide all the geometry a diagnostic needs. A planned Spine Flexion diagnostic, for example, is expected to use a short manual annotation to provide information that the pose model does not capture reliably.
A landmark is a coordinate. A diagnostic requires a definition.
That definition determines what is measured, when it is measured and which conditions have to be satisfied before we accept the result.
Keeping measurement and interpretation separate
A measurement gives us a quantitative result. It does not yet tell us what that result means.
That distinction is important. A measured angle should not jump directly from geometry to a conclusion about movement quality, let alone to an explanation of why someone moved that way. We therefore separate each diagnostic into four stages: Measurement, Observation, Interpretation and Explanation.
Measurement produces the quantitative result. Observation turns accepted measurements into structured evidence about the movement. Interpretation is where that evidence receives meaning, for example by relating it to a defined reference or category. Only then does Explanation communicate the accepted result to the user, through text and visual evidence.
- Measurement
- Observation
- Interpretation
- Explanation
Keeping these stages separate gives us a practical constraint: changing an interpretation cannot alter the measurement underneath it, and changing the user-facing explanation cannot create a new interpretation. A result shown in Glance can therefore be traced back to the observation, measurement and movement evidence that produced it.
A measurement must not only be clearly defined; it must also be sensitive to meaningful changes in the movement.
Does the measurement respond when the movement changes?
A diagnostic should respond when the movement characteristic it is designed to measure changes. During development, we explore this by deliberately introducing observable differences in movement and running each recording through the same diagnostic pipeline.
One exploratory check uses frontal-view recordings with progressively greater leftward trunk lean: neutral, mild, medium and strong. Each recording passes through the same implementation so we can inspect whether the trunk-position signal changes in the same direction.
Neutral
Mild left
Medium left
Strong left
These experiments are deliberately limited in what they claim. They are not validation against laboratory motion capture. They answer a more immediate question: when an observable movement characteristic changes, can we see that change in the measurement?
Connecting individual findings
Each Glance diagnostic is built to stand on its own. Squat Depth, Tibia Inclination and Trunk Inclination each follow the same path from measurement to observation, interpretation and explanation.
But movement does not happen one diagnostic at a time.
Tibia inclination is a good example. On its own, it describes how far the shin inclines relative to vertical at a defined point in the squat. Consider it alongside trunk inclination, and we can begin to describe how different parts of the body contribute to the same movement strategy.
This is where movement synthesis comes in. Rather than going back to raw landmarks or measurements, synthesis can work from the structured interpretations produced by the individual diagnostics.
- Tibia Inclination Interpretation
- Trunk Inclination Interpretation
- Other diagnostics Interpretation
Movement Synthesis
For the first release, movement synthesis will focus on this relationship. Research suggests that the relative orientation of the trunk and tibia can help characterize how demands are distributed between the hip and knee extensors more clearly than either angle alone (Barrack et al., 2021; Graber et al., 2023; Straub & Powers, 2024). Glance will therefore combine the structured interpretations of Tibia Inclination and Trunk Inclination using a relationship defined in advance, while keeping both underlying diagnostic results available for review. The synthesis adds context; it does not replace either diagnostic.
From movement to meaning
A short movement video is only the input. The engineering work lies in deciding which evidence we can use, how to measure it and how far the resulting interpretation is allowed to go.
The difficult part is not detecting a body in a video. It is building a path from movement to measurement to meaning that we can inspect, test and improve.
The Deep Squat is the first implementation of that approach.
The movement perspective behind this work has also been shaped by ongoing discussions with Witalij Kühne. I am grateful for his time and continued support.
References
Barrack, A. J., Straub, R. K., Cannon, J., & Powers, C. M. (2021). The relative orientation of the trunk and tibia can be used to estimate the demands on the hip and knee extensors during the barbell back squat. International Journal of Sports Science & Coaching, 16(4), 1004–1010. https://doi.org/10.1177/1747954121997957
Graber, K. A., Halverstadt, A. L., Gill, S. V., Kulkarni, V. S., & Lewis, C. L. (2023). The effect of trunk and shank position on the hip to knee moment ratio in a bilateral squat. Physical Therapy in Sport, 61, 102–107. https://doi.org/10.1016/j.ptsp.2023.03.005
Straub, R. K., & Powers, C. M. (2024). A biomechanical review of the squat exercise: Implications for clinical practice. International Journal of Sports Physical Therapy, 19(4), 490–501. https://doi.org/10.26603/001c.94600