Researchers at Peru’s ESAN University have developed an open-source approach that enables a commercial humanoid robot to reproduce human movements captured on conventional video, combining artificial intelligence, reinforcement learning and physical testing to address one of robotics’ most persistent technical challenges.
Humanoid robots are moving beyond scripted demonstrations toward systems capable of learning physical actions from ordinary video.
At ESAN University in Peru, researchers are investigating how to translate recorded human movement into executable instructions for a commercial humanoid robot, tackling the difficult transition between visual observation, digital simulation and stable physical performance.
The project, developed within the university’s Smart Lab at ESAN R&D Labs, centers on a Unitree G1 humanoid robot. Its researchers have developed a pipeline that reconstructs movement from conventional monocular video, trains the robot to imitate that movement in simulation and then tests whether the resulting behavior can be reproduced on actual hardware.
The work is described in a scientific paper submitted for evaluation, titled Reproducible Motion Imitation from Monocular Video on a Commercial Humanoid: An Open-Source Deployment Pipeline. The submission is an account of the team’s research, not a peer-reviewed publication. Its emphasis is not simply on making a robot copy a choreography, but on documenting the steps, checks and constraints required to make imitation repeatable.
That distinction matters. A convincing robotic demonstration can conceal extensive manual adjustment, favorable laboratory conditions or repeated attempts. Reproducible research instead asks whether the process can be examined, repeated and improved by other teams. ESAN’s approach combines artificial intelligence, three-dimensional reconstruction, reinforcement learning, simulation and physical testing, with open-source software and accessible computing resources.
A video may look like a straightforward teaching tool. For a robot, however, it is an incomplete record of the action being performed. A conventional camera captures a two-dimensional view of a three-dimensional body. It does not directly reveal every joint angle, the forces acting through the feet or the subject’s position.
The difficulty increases when the target is a humanoid machine rather than another person. Human joints, muscles and balance mechanisms differ from those of a commercial robot. A motion that appears natural on screen may exceed a robot’s joint range, demand excessive acceleration or shift its center of mass beyond a stable position.
Researchers therefore need to interpret the recorded motion rather than copy it. The process involves extracting information about human posture, reconstructing a usable representation of movement and adapting that representation to the robot’s mechanical configuration. Each transformation introduces opportunities for error, making verification an essential part of the experiment.
Monocular video is particularly relevant because it requires only one camera viewpoint. Unlike specialized motion capture installations, which may depend on multiple synchronized cameras or markers attached to a performer, ordinary footage can be collected easily. That accessibility could make movement imitation research easier for laboratories without extensive capture infrastructure.
Accessibility does not eliminate ambiguity. A limb can disappear behind the body, a camera angle can distort perceived depth and rapid motion can obscure important transitions. A reconstructed movement sequence must therefore be treated as an estimate, not a perfect measurement of the original action. The challenge is to produce a representation sufficiently consistent for training.
Once the movement has been reconstructed, the next problem is teaching the machine to execute it. ESAN’s research uses reinforcement learning, an approach in which a control policy is developed through repeated interaction with a simulated environment. Training can explore how a robot responds to movement demands while reducing the risks of physical experimentation.
Simulation is valuable precisely because humanoid motion is unforgiving. A poorly timed step or unstable change of posture can cause a robot to fall. In a virtual environment, researchers can examine these failures, adjust their approach and repeat experiments without damaging equipment. Yet simulated success is only an intermediate result.
The decisive test is whether a policy developed digitally can operate on a real machine. Differences between simulated dynamics and physical hardware can affect friction, timing, actuator response and balance. These discrepancies are often described as the simulation-to-reality gap, one of the central obstacles in deploying learned robotic behaviors outside simulation.
ESAN reports that its researchers transferred a choreography learned in simulation to the Unitree G1 and executed it consistently across consecutive physical tests. The reported result demonstrates that the team achieved a functioning transfer for that sequence. It does not establish that the robot can imitate any arbitrary human action or work in uncontrolled settings.
That boundary is important when interpreting progress in humanoid robotics. Reproducing a particular choreography is a narrower accomplishment than developing a general-purpose machine that can observe unfamiliar tasks and perform them safely. Nevertheless, a repeatable transfer from video to simulation and then to hardware provides a useful experimental foundation for exploring more complex behavior.
The research also produced practical lessons about where training and deployment can fail. Rather than treating unsuccessful attempts as incidental setbacks, the team used its findings to establish technical verification procedures intended to identify potential problems before a movement is deployed on the physical robot. Such checks can make experimentation more systematic and reduce testing.
Verification is especially significant because a movement can appear plausible in a digital representation while remaining unsuitable for execution. Researchers must consider whether the intended sequence respects the robot’s limits and whether its transitions preserve stability. Identifying these issues early helps separate errors in movement reconstruction from problems arising during control or physical deployment.
The open-source dimension gives the project another purpose. Robotics research can be difficult to reproduce when methods depend on proprietary tools, expensive infrastructure or undocumented configuration decisions. A deployment pipeline built around more accessible resources may help other laboratories understand how an experiment was assembled and where adaptations are needed for their equipment.
Open-source tools alone do not guarantee reproducibility. Researchers also need clear descriptions of assumptions, training settings, validation methods and the conditions under which tests were performed. ESAN’s focus on technical checks and the documentation of difficulties reflects the wider importance of making the process visible, rather than presenting an isolated success.
For the university, this is part of a broader effort to turn access to advanced machines into applied scientific knowledge. Smart Lab operates within ESAN R&D Labs, a research hub that brings together facilities for experimentation and technological development. Its robotics work combines computational methods with tests on physical systems, allowing researchers to compare theoretical expectations against reality.
“Our goal is to turn technological experimentation into an opportunity to conduct research, learn and generate new knowledge,” said Naldi Carrión, Head of ESAN R&D Labs.
That philosophy places equal weight on understanding failure and achieving visible performance. A robot executing a movement is the most immediate result, but the deeper research questions concern why it succeeded, what conditions were necessary and which limitations remain. Answers to those questions can guide the design of later experiments and help other researchers avoid repeating the same mistakes.
The program also extends beyond motion imitation. ESAN R&D Labs is exploring human-robot interaction, including how people perceive and respond to humanoid and quadruped robots. This work introduces behavioral and social questions alongside engineering challenges, reflecting the fact that robots intended for shared environments must be evaluated in relation to the people around them.
A technically capable machine may still behave in ways that people find confusing, intimidating or difficult to anticipate. Studying perception and response can inform how researchers design physical interactions and social experiences with robots. It also creates opportunities for collaboration between engineering, technology and behavioral sciences, beyond mechanical and software specialists.
For students and faculty, a laboratory built around real machines can connect abstract ideas to measurable outcomes. Artificial intelligence models, control policies and simulation environments become subjects of experimental investigation when their outputs must satisfy the limitations of physical hardware. That connection is particularly valuable in a field where visually impressive results do not always translate into dependable performance.
The implications of the work should be viewed carefully. ESAN has described successful consecutive tests and a proposed reproducible deployment pipeline, but the available account does not provide quantitative performance benchmarks, comparative evaluations or evidence of deployment in industrial environments. Without those details, claims about superiority, general applicability or commercial readiness would go beyond the reported findings.
What the project does illustrate is a research direction with broader relevance: reducing the distance between demonstrations of human movement and controlled, testable robotic execution. By combining ordinary video, three-dimensional reconstruction, reinforcement learning and hardware validation, the researchers are examining how sophisticated humanoid behavior might become more accessible to study.
The next challenge for this class of research is not simply adding more movements. It is determining how reliably a method works across different recordings, actions and conditions, and how its failures can be identified before they create physical risks. Those questions will shape whether techniques developed in laboratories can support increasingly adaptable robots.
Future work could examine robustness across camera angles, performers and increasingly demanding sequences.
At ESAN University, the immediate contribution is a practical investigation of that transition. The Unitree G1’s choreography is the visible outcome, but the more consequential work lies in the reconstruction, training, verification and physical testing that made it possible. For humanoid robotics, progress depends as much on understanding those processes as on the movements a machine can perform.
