Evidence
Same model and position · retrieved passages supplied · first output
Data-driven approaches have since expanded beyond body motion to encompass speech-conditioned and text-conditioned synthesis, as well as the animation of apparel and facial avatars. These extensions broaden the range of controllable signals and animated entities that motion generation systems must accommodate.
Passages supplied to the Evidence version
Card 1 · T3M: Text Guided 3D Human Motion Synthesis from Speech
Speech-driven 3D motion synthesis seeks to create lifelike animations based on human speech, with potential uses in virtual reality, gaming, and the film production.Existing approaches reply solely on speech audio for motion generation, leading to inaccurate and inflexible synthesis results.To mitig…
Read full passage excerpt
Speech-driven 3D motion synthesis seeks to create lifelike animations based on human speech, with potential uses in virtual reality, gaming, and the film production.Existing approaches reply solely on speech audio for motion generation, leading to inaccurate and inflexible synthesis results.To mitigate this problem, we introduce a novel textguided 3D human motion synthesis method, termed T3M.Unlike traditional approaches, T3M allows precise control over motion synthesis via textual input, enhancing the degree of diversity and user customization.The experiment results demonstrate that T3M can greatly outperform the state-of-the-art methods in both quantitative metrics and qualitative evaluations.We have publicly released our code at https://github.com/Gloria2tt/naacl2024.
Card 2 · Generating Upper‐Body Motion for Real‐Time Characters Making their Way through Dynamic Environments
Abstract Real‐time character animation in dynamic environments requires the generation of plausible upper‐body movements regardless of the nature of the environment, including non‐rigid obstacles such as vegetation. We propose a flexible model for upper‐body interactions, based on the anticipation o…
Read full passage excerpt
Abstract Real‐time character animation in dynamic environments requires the generation of plausible upper‐body movements regardless of the nature of the environment, including non‐rigid obstacles such as vegetation. We propose a flexible model for upper‐body interactions, based on the anticipation of the character's surroundings, and on antagonistic controllers to adapt the amount of muscular stiffness and response time to better deal with obstacles. Our solution relies on a hybrid method for character animation that couples a keyframe sequence with kinematic constraints and lightweight physics. The dynamic response of the character's upper‐limbs leverages antagonistic controllers, allowing us to tune tension/relaxation in the upper‐body without diverging from the reference keyframe motion. A new sight model, controlled by procedural rules, enables high‐level authoring of the way the character generates interactions by adapting its stiffness and reaction time. As results show, our real …
Card 3 · Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation
Animating stylized characters to match a reference motion sequence is a highly demanded task in film and gaming industries. Existing methods mostly focus on rigid deformations of characters' body, neglecting local deformations on the apparel driven by physical dynamics. They deform apparel the same…
Read full passage excerpt
Animating stylized characters to match a reference motion sequence is a highly demanded task in film and gaming industries. Existing methods mostly focus on rigid deformations of characters' body, neglecting local deformations on the apparel driven by physical dynamics. They deform apparel the same way as the body, leading to results with limited details and unrealistic artifacts, e.g. body-apparel penetration. In contrast, we present a novel method aiming for high-quality motion transfer with realistic apparel animation. As existing datasets lack annotations necessary for generating realistic apparel animations, we build a new dataset named MMDMC, which combines stylized characters from the MikuMikuDance community with real-world Motion Capture data. We then propose a data-driven pipeline that learns to disentangle body and apparel deformations via two neural deformation modules. For body parts, we propose a geodesic attention block to effectively incorporate semantic priors into skel …
Card 4 · GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatars
Photo-realistic and controllable 3D avatars are crucial for various applications such as virtual and mixed reality (VR/MR), telepresence, gaming, and film production. Traditional methods for avatar creation often involve time-consuming scanning and reconstruction processes for each avatar, which lim…
Read full passage excerpt
Photo-realistic and controllable 3D avatars are crucial for various applications such as virtual and mixed reality (VR/MR), telepresence, gaming, and film production. Traditional methods for avatar creation often involve time-consuming scanning and reconstruction processes for each avatar, which limits their scalability. Furthermore, these methods do not offer the flexibility to sample new identities or modify existing ones. On the other hand, by learning a strong prior from data, generative models provide a promising alternative to traditional reconstruction methods, easing the time constraints for both data capture and processing. Additionally, generative methods enable downstream applications beyond reconstruction, such as editing and stylization. Nonetheless, the research on generative 3D avatars is still in its infancy, and therefore current methods still have limitations such as creating static avatars, lacking photo-realism, having incomplete facial details, or having limited dr …
Model-reported card use
Card 1 · fact
Affected output: speech-conditioned and text-conditioned synthesis
Model explanation: This row supplied the concrete fact that speech-conditioned motion synthesis is an established data-driven task, which I used to name speech-conditioned synthesis as one expansion beyond body motion.
Card 3 · fact
Affected output: the animation of apparel and facial avatars
Model explanation: This row supplied the concrete fact that apparel animation for stylized characters is a data-driven motion task, which I used to name apparel animation as one expansion beyond body motion.
Card 4 · fact
Affected output: the animation of apparel and facial avatars
Model explanation: This row supplied the concrete fact that controllable 3D avatar generation is a data-driven task, which I used to name facial avatar animation as one expansion beyond body motion.