https://hear.irmv.top/ 2026-09-16 monthly 1.0 https://hear.irmv.top/static/images/headpic.webp HEAR framework for Vision-Sound-Language-Action (VSLA) sound-centric robot manipulation HEAR reframes robot manipulation from a vision-dominant setting into a sound-causal one, covering speech, event-triggered alarms, continuous process sounds and physical interaction feedback. https://hear.irmv.top/static/images/overview.webp HEAR architecture overview: Historizer, Envisioner, Advancer and Realizer HEAR processes streaming audio continuously, reasons over multi-sensory context, learns temporal progression through future-audio prediction, and produces smooth flow-matching action chunks. https://hear.irmv.top/static/images/Historizer.webp Historizer: streaming causal audio memory across the Blind Execution Interval The Historizer is a streaming stateful transformer that maintains a compact causal audio memory across execution gaps. https://hear.irmv.top/static/images/Envisioner.webp Envisioner: hierarchical omni-modal reasoning for Vision-Sound-Language-Action control The Envisioner fuses vision, language, proprioception, the current audio window and causal audio memory into a control-ready representation. https://hear.irmv.top/static/images/Advancer.webp Advancer: audio world model predicting near-future audio codes The Advancer is formulated as an audio world model that injects temporal grounding by predicting near-future audio codes during training. https://hear.irmv.top/static/images/posters/moka-coffee.jpg HEAR real-robot Moka Coffee: listening for the sputtering phase A Franka Panda monitors the acoustics of an uninstrumented Moka pot and starts pouring only after detecting the characteristic sputtering phase. Long-horizon sound-centric process monitoring with a single ambient microphone. https://hear.irmv.top/static/videos/moka_pot_all_web.mp4 50 2026-07-27 yes no https://hear.irmv.top/static/images/posters/answer-phone.jpg HEAR real-robot Answer Phone: multi-stage speech and event tracking The robot waits for a ringtone, retrieves a handset, listens to a call, and returns the phone after the end-of-call cue, under severe visual aliasing. https://hear.irmv.top/static/videos/telephone_all.mp4 41 2026-07-27 yes no https://hear.irmv.top/static/images/posters/shake-bottle-empty.jpg HEAR real-robot Shake Bottle (Empty): active acoustic sensing Hidden container state cannot be recovered passively; the policy generates a repeatable shaking motion and reads the resulting rattle to decide whether the bottle is empty. https://hear.irmv.top/static/videos/empty_all.mp4 32 2026-07-27 yes no https://hear.irmv.top/static/images/posters/shake-bottle-occupied.jpg HEAR real-robot Shake Bottle (Occupied): discriminating hidden physical state by sound The companion case to the empty bottle: the same active acoustic probing behaviour under a different hidden physical state. https://hear.irmv.top/static/videos/occupied_all.mp4 32 2026-07-27 yes no https://hear.irmv.top/static/images/posters/real-alarm-clock.jpg HEAR real-robot Real Alarm Clock: sound-causal waiting under domain shift The simulated alarm task transferred to the physical world, where reverberation and background chatter distort the cue. The robot suppresses the visually tempting early press and acts only after the sustained ring begins. https://hear.irmv.top/static/videos/alarm_all.mp4 31 2026-07-27 yes no https://hear.irmv.top/zh/ 2026-09-16 monthly 0.9 https://hear.irmv.top/llms.txt 2026-09-16 monthly 0.6 https://hear.irmv.top/llms-full.txt 2026-09-16 monthly 0.6 https://hear.irmv.top/index.md 2026-09-16 monthly 0.6 https://hear.irmv.top/zh/index.md 2026-09-16 monthly 0.6 https://hear.irmv.top/cite.bib 2026-09-16 yearly 0.5