HAF: Adapting Generalist VLAs to Humanoid Whole-Body Loco-manipulation via Hierarchical Action Flow and Spectral Latent RL

1. Introduction & Executive Overview

Deploying pretrained generalist Vision-Language-Action (VLA) foundation models on high-dimensional humanoid robots introduces severe control challenges. While generalist VLA models have demonstrated strong multi-task capabilities on tabletop or fixed-base manipulators, applying them directly to humanoid whole-body loco-manipulation is non-trivial. Humanoid control requires the simultaneous synchronization of locomotion, waist posture, and bimanual manipulation across high-dimensional action spaces.

As highlighted by Gu et al., conventional single-stage VLA architectures generate all body actions in a single forward pass or flat denoising step. This single-stage action generation entangles lower-body locomotion with upper-body control. In physical deployments, minor perturbations in base locomotion propagate upwards, inducing erratic compensatory upper-body movements that compromise balance and degrade fine manipulation precision. Furthermore, post-training VLA policies via online reinforcement learning (RL) directly on physical hardware introduces severe safety hazards, high computational overhead, and the risk of eroding pretrained foundation priors.

To solve these bottlenecks, Gu et al. introduce the Humanoid Adaptation Framework (HAF). HAF transfers off-the-shelf, generalist VLA models to humanoid whole-body loco-manipulation through two complementary modules:

1. HAF-VLA: A hierarchical action-flow generator that restructures single-stage VLA generation into a progressive, kinematics-aligned action pipeline.
2. HAF-Steer: A spectral latent reinforcement learning optimizer that leverages flow-matching invertibility and frequency-domain compression to refine frozen VLA policies safely during real-world exploration.

---

2. Architectural Mechanics of HAF-VLA

Instead of producing full-body actions in a flat, single-stage step, HAF-VLA structures action generation according to the forward-kinematic dependencies of humanoid robots. Conditioned on an egocentric RGB image observation $I_t$, robot proprioception $q_t$, and a language instruction $\ell$, the VLA predicts an action chunk $A_t = [a_t, \dots, a_{t+H-1}] \in \mathbb{R}^{H \times D}$, where $H$ is the action prediction horizon and $D$ is the per-step action dimension.

Gu et al. decompose each instantaneous action $a_t$ into four disjoint kinematic sub-components: $a_t = [a_t^{\text{move}}, a_t^{\text{head}}, a_t^{\text{waist}}, a_t^{\text{manip}}]$ representing base locomotion commands, head gaze orientation, waist posture adjustments, and bimanual manipulation targets.

+-----------------------------------------------------------------------------------+
|                            HAF-VLA Generation Pipeline                            |
+-----------------------------------------------------------------------------------+
|  Observation (O_t, l) ---> Shared VL Prefix Encoder ---> Prefix Cache (P_t)       |
|                                                                                   |
|  Stage 1: Noise (e_1) + P_t                      ---> Action Subset A_1 (Base/Head) |
|                                                            |                      |
|                                                     Cache Operator (C_1^t)        |
|                                                            v                      |
|  Stage 2: Noise (e_2) + P_t + C_1^t              ---> Action Subset A_2 (+Waist)  |
|                                                            |                      |
|                                                     Cache Operator (C_2^t)        |
|                                                            v                      |
|  Stage 3: Noise (e_3) + P_t + C_1^t + C_2^t      ---> Full Action A_3 (+Arm/Manip)|
+-----------------------------------------------------------------------------------+

3-Stage Progressive Denoising Pipeline

HAF-VLA constructs three nested cumulative action index sets satisfying the inclusion relation $A_1 \subset A_2 \subset A_3$:

• Stage 1 ($A_1 = A_{\text{move}} \cup A_{\text{head}}$): The action expert generates base locomotion velocities, task skill modes, and head orientation commands to establish dynamic stability and direct visual attention.
• Stage 2 ($A_2 = A_1 \cup A_{\text{waist}}$): Active action dimensions expand to incorporate waist actuation, adjusting torso posture to re-orient the upper-body workspace.
• Stage 3 ($A_3 = A_2 \cup A_{\text{manip}}$): The complete action space is activated to output fine dual-arm manipulation targets alongside refined base and torso trajectories.

Because later stages operate over cumulative action sets, they can iteratively refine the coarse action predictions generated in earlier stages, preventing rigid decoupling while ensuring kinematic coherence.

Cross-Stage Key-Value (KV) Cache Conditioning

To link generation stages without re-running the full vision-language backbone, HAF-VLA uses cross-stage Key-Value (KV) cache conditioning. At step $t$, the vision-language prefix cache $P_t = \mathcal{F}_{\text{prefix}}(o_t, \ell)$ is computed once and shared across all stages.

When Stage 1 completes, a re-encoding cache operator $C_1^t = \text{Cache}_\theta(A_1^t \mid P_t, e_1)$ compresses the predicted clean actions into an action KV cache. Stage 2 takes $C_1^t$ as additional context, and its output is similarly cached into $C_2^t$. Stage 3 is conditioned on $(P_t, C_1^t, C_2^t, e_3)$, where $e_s$ represents a trainable stage embedding for stage index $s \in \{1, 2, 3\}$.

Stage Index	Active Kinematic Sub-components	Cached Context Dependencies
Stage 1	Locomotion & Head ($A_{\text{move}} \cup A_{\text{head}}$)	Vision-Language Prefix Cache ($P_t$)
Stage 2	Locomotion, Head & Waist ($A_1 \cup A_{\text{waist}}$)	Shared Prefix ($P_t$) + Stage 1 Action Cache ($C_1^t$)
Stage 3	Full Body: Locomotion, Head, Waist & Dual-Arm ($A_2 \cup A_{\text{manip}}$)	Shared Prefix ($P_t$) + Stage 1 Cache ($C_1^t$) + Stage 2 Cache ($C_2^t$)

Training and Real-World Execution

During training, Gu et al. utilize teacher forcing to compute cross-stage caches $C_1^t$ and $C_2^t$ from ground-truth masked actions rather than noisy model outputs, preventing error compounding across stages. The shared action expert is optimized using a stage-wise flow-matching velocity loss evaluated under the Frobenius norm: $\mathcal{L}_{\text{HAF}} = \mathbb{E}_{A, \{\epsilon_s, \tau_s\}_{s=1}^3} \left[ \sum_{s=1}^3 \frac{1}{HD} \| v_\theta(X_{\tau_s, t}^s, \tau_s; h_t^s) - u_t^s \|_F^2 \right]$

During real-robot inference, the three stages execute sequentially, with each stage running 10 flow-denoising steps (30 total steps). The system adopts a receding-horizon execution scheme: HAF-VLA predicts an action chunk of length $H=100$ steps, the low-level controller executes the first 40 steps, and the next VLA inference cycle is triggered. On a single NVIDIA RTX 5090 GPU, the complete 3-stage inference pipeline runs in approximately 0.12 seconds.

---

3. HAF-Steer: Spectral Latent RL for Safe Policy Refinement

Deploying policies trained purely via offline behavior cloning (BC) leaves robots vulnerable to deployment-time distribution shifts. However, fine-tuning large VLA backbones directly with online RL on physical robots presents severe safety hazards: unconstrained updates can induce erratic motions that physically damage hardware, demand prohibitive compute, and destroy pretrained foundation priors.

+-----------------------------------------------------------------------------------+
|                             HAF-Steer Optimization                               |
+-----------------------------------------------------------------------------------+
|  Expert Action A_i* ---> Flow Reversal (F_theta^-1) ---> Initial Noise e_i*        |
|                                                                 |                 |
|                                                            DCT (Top-K=8)          |
|                                                                 v                 |
|  Online Buffer D_on \                                  Spectral Targets z_i*      |
|                      ===> Mixed SAC Update + BC Reg.            |                 |
|  Offline Buffer D_off /        (Frozen VLA)                     v                 |
|                                                          Spectral Replay D_off    |
+-----------------------------------------------------------------------------------+
|                                Inference Pipeline                                 |
|  Observation x_t ---> RL Actor (pi_psi) ---> z_t ---> IDCT_K ---> Noise e_t        |
|                                                                     |             |
|  Executable Action A_t <--- Frozen Flow-Matching VLA (F_theta) <-----/            |
+-----------------------------------------------------------------------------------+

Limitations of Prior Latent Noise RL

Previous latent noise RL techniques attempt to bypass backbone fine-tuning, but introduce critical trade-offs:

• Single-Vector Repetition (e.g., DSRL): Repeats a single noise vector across the temporal horizon. Gu et al. observed that this construction severely degrades temporal expressiveness, frequently causing high-frequency joint jerking and transient instability on high-dimensional humanoids.
• Full-Dimensional Noise Optimization (e.g., UniSteer): Optimizes uncompressed temporal noise over the entire action horizon ($H \times D$), creating a massive search space that results in low sample efficiency.

Flow Reversal and DCT Spectral Parameterization

HAF-Steer restricts RL optimization to a low-dimensional Discrete Cosine Transform (DCT) noise subspace. Truncating the temporal noise representation to the first $K=8$ spectral coefficients acts as an implicit physical low-pass filter. This design filters out high-frequency exploratory jitter that causes joint torque fluctuations and hardware tripping during online exploration.

The flow-matching inversion and parameterization pipeline proceeds as follows:

1. Numerical Flow Reversal: Demonstrated expert trajectories $A_i^*$ are mapped back to their corresponding initial Gaussian noise space $\epsilon_i^*$ via numerical backward integration of the frozen flow field ($F_\theta^{-1}$): $\epsilon_i^* = F_\theta^{-1}(A_i^* \mid \zeta_i^*)$
2. Spectral Compression: A DCT is applied along the temporal dimension of $\epsilon_i^*$, retaining only the first $K=8$ low-frequency coefficients to yield $c_i^* \in \mathbb{R}^{K \times D}$. This truncates high-frequency noise components and compresses the exploration space from $H \times D$ down to $K \times D$.
3. Normalization: The coefficients are normalized using precomputed dataset-wide mean and standard deviation matrices ($\mu_c, \sigma_c$) across the demonstration dataset: $z_i^* = (c_i^* - \mu_c) / (\sigma_c + \delta)$

Two-Stage Policy Training Scheme

1. Behavior-Cloning (BC) Initialization: The stochastic spectral actor $\pi_\psi(z \mid x)$ is initialized by regressing its mean output $\mu_\psi(x)$ toward offline expert spectral actions in $\mathcal{D}_{\text{off}}$: $\mathcal{L}_{\text{BC}} = \mathbb{E}_{(x_i, z_i^*) \sim \mathcal{D}_{\text{off}}} \left[ \| \mu_\psi(x_i) - z_i^* \|^2_2 \right]$
2. Mixed Offline-Online SAC: The actor interacts with the environment, storing rollouts in an online replay buffer $\mathcal{D}_{\text{on}}$. Mini-batches are sampled from $B = \mathcal{D}_{\text{off}} \cup \mathcal{D}_{\text{on}}$ to update the policy using Soft Actor-Critic (SAC). BC regularization is applied strictly to offline data to prevent policy drift while maintaining pretrained performance: $\mathcal{L}_\pi = \mathbb{E}_{x_i \sim B, z_i \sim \pi_\psi} \left[ \alpha \log \pi_\psi(z_i \mid x_i) - \min_j Q_{\omega_j}(x_i, z_i) \right] + \lambda_{\text{BC}} \mathbb{E}_{(x_i, z_i^*) \sim \mathcal{D}_{\text{off}}} \left[ \| \mu_\psi(x_i) - z_i^* \|^2_2 \right]$

Throughout adaptation, the entire VLA backbone remains frozen.

Signal Flow During Execution

The end-to-end inference signal flow follows a discrete sequence:

1. Observation Input ($x_t$): The agent receives observation $x_t = (I_t, q_t)$.
2. Spectral Action Sampling ($z_t$): The RL actor samples normalized spectral action $z_t \sim \pi_\psi(\cdot \mid x_t)$.
3. De-normalization ($c_t$): Spectral coefficients are rescaled via $c_t = \mu_c + (\sigma_c + \delta) \odot z_t$.
4. IDCT Reconstruction ($\epsilon_t$): Inverse DCT zero-pads truncated higher frequencies to construct full temporal noise $\epsilon_t = \text{IDCT}_K(c_t)$.
5. Frozen VLA Flow Generation ($A_t$): The frozen VLA backbone decodes initial noise $\epsilon_t$ into an executable action chunk: $A_t = F_\theta(\epsilon_t \mid \zeta_t)$

---

4. Empirical Evaluation & Task Performance Benchmarks

Gu et al. validated HAF on two physical humanoid platforms: TienKung 2.0 and TienKung 3.0. Teleoperation data collection comprised 120 trajectories per task using a head IMU for gaze control, kinematically matched master arms for dual-arm manipulation, and handheld joysticks for locomotion and waist posture.

+-----------------------------------------------------------------------------------+
|                           Real-World Hardware Setup                               |
+-----------------------------------------------------------------------------------+
|  Perception: Onboard Egocentric RGB Camera + Head-Mounted IMU                      |
|  Teleoperation: Master Arms (Arms/Hands) + Joystick (Legs/Waist)                  |
|  Platforms: TienKung 2.0 & TienKung 3.0 Humanoid Robots                           |
+-----------------------------------------------------------------------------------+

Evaluation Metric & Task Suite

The framework was evaluated across seven real-world household tasks: Laundry Loading, Clothes Retrieval, Table Tidy, Basket Transfer, Toy Storage, Ball Tossing, and Box Transfer.

To measure performance across long sequential horizons, Gu et al. utilized a normalized milestone task score: $\text{Score } (\%) = \frac{s_{\text{achieved}}}{s_{\text{max}}} \times 100$ where milestone sub-goals were assigned discrete partial credit scores across 10 independent trials per task.

+-----------------------------------------------------------------------------------+
|                                Task Suite Overview                                |
+-----------------------------------------------------------------------------------+
|  1. Laundry Loading   : Pick clothes from bed, walk, load into washing machine    |
|  2. Clothes Retrieval : Retrieve clothing from drying rod, hang in wardrobe       |
|  3. Table Tidy        : Walk to table, close laptop lid, throw trash in bin       |
|  4. Basket Transfer   : Squat, pick basket, turn, walk, place on table           |
|  5. Toy Storage       : Pick toy from table, walk, deposit into basket            |
|  6. Ball Tossing      : Pick ball, walk toward box, throw ball into target box    |
|  7. Box Transfer      : Pick box from table, navigate to second table, set down   |
+-----------------------------------------------------------------------------------+

Comparative Task Performance

HAF-VLA achieved superior whole-body coordination compared to state-of-the-art single-stage imitation learning and VLA baselines.

Average Normalized Real-World Task Score (%)

HAF-VLA (Gu et al.) [==========================================] 70.5%
pi0.5               [===================================] 53.3%
GR00T N1.7          [=========================] 37.5%
Cosmos Policy       [==================] 27.5%
ACT                 [=================] 25.0%

Model	Average Normalized Task Score (%)	Primary Failure Modes Observed by Authors
ACT	25.0%	Error accumulation over long horizons; lacks generalist VLA priors.
Cosmos Policy	27.5%	High online inference latency; inaccurate spatial placement.
GR00T N1.7	37.5%	Frequent locomotion drift; uncontrolled base-arm interference.
$\pi0.5$	53.3%	Unstable base movements triggering upper-body compensation errors.
HAF-VLA (Gu et al.)	70.5%	High balance stability; precise multi-stage coordination.

Architectural Ablation Synthesis

To isolate the specific benefits of progressive kinematics-aligned generation, Gu et al. conducted targeted ablation experiments on the Laundry Loading task, comparing full HAF-VLA against two architectural variants (each operating with 30 total flow-matching denoising steps):

1. All-Joint Denoising: Denoises all action dimensions simultaneously across all three stages without kinematic partitioning.
2. Arm-First Hierarchy: Reverses the structural order, generating arm manipulation targets in Stage 1 before predicting base locomotion and waist posture in later stages.

Architectural Ablations on Laundry Loading (Normalized Score %)

HAF-VLA (Base-First) [==========================================] High Performance
All-Joint Denoising  [=======================] Reduced Performance
Arm-First Hierarchy  [==========] Critical Failure

The ablation results highlight a core principle in humanoid control: generating arm targets prior to establishing base locomotion and posture context (Arm-First Hierarchy) results in severe task failures. When arm end-effector targets are fixed first, subsequent base motion shifts render the planned arm configurations physically unreachable or trigger invalid Inverse Kinematics (IK) solves. Conversely, HAF-VLA establishes base stability and workspace orientation first, allowing Stage 3 arm targets to be generated within a stable, grounded reference frame.

Post-Training Refinement via HAF-Steer

Applying HAF-Steer to frozen VLA backbones provided substantial performance improvements across both in-distribution and out-of-distribution (OOD) settings. Importantly, evaluating the base models prior to RL refinement reveals distinct baseline starting points: base $\pi0.5$ achieves ~35% average success, whereas base HAF-VLA achieves ~60% average success.

Average Success Rate Across Tasks & Settings (%)

HAF-VLA + HAF-Steer [==========================================] ~92%
HAF-VLA + Noise BC  [==================================] ~75%
HAF-VLA Base Model  [==========================] ~60%
HAF-VLA + DSRL      [============] ~30%

pi0.5 + HAF-Steer   [========================================] ~87%
pi0.5 + Noise BC    [===========================] ~62%
pi0.5 + DSRL        [=================] ~40%
pi0.5 Base Model    [==============]] ~35%

• Base VLA Performance: Base $\pi0.5$ achieves ~35% average task success, while base HAF-VLA achieves ~60% average task success due to its progressive kinematic structure.
• Single-Vector Repetition (DSRL): Degrades performance across backbones (~40% on $\pi0.5$ and ~30% on HAF-VLA) due to unconstrained high-frequency joint jerking during online rollouts.
• Noise BC Initialization: Improves performance to ~62% on $\pi0.5$ and ~75% on HAF-VLA by regressing toward offline spectral targets.
• HAF-Steer Refinement: Achieves ~92% average success when paired with HAF-VLA (and ~87% when paired with $\pi0.5$), demonstrating the efficacy of spectral latent RL over frozen backbones.

Robustness Under Environmental Disturbances

Gu et al. evaluated generalization by introducing controlled visual and physical perturbations:

• Visual Distraction (Laundry Loading with an unseen chair near the navigation path): HAF-VLA maintained a normalized score of 40.0%, compared to 26.7% for $\pi0.5$.
• Positional Perturbation (Clothes Retrieval with a 20 cm backward starting shift): HAF-VLA achieved a normalized score of 43.3%, outperforming $\pi0.5$ (36.7%).

---

5. Summary & Strategic Takeaways for AI Safety and Robotics

The HAF framework demonstrates that generalist VLA models can be adapted to high-dimensional humanoid whole-body control without incurring the high cost of pretraining humanoid models from scratch or risking dangerous physical exploration during online RL fine-tuning.

Key Takeaways

• Kinematic Decoupling Suppresses Cascading Physical Failures: Entangling locomotion, waist posture, and manipulation in a single generative step creates structural instability in high-dimensional humanoids. Progressive, kinematics-aligned action denoising ($A_1 \subset A_2 \subset A_3$) ensures base stabilization prior to fine dual-arm execution, preventing lower-body perturbations from triggering erratic upper-body movements.
• Spectral Compression Constrains Physical Exploration Risks: Performing online RL in full action space or uncompressed temporal noise ($H \times D$) introduces safety risks during real-robot rollouts. Truncating noise optimization to low-frequency DCT coefficients ($K=8$) functions as an implicit physical low-pass filter natively embedded within the action space. This filters out high-frequency noise components that cause rapid joint torque spikes, motor tripping, and hardware wear during physical exploration.
• Frozen-Backbone Adaptation Preserves Pretrained Foundation Priors: Direct fine-tuning of multi-billion parameter VLA backbones risks catastrophic forgetting and requires immense compute. By performing flow reversal ($F_\theta^{-1}$) and steering initial Gaussian noise while keeping the VLA frozen, HAF-Steer updates less than 1% of total parameters while recovering robust task-level adaptation.
• Cross-Stage KV Caching Maintains Global Coherence at Low Latency: Progressive multi-stage execution can introduce latency bottlenecks. Utilizing cross-stage KV caches ($C_1^t, C_2^t$) allows later stages to re-use encoded prefix context ($P_t$) and early-stage action context efficiently, running full whole-body generation cycles in ~0.12 seconds on commercial hardware.

Read the full paper on arXiv · PDF