Words in Motion

Extracting Interpretable Control Vectors for Motion Transformers

---

Ömer Şahin Taş FZI Research Center for Information Technology

Royden Wagner Karlsruhe Institute of Technology (KIT)

Paper (arXiv) Code OpenReview BibTeX Poster Video

ICLR 2025

TLDR: We measure neural collapse toward interpretable features in hidden states. By measuring the collapse towards natural language-aligned features, we identify well-separated directions that enable us to fit interpretable control vectors. We then optimize these control vectors using sparse autoencoders (SAEs). Finally, we use linearity measures to compare steered outputs against a Koopman autoencoder and SAEs with various layers and activation functions, including convolutional and MLPMixer layers.

*equal contribution

---

Introduction

Deep learning models that achieve higher accuracy typically rely on the increased complexity of their architectures and representations . This, in turn, renders them difficult to interpret in terms of semantically meaningful concepts. Interpretability methods aim to uncover how models process information, often by analyzing whether learned features align with semantically meaningful concepts Concept-based interpretability. Kim et al. propose explaining predictions with human-interpretable concepts, rather than relying on sample-based raw features . They first choose a set of examples that represent distinct concepts and measure the influence of high-level concepts on the model's decisions, thereby providing global explanations. In a recent text-to-image diffusion setting, Conceptor decomposes a concept into a weighted combination of interpretable elements..

The manifold hypothesis suggests that high-dimensional data often lies on a lower-dimensional manifold. Deep learning models learn representations that approximately capture the manifold's geometry by mapping data into a space where related inputs are closer together . Training objectives further shape this representation space: loss functions encourage the clustering of data samples in latent space . Together with regularizers that prevent overfitting, these clusters become more distinct over the course of training, a phenomenon known as neural collapse A recent line of work introduces the term neural collapse to describe a desirable learning behavior of deep neural networks for classification. It refers to the phenomenon that learned top-layer representations form semantic clusters, which collapse to their means at the end of training. In addition, the cluster means transform progressively into equidistant vectors when centered around the global mean. Therefore, neural collapse facilitates classification tasks and is considered a desirable learning behavior for both supervised and self-supervised learning . Neural collapse is not to be confused with representation collapse , where learned representations across all classes collapse to redundant or trivial solutions (e.g., zero vectors). . Therefore, we focus on the structure of the learned representations Structure of the latent space. show that consistent regularities naturally emerge from the training process of word embeddings. This phenomenon, commonly referred to as the word2vec hypothesis, suggests that learned embeddings capture both semantic and syntactic relationships between words through consistent vector offsets in latent space. While the observed linear offsets naturally fit a flat latent space, non-Euclidean geometric models (e.g., Riemannian manifolds) can better capture structural distortions . In those cases, "vector arithmetic" can be seen as an approximation to geodesic operations on a curved latent space. to predict model behavior, with a focus on transformer models for motion forecasting.

Following Ben-Shaul et al. , we use linear probes to measure neural collapse toward interpretable features in hidden states. High probing accuracy implies separability of features, which suggest functionally important directions in hidden states. Building on the insight that interpretable features are embedded in hidden states, we fit control vectors Also referred to as steering vectors , style vectors , or activation addition . Control vectors are used for a form of activation steering, where concept-based vectors are added to activations (i.e., hidden states) of transformer models. In natural language processing , control vectors allow targeted adjustments to model outputs by modifying hidden states without the need for fine-tuning or prompt engineering. Control vectors are a set of vectors that capture the difference of hidden states with opposing concepts or features . This approach requires a well-structured latent space, where samples are clustered according to classes or features (e.g., a high degree of neural collapse, see the neural collapse section). to the directions between hidden states with opposing features.

To further enhance this approach, we use sparse autoencoders (SAEs) A key goal of interpretability research is to decompose models and gain a mechanistic interpretation of how their components function. Sparse autoencoders (SAEs) leverage the linear representation hypothesis and approximate the model's activations with a linear combination of feature directions. By enforcing sparsity in latent space, they separate features into distinct, interpretable representations . Related autoencoders linearize learned representations either by manifold flattening or using Koopman operators . to extract more distinct features from hidden states . We evaluate sparse autoencoders with fully-connected, convolutional, and MLPMixer layers; and different activation functions. Our experiments with sparse autoencoders of varying sparse intermediate dimensions show that enforcing sparsity leads to more linear changes in prediction when scaling control vectors.

We apply our method to recent multimodal motion transformers . They process features of past motion sequences (i.e., past positions, orientation, acceleration, and speed) and environment context (i.e., map data and traffic light states), and transform them into future motion sequences. Like other transformer models, they rely on learned representations of these features, resulting in hidden states Hidden state activations. Transformers consist of attention blocks, followed by simple feed-forward networks, whose hidden state activations are analyzed for interpretability. Elhage et al. explore two key hypotheses that describe how these activations capture meaningful structures: the linear representation hypothesis and the superposition hypothesis . These hypotheses essentially state that neural networks represent features as directions in their activation space, and that representations can be decomposed into independent features. that are difficult to interpret and control. We focus on analyzing interpretable motion features that are physically measurable, such as speed, acceleration, direction, and agent type. By leveraging these features, our approach enables interpretable control over generated forecasts and facilitates zero-shot generalization.

Specifically, in this work:

• We argue that, to fit control vectors, latent space regularities with separable features are necessary. We use linear probing and show that neural collapse toward interpretable features occurs in hidden states of recent motion transformers, indicating a structured latent space.
• We fit control vectors using hidden states with opposing features. By modifying hidden states at inference, we show that control vectors describe functionally important directions. Similar to the vector arithmetic in word2vec, we obtain predictions consistent with the current driving environment.
• We use sparse autoencoders to optimize our control vectors. Notably, enforcing sparsity leads to more linear changes in predictions when scaling control vectors. We use linearity measures to compare these results against a Koopman autoencoder and SAEs with various layers and activation functions, including convolutional and MLPMixer layers.

Our method differs from prior works in several aspects. We measure neural collapse in multimodal models for motion forecasting (i.e., regression) instead of unimodal image classifiers or language models . Unlike Conmy et al. , we do not manually suppress SAE features in control vectors. Furthermore, we do not use our SAEs during inference , but to optimize control vectors beforehand, resulting in negligible computational overhead.

---

Method

Motion feature classification using natural language

In contrast to natural language, where words naturally carry semantic meaning, motion lacks predefined labels. Therefore, we identify human-interpretable motion features by quantizing them into discrete subclasses as in natural language.

Initially, we classify motion direction using the cumulative sum of differences in yaw angles, assigning it to either left, straight, or right. Additionally, we introduce a stationary class for stationary objects, where direction lacks semantic significance. We define further classes for speed, dividing the speed values into four intervals: high, moderate, low, and backwards. Lastly, we analyze the change in acceleration by comparing the integral of speed over time to the projected displacement with initial speed. Accordingly, we classify acceleration profiles as either accelerating, decelerating, or constant. Our thresholds for motion features are based on insights from Ettinger et al. and Seff et al.. The threshold values are detailed in the appendix.

Diagram: Method overview Step 0: Transformer model Step 1: Linear probing Step 2: PCA control vectors Step 3: SAE training Step 4: SAE optimized control vectors

Figure 1: We measure the degree to which these interpretable features are embedded in the hidden states \mathbf{H}_{i,:} of transformer models with linear probes. Furthermore, we use our discrete features and sparse autoencoding to fit interpretable control vectors \mathbf{V}_{i,:} that allow for modifying motion forecasts at inference. The training of the sparse autoencoder is shown with red arrows ({\color{#f1615c}\rightarrow}) and the fitting of control vectors with blue arrows ({\color{#5eb8e7}\rightarrow}). Overview diagram for the motion transformer architecture used in our experiments, highlighting the flow from inputs through transformer modules to forecasted trajectories. We measure the degree to which these interpretable features are embedded in the hidden states \mathbf{H}_{i,:} of transformer models with linear probes. We measure the degree to which these interpretable features are embedded in the hidden states \mathbf{H}_{i,:} of transformer models with linear probes. Furthermore, we use our discrete features and sparse autoencoding to fit interpretable control vectors \mathbf{V}_{i,:} that allow for modifying motion forecasts at inference. We measure the degree to which these interpretable features are embedded in the hidden states \mathbf{H}_{i,:} of transformer models with linear probes. Furthermore, we use our discrete features and sparse autoencoding to fit interpretable control vectors \mathbf{V}_{i,:} that allow for modifying motion forecasts at inference. The training of the sparse autoencoder is shown with red arrows ({\color{#f1615c}\rightarrow}) We measure the degree to which these interpretable features are embedded in the hidden states \mathbf{H}_{i,:} of transformer models with linear probes. Furthermore, we use our discrete features and sparse autoencoding to fit interpretable control vectors \mathbf{V}_{i,:} that allow for modifying motion forecasts at inference. The training of the sparse autoencoder is shown with red arrows ({\color{#f1615c}\rightarrow}) and the fitting of control vectors with blue arrows ({\color{#5eb8e7}\rightarrow}).

Neural collapse as a metric of interpretability

We use neural collapse as a metric of interpretability. Specifically, we focus on interpreting hidden states (i.e., activations or latent representations) and evaluate whether hidden states embed interpretable features. We measure how close abstract hidden states are related to interpretable semantics using linear probing accuracy Ben-Shaul et al. show that linear probing accuracy is consistent with the accuracy of nearest class center classifiers, which are typically used to measure neural collapse .. We train linear probes (i.e., linear classifiers detached from the overall gradient computation) on top of hidden states (\mathbf{H}_{i,:} in Figure 1). During training, we track their accuracy in classifying our interpretable features on validation sets. Adapted to motion forecasting, we choose the aforementioned motion features as interpretable semantics.

Besides linear probing accuracy, following Chen & He , we use the mean of the standard deviation of the \ell_2-normalized embedding to measure representation collapse. Representation collapse refers to an undesirable learning behavior where learned embeddings collapse into redundant or trivial representations . Redundant representations have a standard deviation close to zero. In a way, representing the opposite of neural collapse. As shown in Chen & He , rich representations have a standard deviation close to 1/\sqrt{\text{dim}}, where dim is the hidden dimension.

Interpretable control vectors

We use our interpretable features to form pairs of opposing features. For each pair, we build a dataset and extract the corresponding hidden states. Next, we compute the element-wise difference between the hidden states of samples with these opposing features. Finally, following Zou et al. we apply principal component analysis (PCA) with a single component as a pooling method. This reduces the computed differences to a single scalar per hidden dimension to generate control vectors (\mathbf{V}_{i,:} in Figure 1).

We optimize our control vectors using SAEs. SAEs extract distinct features in hidden states by encoding and reconstructing them from sparse intermediate representations (\mathbf{S}_{i,:} in Figure 1). We hypothesize that sparse intermediate representations enable a more linear decomposition of our interpretable features, and hence, more distinct control vectors. Therefore, we generate intermediate control vectors \mathbf{V}'_{i,:} by pooling the differences between hidden states with opposing features (\mathbf{H}_{i,:}^\text{pos} vs. \mathbf{H}_{i,:}^\text{neg}). Specifically, we compute

\mathbf{S}_{i,:}^\text{pos} = \text{ReLU}\Big(\mathbf{W}_\text{enc}\big(\mathbf{H}_{i,:}^\text{pos} - \mathbf{b}_\text{dec}\big) + \mathbf{b}_\text{enc}\Big),

where \mathbf{W} and \mathbf{b} denote weights and biases of the SAE. Similarly, we compute \mathbf{S}_{i,:}^\text{neg} and obtain the intermediate control vectors as

\mathbf{V}'_{i,:} = \text{PCA}\big(\mathbf{S}_{i,:}^\text{pos} - \mathbf{S}_{i,:}^\text{neg}\big).

Leveraging the Johnson-Lindenstrauss LemmaJohnson & Lindenstrauss state that a set of points in high-dimensional space can be projected into a lower-dimensional space while approximately preserving the pairwise distances between points ., we use the SAE decoder to project the intermediate control vectors back to the hidden dimension of the motion encoder

\mathbf{V}_{i,:} = \mathbf{W}_\text{dec} \mathbf{V}'_{i,:} + \mathbf{b}_\text{dec}.

This enables using sparse autoencoders of arbitrary sparse intermediate dimensions for generating control vectors of fixed dimension. At inference, we scale the control vectors with a temperature parameter (\tau in Figure 1) to control the strength of the corresponding features of a given sample.

---

Experimental setup

Motion forecasting models

We study three recent motion transformers for self-driving: Wayformer , HPTR and RedMotion . All models process a maximum of 48 surrounding traffic agents as environment context. For the Argoverse 2 Forecasting (abbr. AV2F) dataset, we use past motion sequences with 50 time steps (representing 5 seconds) as input. For the Waymo Open Motion (abbr. Waymo) dataset, we use past motion sequences with 11 steps (representing 1.1 seconds) as input. Further details on model architectures and fusion mechanisms are presented in the appendix.

Linear probes

We add linear probes for our quantized and interpretable motion features (see the motion feature classification section) to hidden state of all models (\mathbf{H}_{i,:}^{(m)} in Figure 1, where m \in \{0, 1, 2\} is the module number and i is the temporal index). These classifiers are learned during training using regular cross-entropy loss to classify speed, acceleration, direction, and the agent classes from hidden states. We decouple this objective from the overall gradient computation. Therefore, these classifiers do not contribute to the alignment of hidden states, but exclusively measure neural collapse toward interpretable features.

Control vectors

Using our interpretable motion features, we build pairs of opposing features. Specifically, we generate speed control vectors representing the direction from low to high speed, acceleration control vectors representing the direction from decelerating to accelerating, and direction control vectors representing the direction from turning left to turning right, and agent control vectors representing the direction from pedestrian to vehicle. For each pair, we use the hidden states \mathbf{H}_{i,:}^{(m)} from module m=2 and the last embedding per motion sequence (i=-1), as it is closest to the start of the prediction.

Sparse autoencoders

We train SAEs as an auxiliary model with sparse intermediate dimensions of 512, 256, 128, 64, 32, and 16. The total loss combines \ell_2 reconstruction loss with an \ell_1 sparsity penalty: \ell_2 ensures accurate reconstruction, while \ell_1 promotes sparsity by minimizing small, noise-like activations. The \ell_1 must be carefully scaled to avoid deadening important features . We scale it by 3e-4. We optimize the models over 10000 epochs using the Adam optimizer and a batch size of 528. The final loss values are provided in the appendix.

---

Results

Extracting interpretable features for motion

Our approach relies on a well-structured latent space, where samples are clustered with respect to interpretable features. First, we ensure that our features are not highly correlated, as confirmed by the Spearman feature correlation analysis in the appendix. Next, we report linear probing accuracy for interpretable features during training on the AV2F and Waymo datasets.

The figures below show the linear probing accuracies for our interpretable motion features for the AV2F dataset. The scores are computed on the validation split over the course of training. All models achieve similar accuracy scores, while the Wayformer model achieves slightly higher scores for classifying acceleration and lower scores for agent classes. Overall, we measure high linear probing accuracy for all interpretable features. This shows that all models likely exhibit neural collapse toward interpretable features.

Figure 2: Linear probing accuracies for RedMotion, Wayformer, and HPTR on the validation split of the AV2F dataset.

The representation quality metric normalized standard deviation of embeddings is shown in the figure above. Both HPTR and RedMotion learn to generate embeddings with a normalized standard deviation close to the desired value of 1 / \sqrt{\text{dim}}, where \text{dim} is the hidden dimension. The scores for Wayformer are lower, which reflects differences between attention-based and MLP-based motion encoders.

Figure 3: Normalized standard deviation representation quality metric for RedMotion, Wayformer, and HPTR.

The figure below shows the linear probing accuracies for our interpretable features on the Waymo dataset. Here, we report the scores for each of the three hidden states \mathbf{H}_i in the RedMotion model (i.e., after each module m in the motion encoder, see Figure 1). Similar accuracy scores are reached for all features at all three hidden states. The accuracies for the speed and acceleration classes continuously improve, while those for direction classes reach 0.80 early on. Compared to the direction scores on the AV2F dataset, the scores on the Waymo dataset "jump" earlier. We hypothesize that this is linked to the shorter input motion sequence on Waymo (1.1 seconds vs. 5 seconds), which limits the amount of possible movements. In contrast to the AV2F dataset, higher accuracies are achieved for classifying speed. Overall, the highest scores are reached for classifying agent types, as on the AV2F dataset.

Figure 4: Linear probing accuracies at module 0, module 1 and module 2 for classifying speed, acceleration, direction, and agent type on the validation split of the Waymo dataset.

In addition to linear probing, we measure neural collapse using class-distance normalized variance (CDNV) , see the appendix. On the Waymo dataset, the within-class variance values and the mean distance norm for RedMotion are 10.68 and 11.24, respectively, resulting in a CDNV of 0.95. On the AV2F dataset, these values are 5.73 and 2.32, yielding a CDNV of 2.46. We hypothesize that the higher CDNV value on AV2F is caused by the longer past motion sequence (i.e., 5 seconds vs. 1.1 seconds on Waymo), allowing for a greater range of potential movements.

Modifying hidden states of motion transformers at inference

Building on the insight that hidden states are likely collapsed toward our interpretable features, we fit control vectors using opposing features. These control vectors allow for modifying motion forecasts at inference. Specifically, we build pairs of opposing features for the AV2F and the Waymo dataset. Then, we fit sets of control vectors (\mathbf{V}_i in Figure 1) as described in the control vectors section. At inference, we add the control vectors generated for the last temporal index (i = -1) to all embeddings (i \in \{0, \ldots, 49\} for AV2F, i \in \{0, \ldots, 10\} for Waymo).

Qualitative results

Figure 5 shows a qualitative example from the AV2F dataset, where we modify hidden states using our control vector for acceleration scaled with different temperatures \tau. Subfigure a shows the default (i.e., non-controlled) top-1 (i.e., most likely) motion forecast. In subfigures b and c, we apply our acceleration control vector with \tau=-20 and \tau=100 to enforce a strong deceleration and a moderate acceleration, respectively.

(a) Default motion forecast

(b) Enforced strong deceleration

(c) Enforced acceleration

Figure 5: Modifying hidden states to control a vehicle at an intersection. We add our acceleration control vector scaled with \tau=-20 and \tau=100 to enforce a strong deceleration and a moderate acceleration. The focal agent is highlighted in orange, dynamic agents are blue, and static agents are grey. Lanes are black lines and road markings are white lines.

Figure 6 shows the focal frames for our acceleration control vector across all \tau values from -100 to 100. Use the slider to sweep the full range; \tau=0 is the default forecast, negative values decelerate, and positive values accelerate.

Control strength (\tau):

Selected: 0 (\tau=0)

Default motion forecast (\tau=0)

Figure 6: Modifying hidden states across the full \tau sweep. The slider steps through every focal frame from \tau=-100 to \tau=100; negative values decelerate, while positive values accelerate while respecting the scene context.

In the appendix, we include an example of our direction control vector. Overall, these qualitative results support the finding that the hidden states of motion sequences are arranged with respect to our discrete sets of motion features.

Similarity-based comparison of control vectors

In this section, we evaluate how control vectors obtained using SAEs differ from those derived via plain PCA. For comparison, we train SAEs with varying sparse intermediate dimensions: 512, 256, 128, 64, 32, and 16. For each control vector, we calculate its pairwise angles with the control vectors for controlling other features. Table 1 presents the angular distances between control vectors of speed, acceleration, direction, and agent type generated with plain PCA and our SAE with a sparse intermediate dimension 128. As expected, the similarity between speed and acceleration, speed and agent type, and acceleration and agent type is notably high, while the similarity involving direction and other vectors is significantly lower. This result aligns with expectations, as positive speed and acceleration controls lead to faster motion, and our agent type control vector represents transition between agent types from pedestrian to vehicle, which is associated with faster motion, as well. Angular-distance results for the remaining SAE dimensions are in the appendix. The similarity of the control vectors generated using the SAE with an intermediate dimension of 128 is the highest.

Plain PCA & Plain PCA (°)	speed	acceleration	direction	agent
speed	0.0	11.5	122.6	10.9
acceleration		0.0	126.8	6.8
direction			0.0	128.7
agent				0.0

SAE & SAE (°)	speed	acceleration	direction	agent
speed	0.0	9.5	120.6	7.8
acceleration		0.0	122.9	7.0
direction			0.0	125.8
agent				0.0

Table 1: Comparison of control vectors, with angles measured in degrees.

Quantitative evaluation of SAEs for optimizing control vectors

We empirically analyze the temporal-causal relationship between modifications on hidden states of past motion and motion forecasts. Specifically, we measure the linearity of relative speed changes in forecasts when scaling our speed control vectors. We use the Pearson correlation coefficient, the coefficient of determination (R²), and the straightness index (S-idx) as linearity measures. Given the large range of scenarios in the Waymo dataset, we focus on relative speed changes within a range of \pm 50\% (see the appendix). Higher linearity implies improved controllability.

We compute linearity measures for control vectors optimized using regular SAEs with varying sparse intermediate dimensions. We achieve the highest scores using the SAE with a dimension of 128 (see Table 2). Therefore, we use this dimension in the rest of our evaluations.

Autoencoder	Pearson	R²	S-idx
SAE-512	0.990	0.974	0.984
SAE-256	0.990	0.974	0.985
SAE-128	0.993	0.984	0.988
SAE-64	0.991	0.976	0.985
SAE-32	0.990	0.959	0.985
SAE-16	0.982	0.770	0.958

Table 2: Scaling sparse autoencoders.

In the following, we evaluate autoencoders with different activation functions and layer types. Following , we use JumpReLU with a threshold \theta = 0.001 and regular ReLU activation functions. Moreover, we evaluate regular SAEs with fully-connected layers, with MLPMixer layers (Sparse MLPMixer), and with convolutional layers (ConvSAE). For Sparse MLPMixer and ConvSAE, we use large patch and kernel sizes to approximate the global receptive fields of fully-connected hidden units in regular SAEs. Furthermore, we evaluate a consistent Koopman autoencoder (KoopmanAE) to include a method that models temporal dynamics between embeddings (see the appendix).

Table 3 presents linearity measures for different control vectors derived from both plain PCA pooling and SAE methods. Overall, the regular SAEs achieve the highest Pearson and R² scores. JumpReLU activation functions improve the R² scores marginally compared to ReLU activation functions. The SAE version of does not improve the linearity scores. We hypothesize that this is due to reduced decoding flexibility since they transpose the encoder weights instead of learning the decoder weights (i.e., \mathbf{W}_\text{dec} = \mathbf{W}_\text{enc}^\top).

Autoencoder	Activation	Pooling	Patch/kernel size	Pearson	R²	S-idx
--	--	PCA	--	0.988	0.969	0.981
SAE (Bricken 2023)	ReLU	PCA	--	0.993	0.984	0.988
SAE (Rajamanoharan 2024)	JumpReLU	PCA	--	0.993	0.986	0.988
SAE (Cunningham 2023)	ReLU	PCA	--	0.987	0.971	0.980
Sparse MLPMixer	ReLU	PCA	64	0.992	0.980	0.986
Sparse MLPMixer	JumpReLU	PCA	64	0.992	0.981	0.986
Sparse MLPMixer	ReLU	PCA	32	0.990	0.978	0.985
Sparse MLPMixer	JumpReLU	PCA	32	0.991	0.980	0.986
ConvSAE	ReLU	PCA	64	0.986	0.383	0.991
ConvSAE	JumpReLU	PCA	64	0.987	0.861	0.978
ConvSAE	ReLU	PCA	32	0.988	0.622	0.986
ConvSAE	JumpReLU	PCA	32	0.989	0.623	0.986
KoopmanAE (Azencot 2020)	tanh	PCA	--	0.991	-0.057	1.000

Table 3: Linearity measures for optimized control vectors: Pearson correlation coefficient, coefficient of determination (R²), and straightness index (S-idx).

The ConvSAE with a kernel size k = 64 and the KoopmanAE achieve the highest straightness index, yet the lowest R² scores. As shown in Figure 7 and in the appendix, the range of temperatures \tau is much higher for this ConvSAE and significantly lower for the KoopmanAE than for e.g. the regular SAE. This lowers the R² score but does not affect the straightness index. For the ConvSAE, we hypothesize that this is due to strong activation shrinkage . Therefore, the JumpReLU configuration of this SAE-type leads to a significantly smaller \tau range (see the appendix), which in turn leads to higher R² scores (see Table 3). For the KoopmanAE, the opposite is likely, since activation shrinkage is caused by sparsity terms, which are not included in the loss function of . Notably, activation steering with our SAE-based control vector has an almost 1-to-1 ratio between \tau and relative speed changes (i.e., \tau = -50 corresponds to roughly -50\%). This improves R² scores and enables an intuitive interface. Furthermore, improved controllability with SAEs indicates that sparse intermediate representations capture more distinct features.

Figure 7: Calibration curves of plain PCA-based speed control vectors and control vectors optimized using SAEs for relative speed changes in forecasts of \pm 50\%.

Relation of probing accuracy to linearity measures for control vectors

We train a RedMotion model on the AV2F dataset using the same trajectory lengths as in the Waymo dataset (1.1 s past and 8 s future), while leaving all other hyperparameters as described in the appendix. Table 4 shows the probing accuracy and linearity measures of a speed control vector for this model (see the appendix for the calibration curve). Compared with a model trained on the Waymo dataset, the AV2F model achieves both a lower probing accuracy and significantly lower linearity measures. These results support our argument that latent space regularities with separable features are necessary to fit precise control vectors.

Dataset	Probing accuracy	Pearson	R²	S-idx
AV2F	0.753	0.877	0.275	0.891
Waymo	0.945	0.988	0.969	0.981

Table 4: Higher probing accuracy enables higher linearity measures. We train RedMotion models on the Waymo and AV2F datasets using the same trajectory lengths. We report the probing accuracies for speed classes and the linearity measures for the corresponding PCA-based control vectors.

In the appendix, we present an ablation study analyzing our method's sensitivity to hidden states from different modules (see the appendix) and to varying speed thresholds (see the appendix). Our method performs best with a sparse intermediate dimension of 128 and hidden states from module m=2; and is more sensitive to low than to high speed thresholds.

Zero-shot generalization with control vectors

Domain shifts between training and test data significantly degrade the performance of many learning algorithms. Zero-shot generalization methods compensate for such domain shifts without further training or fine-tuning . In motion forecasting, common domain shifts are more or less aggressive driving styles that result in higher or lower future speeds, respectively. We simulate this domain shift by reducing the future speed in the Waymo validation split by approximately 50%. Specifically, we take the first half of future waypoints and linearly upsample this sequence to the original length.

Table 5 shows the results of a RedMotion model trained on the regular training split on this validation split with domain shift. We provide an overview of the used motion forecasting metrics in the appendix. Without the use of our control vectors, high distance-based errors, miss, and overlap rates are obtained. Using the calibration curve in Figure 7, we compensate for this domain shift by applying our SAE-128 control vector with a temperature \tau = -50. This significantly reduces the distance-based errors, the overlap, and the miss rates without further training. In addition, we show the results of applying our control vector with a temperature of \tau = -30 and \tau = -70, which improves all scores over the baseline as well.

Control vector	Temperature \tau	minADE \downarrow	Brier minADE \downarrow	minFDE \downarrow	Brier minFDE \downarrow	Overlap rate \downarrow	Miss rate \downarrow
None		3.271	6.547	4.617	8.933	0.220	0.580
SAE-128	-30	1.685	4.838	2.870	8.429	0.179	0.224
SAE-128	-50	1.174	2.759	1.798	4.329	0.174	0.236
SAE-128	-70	1.808	3.576	2.035	3.676	0.189	0.302

Table 5: Zero-shot generalization to a Waymo dataset version with reduced future speeds. Best scores are bold, second best are underlined.

---

Conclusion

In this work, we take a step toward mechanistic interpretability and controllability of motion transformers. We analyze "words in motion" by examining the representations associated with quantized motion features. Specifically, we show that neural collapse toward interpretable classes of features occurs in recent motion transformers. The high degree of neural collapse indicates a well-separated latent space, that enables to fit precise control vectors to opposing features and modify predictions at inference. We further refine this approach by optimizing our control vectors using sparse autoencoding, resulting in higher linearity. Finally, we compensate for domain shift and enable zero-shot generalization to unseen dataset characteristics. Our findings highlight the effectiveness of sparse dictionary learning and the use of SAEs for improving interpretability.

We assumed a flat latent space and relied on vector arithmetic. We leave a detailed investigation of how SAE parameterization might help address potential latent space curvature for future work. Furthermore, we have empirically shown a connection between neural collapse and the structure of the latent space for using control vectors, although our analysis remained limited to probing accuracy and class-distance normalized variance. Our findings enable new applications in robotics and self-driving. We identify safety validation in latent space as a promising direction, particularly for end-to-end driving. Using control vectors to modify internal representations and adjust trajectories via instruction-based inputs is also a valuable application. Finally, future work can explore the use of other embedding methods (e.g., ), as well as incorporate features from other modalities by capturing both static and dynamic scene elements.