Research

Representation Learning & Model Control

Shaping, reading, and steering learned representations: self-supervised pre-training, image tokenizers, probing, and control vectors.

Multimodal & Generative Modeling

Understanding and generating scenes and motion: multi-sensor and vision-language benchmarks, plus diffusion and flow-matching models.

Planning & Decision-Making

Acting under uncertainty: POMDP tree search, bandits, MPC, convex relaxations, and arbitration between rule-based and learned planners.

Selected Publications

Wavelet Phase Diffusion for Structurally and Semantically Consistent Sim-to-Real Translation
Kaiwen Wang, Frank Bieder, Yinzhe Shen, Carlos Fernandez, Jan-Hendrik Pauls, Ömer Şahin Taş
WACV, 2027 Multimodal & Generative Modeling
project / arXiv / code / models

Unpaired sim-to-real image and video translation combining localized wavelet phase injection for structural consistency with low-frequency randomization for realistic appearance.

Patch-Derived or Register Tokens? Comparing Variable-Length Image Tokenizers
Philipp Schenkel, Tobias Schürmann, Ömer Şahin Taş
ECCV Workshops, 2026 Representation Learning & Model Control Multimodal & Generative Modeling
arXiv

Compares patch-derived and register-based variable-length image tokenizers, showing how representation alignment shapes spatial token locality and downstream generation.

The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset
Richard Schwarzkopf, Fabian Immel, Alexander Blumberg, Jonas Merkert, Nils Alexander Rack, Kaiwen Wang, Fabian Konstantinidis, Julian Truetsch, Carlos Fernandez, Annika Bätz, Kevin Rösch, Marlon Steiner, Willi Poh, Yinzhe Shen, Royden Wagner, Felix Hauser, Dominik Strutz, Jaime Villa, Gleb Stepanov, Holger Caesar, Ömer Şahin Taş, Frank Bieder, Jan-Hendrik Pauls, Christoph Stiller
NeurIPS, 2026 Multimodal & Generative Modeling
project / dataset / arXiv

A synchronized camera, long-range lidar, 4D imaging radar, and 3D HD map dataset with benchmarks for online mapping, depth estimation, novel view synthesis, and end-to-end driving.

Reasoning models do not yet follow their reasoning in autonomous driving: The KITScenes LongTail Dataset
Royden Wagner*, Ömer Şahin Taş*, Jaime Villa, Felix Hauser, Yinzhe Shen, Marlon Steiner, et al.
arXiv preprint, 2026 Multimodal & Generative Modeling Representation Learning & Model Control
dataset / paper / benchmark

A multimodal, multilingual dataset and benchmark to evaluate in-context learning and chain-of-thought reasoning in VLMs/VLAs.

Mosaic: An Extensible Framework for Composing Rule-Based and Learned Motion Planners
Nick Le Large, Marlon Steiner, Lingguang Wang, Willi Poh, Jan-Hendrik Pauls, Ömer Şahin Taş, Christoph Stiller
IROS, 2026 Planning & Decision-Making
project / arXiv / slides / code / poster

Composes rule-based and learned motion planners via arbitration graphs that decouple trajectory verification and selection from individual planner generation.

RetroMotion: Retrocausal Motion Forecasting Models are Instructable
Royden Wagner, Ömer Şahin Taş, Felix Hauser, Marlon Steiner, Dominik Strutz, Abhishek Vivekanandan, Jaime Villa, Yinzhe Shen, Carlos Fernandez, Christoph Stiller
CVPR Workshops, 2026 — Awarded an honorable mention in the 2025 Waymo Interaction Prediction Challenge Multimodal & Generative Modeling
arXiv / code / poster

A model generating joint trajectories by re-encoding marginals.

FlowDrive: moderated flow matching with data balancing for trajectory planning
Lingguang Wang, Ömer Şahin Taş, Marlon Steiner, Christoph Stiller
arXiv preprint, 2025 Multimodal & Generative Modeling Planning & Decision-Making
arXiv / code

A conditional flow-matching planner that generates trajectories via learned rectified flow.

Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers
Ömer Şahin Taş*, Royden Wagner*
ICLR, 2025 Representation Learning & Model Control
project / arXiv / code / OpenReview / video / poster

We find that neural collapse and concept alignment enable learning interpretable control vectors, which we refine with sparse autoencoders.

Optimal Control of Hybrid Systems via Measure Relaxations
Etienne Buehrle, Ömer Şahin Taş, Christoph Stiller
CDC, 2025 Planning & Decision-Making
arXiv / code

A convex occupation-measure relaxation on graphs of convex sets for globally bounding and planning hybrid trajectories under temporal logic specifications.

Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Driving
Yinzhe Shen, Ömer Şahin Taş, Kaiwen Wang, Royden Wagner, Christoph Stiller
TMLR, 2025 Representation Learning & Model Control
arXiv / code / OpenReview / video / poster

An end-to-end autonomous driving approach that separates semantic and motion learning into parallel decoders to mitigate negative transfer.

JointMotion: Joint Self-supervision for Joint Motion Prediction
Royden Wagner*, Ömer Şahin Taş*, Marvin Klemp, Carlos Fernandez
CoRL, 2024 Representation Learning & Model Control
arXiv / code / OpenReview / video / poster

JointMotion connects scene-level motion and environment embeddings via a non-contrastive alignment objective, then applies masked polyline modeling to unify global context and instance-level representation.

RedMotion: Motion Prediction via Redundancy Reduction
Royden Wagner, Ömer Şahin Taş, Marvin Klemp, Carlos Fernandez, Christoph Stiller
TMLR, 2024 Representation Learning & Model Control
arXiv / code / OpenReview

RedMotion fuses local road features into a global embedding via an internal decoder, then applies self-supervised redundancy reduction across augmented views to unify local and global road representations.

Decision-theoretic MPC: Motion Planning with Weighted Maneuver Preferences Under Uncertainty
Ömer Şahin Taş, Philipp Heinrich Brusius, Christoph Stiller
arXiv preprint, 2023 Planning & Decision-Making
arXiv / code / video

Continuous motion planning that weights candidate maneuver classes under perception and intention uncertainty while enforcing chance-constrained emergency safety (subsumes our IV 2018, ITSC 2018, and RSSW 2020 papers).

Motion Planning for Autonomous Vehicles in Partially Observable Environments
Ömer Şahin Taş
Ph.D. thesis, 2022 Planning & Decision-Making
pdf / code / video

My dissertation formulates motion planning under uncertainty and solves it using nonlinear MPC and Monte Carlo tree search.

Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces
Ömer Şahin Taş, Felix Hauser, Martin Lauer
IEEE Intelligent Vehicles Symposium (IV), 2021 Planning & Decision-Making
arXiv / slides / video

Lipschitz-based bandits exploit the continuity of Q-values. We harness this smoothness in sampling-based POMDPs to accelerate search in large discrete action sets. Furthermore, we refine the bandits with variance-based updates to handle local discontinuities.

Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains
Johannes Fischer*, Ömer Şahin Taş*
ICML, 2020 Planning & Decision-Making
paper / supplemental / code / video

IPFT solves continuous ρPOMDPs by combining MCTS with particle-based belief updates. It integrates an information-theoretic shaping term to accelerate search in problems where gathering information is crucial for optimal decision making.

* equal contribution. See also Google Scholar, or the complete chronological list (2012-2027) on Publications.