Representation Learning & Model Control
Shaping, reading, and steering learned representations: self-supervised pre-training, image tokenizers, probing, and control vectors.
Shaping, reading, and steering learned representations: self-supervised pre-training, image tokenizers, probing, and control vectors.
Understanding and generating scenes and motion: multi-sensor and vision-language benchmarks, plus diffusion and flow-matching models.
Acting under uncertainty: POMDP tree search, bandits, MPC, convex relaxations, and arbitration between rule-based and learned planners.
|
Wavelet Phase Diffusion for Structurally and Semantically Consistent Sim-to-Real Translation
Kaiwen Wang, Frank Bieder, Yinzhe Shen, Carlos Fernandez, Jan-Hendrik Pauls, Ömer Şahin Taş WACV, 2027 Multimodal & Generative Modeling project / arXiv / code / models Unpaired sim-to-real image and video translation combining localized wavelet phase injection for structural consistency with low-frequency randomization for realistic appearance. |
|
Patch-Derived or Register Tokens? Comparing Variable-Length Image Tokenizers
Philipp Schenkel, Tobias Schürmann, Ömer Şahin Taş ECCV Workshops, 2026 Representation Learning & Model Control Multimodal & Generative Modeling arXiv Compares patch-derived and register-based variable-length image tokenizers, showing how representation alignment shapes spatial token locality and downstream generation. |
|
The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset
Richard Schwarzkopf, Fabian Immel, Alexander Blumberg, Jonas Merkert, Nils Alexander Rack, Kaiwen Wang, Fabian Konstantinidis, Julian Truetsch, Carlos Fernandez, Annika Bätz, Kevin Rösch, Marlon Steiner, Willi Poh, Yinzhe Shen, Royden Wagner, Felix Hauser, Dominik Strutz, Jaime Villa, Gleb Stepanov, Holger Caesar, Ömer Şahin Taş, Frank Bieder, Jan-Hendrik Pauls, Christoph Stiller NeurIPS, 2026 Multimodal & Generative Modeling project / dataset / arXiv A synchronized camera, long-range lidar, 4D imaging radar, and 3D HD map dataset with benchmarks for online mapping, depth estimation, novel view synthesis, and end-to-end driving. |
|
Reasoning models do not yet follow their reasoning in autonomous driving: The KITScenes LongTail Dataset
Royden Wagner*, Ömer Şahin Taş*, Jaime Villa, Felix Hauser, Yinzhe Shen, Marlon Steiner, et al. arXiv preprint, 2026 Multimodal & Generative Modeling Representation Learning & Model Control dataset / paper / benchmark A multimodal, multilingual dataset and benchmark to evaluate in-context learning and chain-of-thought reasoning in VLMs/VLAs. |
|
Mosaic: An Extensible Framework for Composing Rule-Based and Learned Motion Planners
Nick Le Large, Marlon Steiner, Lingguang Wang, Willi Poh, Jan-Hendrik Pauls, Ömer Şahin Taş, Christoph Stiller IROS, 2026 Planning & Decision-Making project / arXiv / slides / code / poster Composes rule-based and learned motion planners via arbitration graphs that decouple trajectory verification and selection from individual planner generation. |
|
RetroMotion: Retrocausal Motion Forecasting Models are Instructable
Royden Wagner, Ömer Şahin Taş, Felix Hauser, Marlon Steiner, Dominik Strutz, Abhishek Vivekanandan, Jaime Villa, Yinzhe Shen, Carlos Fernandez, Christoph Stiller CVPR Workshops, 2026 — Awarded an honorable mention in the 2025 Waymo Interaction Prediction Challenge Multimodal & Generative Modeling arXiv / code / poster A model generating joint trajectories by re-encoding marginals. |
|
FlowDrive: moderated flow matching with data balancing for trajectory planning
Lingguang Wang, Ömer Şahin Taş, Marlon Steiner, Christoph Stiller arXiv preprint, 2025 Multimodal & Generative Modeling Planning & Decision-Making arXiv / code A conditional flow-matching planner that generates trajectories via learned rectified flow. |
|
Words in Motion: Extracting Interpretable Control Vectors for Motion Transformers
Ömer Şahin Taş*, Royden Wagner* ICLR, 2025 Representation Learning & Model Control project / arXiv / code / OpenReview / video / poster We find that neural collapse and concept alignment enable learning interpretable control vectors, which we refine with sparse autoencoders. |
|
Optimal Control of Hybrid Systems via Measure Relaxations
Etienne Buehrle, Ömer Şahin Taş, Christoph Stiller CDC, 2025 Planning & Decision-Making arXiv / code A convex occupation-measure relaxation on graphs of convex sets for globally bounding and planning hybrid trajectories under temporal logic specifications. |
|
Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Driving
Yinzhe Shen, Ömer Şahin Taş, Kaiwen Wang, Royden Wagner, Christoph Stiller TMLR, 2025 Representation Learning & Model Control arXiv / code / OpenReview / video / poster An end-to-end autonomous driving approach that separates semantic and motion learning into parallel decoders to mitigate negative transfer. |
|
JointMotion: Joint Self-supervision for Joint Motion Prediction
Royden Wagner*, Ömer Şahin Taş*, Marvin Klemp, Carlos Fernandez CoRL, 2024 Representation Learning & Model Control arXiv / code / OpenReview / video / poster JointMotion connects scene-level motion and environment embeddings via a non-contrastive alignment objective, then applies masked polyline modeling to unify global context and instance-level representation. |
|
RedMotion: Motion Prediction via Redundancy Reduction
Royden Wagner, Ömer Şahin Taş, Marvin Klemp, Carlos Fernandez, Christoph Stiller TMLR, 2024 Representation Learning & Model Control arXiv / code / OpenReview RedMotion fuses local road features into a global embedding via an internal decoder, then applies self-supervised redundancy reduction across augmented views to unify local and global road representations. |
|
Decision-theoretic MPC: Motion Planning with Weighted Maneuver Preferences Under Uncertainty
Ömer Şahin Taş, Philipp Heinrich Brusius, Christoph Stiller arXiv preprint, 2023 Planning & Decision-Making arXiv / code / video Continuous motion planning that weights candidate maneuver classes under perception and intention uncertainty while enforcing chance-constrained emergency safety (subsumes our IV 2018, ITSC 2018, and RSSW 2020 papers). |
|
Motion Planning for Autonomous Vehicles in Partially Observable Environments
Ömer Şahin Taş Ph.D. thesis, 2022 Planning & Decision-Making pdf / code / video My dissertation formulates motion planning under uncertainty and solves it using nonlinear MPC and Monte Carlo tree search. |
|
Efficient Sampling in POMDPs with Lipschitz Bandits for Motion Planning in Continuous Spaces
Ömer Şahin Taş, Felix Hauser, Martin Lauer IEEE Intelligent Vehicles Symposium (IV), 2021 Planning & Decision-Making arXiv / slides / video Lipschitz-based bandits exploit the continuity of Q-values. We harness this smoothness in sampling-based POMDPs to accelerate search in large discrete action sets. Furthermore, we refine the bandits with variance-based updates to handle local discontinuities. |
|
Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains
Johannes Fischer*, Ömer Şahin Taş* ICML, 2020 Planning & Decision-Making paper / supplemental / code / video IPFT solves continuous ρPOMDPs by combining MCTS with particle-based belief updates. It integrates an information-theoretic shaping term to accelerate search in problems where gathering information is crucial for optimal decision making. |
* equal contribution. See also Google Scholar, or the complete chronological list (2012-2027) on Publications.