Koh, Jing Yu

Scaling Autoregressive Models for Content-Rich Text-to-Image Generation

Yu, J., Xu, Y., Koh, J. Y., Luong, T., Baid, G., Wang, Z., Vasudevan, V., Ku, A., Yang, Y., Ayan, B. K., et al. (2022) · arXiv

Generative Adversarial Networks and Image SynthesisMultimodal Machine Learning Applications

Cited by 371

OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web

Kapoor, R., Butala, Y. P., Russak, M., Koh, J. Y., Kamble, K., Alshikh, W., & Salakhutdinov, R. (2024) · arXiv

Multimodal Machine Learning ApplicationsSpeech and dialogue systems

Cited by 93

VisualWebArena: evaluating multimodal agents on realistic visual web tasks

Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Multimodal Machine Learning ApplicationsSpeech and dialogue systems

Cited by 50

A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning

Kamath, A., Anderson, P., Wang, S., Koh, J. Y., Ku, A., Waters, A., Yang, Y., Baldridge, J., & Parekh, Z. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Multimodal Machine Learning ApplicationsReinforcement Learning in Robotics

Cited by 39

Grounding Language Models to Images for Multimodal Inputs and Outputs

Koh, J. Y., Salakhutdinov, R., & Fried, D. (2023) · arXiv

Multimodal Machine Learning ApplicationsTopic Modeling

Cited by 38

OmniACT: a dataset and benchmark for enabling multimodal generalist autonomous agents for desktop and web

Kapoor, R., Butala, Y. P., Russak, M., Koh, J. Y., Kamble, K., AlShikh, W., & Salakhutdinov, R. (2025) · Lecture Notes in Computer Science

Multimodal Machine Learning ApplicationsSpeech and dialogue systems

Cited by 12

Tree Search for Language Model Agents

Koh, J. Y., McAleer, S., Fried, D., & Salakhutdinov, R. (2024) · arXiv

AI-based Problem Solving and PlanningInformation Retrieval and Search Behavior

Cited by 2

Dissecting Adversarial Robustness of Multimodal LM Agents

Wu, C. H., Shah, R., Koh, J. Y., Salakhutdinov, R., Fried, D., & Raghunathan, A. (2024) · arXiv

Adversarial Robustness in Machine LearningSecurity and Verification in Computing

Cited by 1

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M. C., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · arXiv

Multimodal Machine Learning ApplicationsWeb Data Mining and Analysis

Cited by 1