Ellis Brown

ellis.brown at nyu dot edu

I am a CS PhD Student at NYU Courant advised by Saining Xie and Rob Fergus. I recently graduated from a Master’s at CMU where I was advised by Deepak Pathak and Alyosha Efros. Before that, I was a founding research engineer at BlackRock AI Labs, where I was fortunate to work with Mykel Kochenderfer, Stephen Boyd, and Trevor Hastie on applied research & finance. I earned Bachelors degrees in CS & Math from Vanderbilt University, where I worked on CoCoSci & Vision with Maithilee Kunda. I’m originally from St. Louis, MO.

I am a proud member of the Osage Nation and have recently enjoyed engaging with the Native research community at AISES.

If you haven’t made time for a regular checkin with a doctor recently, please do! Even if you feel perfectly healthy.


May., 2023 Defended my Master’s thesis and graduated from CMU! Excited to start a CS PhD at NYU advised by Saining Xie and Rob Fergus this fall 🎉
Apr., 2023 Our paper Internet Explorer was accepted to ICML 2023!! See you in Hawaii 🌺🏝😎
Jan., 2022 Joined Deepak Pathak’s group in CMU’s Robotics Institute, and will focus on self-supervised learning and curiosity-driven exploration in computer vision.
Aug., 2021 Awarded the Intel Growing The Legacy Scholarship and accepted into the Lighting the Pathway to Faculty Careers program through AISES. Accepted into Google Research’s Computer Science Mentorship Program.
Jul., 2021 Left BlackRock AI Labs to pursue a Master’s degree in Computer Science at CMU. Bittersweet!

selected research (more)

I’m interested in self-supervised learning, representation learning, curiosity-based exploration, and leveraging internet-scale models and data. I am keen to draw inspiration from intelligence in humans and nature—especially as a goal-post rather than a blueprint. My long-term goal is to develop intelligent agents that can generalize and continually adapt as robustly and efficiently as humans do, allowing them to be safely deployed in the real world.



  1. arXiv
    V-IRL: Grounding Virtual Intelligence in Real Life
    Jihan Yang ,  Runyu Ding ,  Ellis Brown ,  Xiaojuan Qi ,  and  Saining Xie
    arXiv preprint arXiv:2402.03310, 2024


  1. diffusion_classifier.jpg
    Your Diffusion Model is Secretly a Zero-Shot Classifier
    Alexander C. LiMihir PrabhudesaiShivam DuggalEllis Brown ,  and  Deepak Pathak
    In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2023
  2. internet_explorer.gif
    Internet Explorer: Targeted Representation Learning on the Open Web
    Alexander C. Li*Ellis Brown*Alexei A. Efros ,  and  Deepak Pathak
    In International Conference on Machine Learning , 2023


  1. 2018_stts.gif
    An Architecture for Spatiotemporal Template-Based Search
    Ellis Brown ,  Soobeen Park ,  Noel Wardord ,  Adriane Seiffert ,  Kazuhiko Kawamura ,  Joseph Lappin ,  and  Maithilee Kunda
    Advances in Cognitive Systems, 2018