Ellis Brown

ellis.brown at nyu dot edu

I am an incoming CS PhD Student at NYU Courant advised by Saining Xie and Rob Fergus. I recently graduated from a Master’s at CMU where I was advised by Deepak Pathak and Alyosha Efros. Before that, I was a founding research engineer at BlackRock AI Labs, where I was fortunate to work with Mykel Kochenderfer, Stephen Boyd, and Trevor Hastie on applied research & finance. I earned Bachelors degrees in CS & Math from Vanderbilt University, where I worked on CoCoSci & Vision with Maithilee Kunda. I’m originally from St. Louis, MO.

I am a proud member of the Osage Nation and have recently enjoyed engaging with the Native research community at AISES. Look out for Killer’s of the Flower Moon in theaters fall, 2023!

If you haven’t made time for a regular checkin with a doctor recently, please do! Even if you feel perfectly healthy.


May., 2023 Defended my Master’s thesis and graduated from CMU! Excited to start a CS PhD at NYU advised by Saining Xie and Rob Fergus this fall 🎉
Apr., 2023 Our paper Internet Explorer was accepted to ICML 2023!! See you in Hawaii 🌺🏝😎
Jan., 2022 Joined Deepak Pathak’s group in CMU’s Robotics Institute, and will focus on self-supervised learning and curiosity-driven exploration in computer vision.
Aug., 2021 Awarded the Intel Growing The Legacy Scholarship and accepted into the Lighting the Pathway to Faculty Careers program through AISES. Accepted into Google Research’s Computer Science Mentorship Program.
Jul., 2021 Left BlackRock AI Labs to pursue a Master’s degree in Computer Science at CMU. Bittersweet!

selected research (more)

I’m interested in self-supervised learning, representation learning, curiosity-based exploration, and leveraging internet-scale models and data. I am keen to draw inspiration from intelligence in humans and nature—especially as a goal-post rather than a blueprint. My long-term goal is to develop intelligent agents that can generalize and continually adapt as robustly and efficiently as humans do, allowing them to be safely deployed in the real world.


  1. Your Diffusion Model is Secretly a Zero-Shot Classifier
    arXiv:cs.LG, 2023
  2. Internet Explorer: Targeted Representation Learning on the Open Web
    In International Conference on Machine Learning, 2023
  3. An Architecture for Spatiotemporal Template-Based Search
    Ellis Brown, Soobeen Park, Noel Wardord, Adriane Seiffert, Kazuhiko Kawamura, Joseph Lappin, and Maithilee Kunda
    Advances in Cognitive Systems, 2018


  1. Linearly Constrained Separable Optimization
    Ellis Brown, Nicholas Moehle, and Mykel J. Kochenderfer
    In JuliaCon 2021 JuMP Track, Jul 2021
  2. AISES-19
    Modeling Uncertainty in Bayesian Neural Networks with Dropout: The effect of weight prior and network architecture selection
    Ellis Brown*, Melanie Manko*, and Ethan Matlin*
    In American Indian Science and Engineering Society National Conference, Oct 2019
    🎖️ Third Place, Graduate Student Research Competition
  3. AISES-17
    Computational Cognitive Systems to Model Information Salience
    Ellis Brown, Adriane Seiffert, Noel Warford, Soobeen Park, and Maithilee Kunda
    In American Indian Science and Engineering Society National Conference, Sep 2017