elb.jpg
Ellis Brown

ellis.brown at nyu dot edu

I am a CS PhD Student at NYU Courant advised by Profs. Saining Xie and Rob Fergus. My research is supported by the NDSEG Fellowship. I am currently interning with Ross Girshick in the PRIOR team at the Allen Institute for AI (Ai2).

I recently graduated from a Master’s at Carnegie Mellon where I was advised by Profs. Deepak Pathak and Alyosha Efros. Before that, I was a founding research engineer at BlackRock AI Labs, where I was fortunate to work with Profs. Mykel Kochenderfer, Stephen Boyd, and Trevor Hastie on applied research & finance. During my undergrad at Vanderbilt, I majored in CS & Math and did research in CogSci & Vision with Prof. Maithilee Kunda. I’m originally from St. Louis, MO.

I am a proud member of the Osage Nation and have recently enjoyed engaging with the Native research community at AISES.

If you haven’t made time for a regular checkin with a doctor recently, please do! Even if you feel perfectly healthy.

news

Sep., 2024 Cambrian was accepted to NeurIPS 2024 as an oral presentation 🪼🎉
Mar., 2024 Thrilled to have been awarded the NDSEG Fellowship to support my PhD research at NYU!
Feb., 2024 I will be joining AllenAI (AI2) as a Resesarch Intern this summer in Seattle, working with Ross Girshick!
Aug., 2023 Excited to be starting my PhD at NYU advised by Profs. Saining Xie and Rob Fergus 🎉🗽
May., 2023 Defended my Master’s thesis and graduated from CMU!
Apr., 2023 Our paper Internet Explorer was accepted to ICML 2023!! See you in Hawaii 🌺🏝😎
Jan., 2022 Joined Prof. Deepak Pathak’s group in CMU’s Robotics Institute, and will focus on self-supervised learning and curiosity-driven exploration in computer vision.
Aug., 2021 Awarded the Intel Growing The Legacy Scholarship and accepted into the Lighting the Pathway to Faculty Careers program through AISES. Accepted into Google Research’s Computer Science Mentorship Program.
Jul., 2021 Left BlackRock AI Labs to pursue a Master’s degree in Computer Science at CMU. Bittersweet!

selected research (all)

I’m interested in self-supervised learning, representation learning, curiosity-based exploration, and leveraging internet-scale models and data. I am keen to draw inspiration from intelligence in humans and nature—especially as a goal-post rather than a blueprint. My long-term goal is to develop intelligent agents that can generalize and continually adapt as robustly and efficiently as humans do, allowing them to be safely deployed in the real world.

publications

2024

  1. cambrian.png
    Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
    Shengbang Tong*Ellis Brown*Penghao Wu* ,  Sanghyun Woo ,  Manoj Middepogu ,  Sai Charitha Akula ,  Jihan Yang ,  Shusheng Yang ,  Adithya Iyer ,  Xichen Pan ,  Austin Wang ,  Rob FergusYann LeCun ,  and  Saining Xie
    In NeurIPS , 2024
    🎖️ Oral Presentation (1.8%)
  2. virl_2.png
    V-IRL: Grounding Virtual Intelligence in Real Life
    Jihan Yang ,  Runyu Ding ,  Ellis Brown ,  Xiaojuan Qi ,  and  Saining Xie
    In ECCV , 2024

2023

  1. diffusion_classifier.jpg
    Your Diffusion Model is Secretly a Zero-Shot Classifier
    Alexander C. LiMihir PrabhudesaiShivam DuggalEllis Brown ,  and  Deepak Pathak
    In ICCV , 2023
  2. internet_explorer.gif
    Internet Explorer: Targeted Representation Learning on the Open Web
    Alexander C. Li*Ellis Brown*Alexei A. Efros ,  and  Deepak Pathak
    In ICML , 2023

code