3 Takeaways From Building AI Agents in Freight Tech
I spent two years building AI agents at a freight tech startup. Production agents, messy business process, millions of conversations. Here's what stuck.
scroll
Lower Yosemite Falls (May 2026)
Yosemite Valley (May 2026)
Valley Floor, Yosemite (May 2026)
Marin Headlands (Apr 2026)
Palace of Fine Arts (Oct 2025)
Golden Gate (Jan 2025)
Ocean Beach (Feb 2022)
Chestnut Street (Feb 2025)
Marin Headlands (Jul 2025)
Tank Hill (Jun 2020)
Big Sur (Dec 2016)
Outskirts of Seattle (Mar 2025)
Marina District (May 2025)
Mallorca Cap de Formentor (Jul 2025)
Mendocino (Apr 2025)
Lower Yosemite Falls (May 2026)
Yosemite Valley (May 2026)
Valley Floor, Yosemite (May 2026)
Marin Headlands (Apr 2026)
Palace of Fine Arts (Oct 2025)
Golden Gate (Jan 2025)
Ocean Beach (Feb 2022)
Chestnut Street (Feb 2025)
Marin Headlands (Jul 2025)
Tank Hill (Jun 2020)
Big Sur (Dec 2016)
Outskirts of Seattle (Mar 2025)
Marina District (May 2025)
Mallorca Cap de Formentor (Jul 2025)
Mendocino (Apr 2025)
Lower Yosemite Falls (May 2026)
Yosemite Valley (May 2026)
Valley Floor, Yosemite (May 2026)
Marin Headlands (Apr 2026)
Palace of Fine Arts (Oct 2025)
Golden Gate (Jan 2025)
Ocean Beach (Feb 2022)
Chestnut Street (Feb 2025)
Marin Headlands (Jul 2025)
Tank Hill (Jun 2020)
Big Sur (Dec 2016)
Outskirts of Seattle (Mar 2025)
Marina District (May 2025)
Mallorca Cap de Formentor (Jul 2025)
Mendocino (Apr 2025)
Evaluating LLMs with Chat Templates
LLMs are typically finetuned on chat-templated datasets, which convert conversations into strings ready for text generation. Matching this format is standard at prediction time, as deviations are often noted to hurt performance. Here, we test this theory using an instruction-following benchmark and the best open-source LLMs available at this time.
Work
- Company
- Replit
- Role
- Member of Technical Staff
- Date
- Company
- Parade.ai
- Role
- Lead AI Engineer
- Date
- Company
- West Monroe Partners
- Role
- Senior Data Scientist
- Date
- Company
- Understory.ai
- Role
- ML Engineer
- Date
Education
- Institution
- University of California, Berkeley
- Degree
- MS in Applied Data Science
- Institution
- University of Pennsylvania
- Degree
- BA in Earth Science & Math