Advancing Our LLM Expertise: An Intensive Workshop with John Berryman

Written by | May 6, 2025

At AgilityFeat and WebRTC.ventures, delivering advanced Large Language Model (LLM) solutions has quickly become a key growth area for our business. To maintain our edge in this rapidly evolving field, our expert team is dedicated to continuous learning and applying the latest best practices for smarter, faster, and more reliable AI applications.

As part of this commitment, we organized and hosted an intensive, in-person workshop at our Panama office, led by John Berryman—an early engineer on GitHub Copilot and co-author of Prompt Engineering for LLMs: The Art and Science of Building Large Language Model-Based Applications. We brought in team members from across Latin America and the US to participate, deepening our collective expertise in LLM implementation and reinforcing our dedication to staying at the forefront of AI innovation. (You can learn more about John’s work at arcturus-labs.com.)

This investment in expertise strengthens our capabilities across both companies—at WebRTC.ventures, where we focus on real-time communication solutions, and at AgilityFeat, where we help businesses scale with LatAm technical talent and AI implementations.

Why Mastering LLM Implementation Matters

LLMs—and Generative AI as a whole—have incredible potential, but turning that into reliable, scalable applications requires more than just calling an API. It demands a deep understanding of LLM architecture, prompt design, retrieval strategies, observability, and more.

Our Panama workshop was a focused effort to build and refine this expertise across our diverse team, from seasoned software developers to UX/UI designers to QA testers.

Hear It From Our Team

Watch the video below to hear directly from our team members about their experiences during the workshop and how it’s inspiring their approach to LLM implementation.

Core Topics That Underpin Successful LLM Implementations

The course covered essential areas that underpin successful LLM implementations, including:

  • Fundamentals of LLM Architecture: Understanding model behavior and system design
  • Advanced Prompt Engineering: Techniques to guide models toward precise, relevant outputs
  • Retrieval Augmented Generation (RAG): Integrating external knowledge bases to improve accuracy and context
  • Agentic AI Frameworks: Building multi-step, autonomous AI workflows
  • Fine-Tuning Strategies: When and how to customize models effectively
  • Evaluation and Testing: Rigorous methods to validate model outputs and ensure quality
  • Observability and Telemetry: Monitoring LLMs in production to detect issues like hallucinations
  • Debugging Complex Behaviors: Practical approaches to troubleshoot and refine AI responses
  • Future Trends in LLMs: Preparing for upcoming advancements and capabilities

Translating Learning into Client Solutions

The insights gained are already enhancing our ability to deliver sophisticated LLM applications that meet client needs, such as:

  • Optimized Performance: Balancing latency and cost for real-time and large-scale deployments
  • Robust Architectures: Designing scalable, maintainable systems tailored to industry-specific requirements
  • Transparent AI: Implementing observability to provide explainable and trustworthy AI behavior
  • Automated Workflows: Leveraging agentic AI for complex, multi-step tasks in regulated environments

Building on Our Workshop Experience 

Our week with John Berryman deepened not just our technical knowledge, but also our strategic approach to building production-ready LLM applications. From advanced prompt engineering to real-time observability, our team is now better equipped to design AI-powered systems that are scalable, reliable, and aligned with your business goals.

Whether you’re exploring LLMs for the first time or looking to optimize an existing implementation, we’re ready to bring this expertise to your next project.

Let’s talk about how we can help you turn AI potential into practical results!

About the author

About the author

Jen Oppenheimer

Since joining our ranks as Chief of Staff in 2020, Jen has played a crucial role in driving communication and alignment across our diverse organization, ensuring that time, information, and decision-making processes are as effective as possible. Additionally, she directs our marketing and oversees much of the day-to-day operations, ensuring that all departments work cohesively towards our common goals.

Recent Blog Posts