Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
Modifying Point Clouds with LLMs
Learn how to build an agent that modifies 3D point clouds using natural language. Discover techniques for LLM spatial reasoning, structured scene graphs, and systematic evaluation for reliable performance.
I’m presenting an agent and natural-language interface that understands and modifies 3D point clouds. Users can ask it to complete various tasks - identifying objects, segmenting regions, even moving elements - and it executes them directly on the point cloud data.
The core challenge is getting an LLM to reason meaningfully over spatial 3D structures, which requires careful prompt engineering, tooling, and a structured scene graph as the agent’s “world model” rather than raw point data. Getting reliable behavior meant running systematic evals: testing the agent across varied phrasings, ambiguous spatial queries, and edge cases like occluded or overlapping objects - then iterating on the tool definitions and system prompt until performance was consistent.
The builder takeaway: treat your data representation as a first-class design decision - the right abstraction layer between the LLM and your domain data is what makes or breaks agent reliability.
Panoriq is a language-driven agent for editing and querying point-clouds.
- LangGraphA low-level orchestration framework for building long-running, stateful, and cyclic multi-agent systems using a graph-based architecture.LangGraph is the specialized, low-level runtime for developing complex AI agents, extending the LangChain ecosystem to handle intricate, stateful workflows. It models the agent's logic as a directed graph: nodes represent actions (LLM calls, tool use), and conditional edges dictate the flow, enabling critical features like cycles (loops) for iterative reasoning. This graph-based approach ensures durable execution, allowing agents to persist through failures and resume operations. Key capabilities include comprehensive memory management via a shared state object and built-in human-in-the-loop functionality (interrupts) for external oversight. This robust framework is trusted by production teams at companies like Klarna and Replit for deploying scalable, resilient agent architectures.
- LangSmithLangSmith is the unified DevOps platform for the LLM application lifecycle: trace, test, evaluate, and monitor your AI agents from prototype to production.LangSmith delivers end-to-end operational intelligence for LLM workflows, integrating observability, evaluation, and deployment into a single platform . Use its detailed tracing to gain sequential visibility into complex reasoning paths, troubleshooting issues like long response times or unexpected errors . The platform enables robust testing by letting you grow benchmarking datasets with real user data, ensuring consistent, trustworthy performance in production . It is framework agnostic, supporting both LangChain-built and custom pipelines .
- NextJSNext.js is the full-stack React framework: build and deploy high-performance, production-ready web applications with integrated server-side rendering (SSR) and static generation (SSG).Next.js, developed by Vercel, is the definitive React framework for the web, enabling developers to build full-stack applications with exceptional speed and scalability. It leverages the latest React features, including Server Components and Server Actions, to deliver a modern architecture. Key optimizations are built-in: flexible rendering strategies (SSR, SSG, Incremental Static Regeneration), automatic code splitting, and Rust-based tooling (like Turbopack) for fast builds. This structure ensures superior performance, better SEO, and a streamlined developer experience for projects of any size.