As enterprises roll out production applications using AI model inferencing, they are finding that they are limited by the amount of memory that can be addressed by a GPU. This episode of Utilizing Tech features Steen Graham, founder of Metrum AI, discussing modern RAG and agentic AI applications with Ace Stryker and Stephen Foskett. Achieving the promise of AI requires access to data, and the memory required to deliver this is increasingly a focus of AI infrastructure providers. Technologies like DiskANN allow workloads to be offloaded to solid-state drives rather than system memory, and this surprisingly results in better performance. Another idea is to offload a large AI model to SSDs and deploy larger models on lower-cost GPUs, and this is showing a great deal of promise. Agentic AI in particular can be run in an asynchronous model, enabling them to take advantage of lower-spec hardware including older GPUs and accelerators, reduced RAM capacity and performance, and even all-CPU infrastructure. All of this suggests that AI can be run with less financial and power resources than generally assumed.

Guest:

Steen Graham is the Founder and CEO of Metrum AI. You can connect with Steen on LinkedIn and learn more about Metrum AI on their website.

Guest Host:

Ace Stryker is the Director of Product Marketing at Solidigm. You can connect with Ace on LinkedIn and learn more about Solidigm and their AI efforts on their dedicated AI landing page or watch their AI Field Day presentations from the recent event.

Hosts:

⁠⁠⁠⁠Stephen Foskett⁠⁠⁠⁠, President of the Tech Field Day Business Unit and Organizer of the ⁠⁠⁠⁠Tech Field Day Event Series⁠⁠⁠⁠

⁠⁠⁠⁠Jeniece Wnorowski⁠⁠⁠⁠, Head of Influencer Marketing at ⁠⁠⁠⁠Solidigm⁠⁠⁠⁠

⁠⁠⁠⁠Scott Shadley⁠⁠⁠⁠, Leadership Narrative Director and Evangelist at ⁠⁠⁠⁠Solidigm⁠⁠⁠⁠

Follow Tech Field Day ⁠⁠⁠⁠on LinkedIn⁠⁠⁠⁠, ⁠⁠⁠⁠on X/Twitter,⁠⁠⁠⁠ ⁠⁠⁠⁠on Bluesky⁠⁠⁠⁠, and ⁠⁠⁠⁠on Mastodon⁠⁠⁠⁠. Visit the ⁠⁠⁠⁠Tech Field Day website⁠⁠⁠⁠ for more information on upcoming events.

For more episodes of Utilizing Tech, head to ⁠⁠⁠⁠the dedicated website⁠⁠⁠⁠ and follow the show ⁠⁠⁠⁠on X/Twitter⁠⁠⁠⁠, ⁠⁠⁠⁠on Bluesky⁠⁠⁠⁠, and ⁠⁠⁠⁠on Mastodon⁠⁠⁠⁠.