This episode features Giulia Solinas (linkedin) (the Al a la Carte Newsletter), an AI and Data Science consultant at CROZ, a Croatian company providing business and technology consulting services. Julia discusses her work at CROZ, focusing on programming solutions that enable LLMs to work with business requests. She also talks about her transition from academia to business, highlighting the bridge between the two and how her academic experience has helped in her current consulting role. The main topic of the episode is Julia's presentation "Unlocking the Power of AI Assistants" at the AI Weekend 2024 Media Festival (part of the Rovinj Conference Weekend Media Festival 2024 ). Julia summarizes the presentation, emphasizing the challenges and considerations of implementing AI assistants in production for organizations. Key discussion points include: Topicality of AI Assistants: The widespread belief that AI will impact businesses and the potential for AI assistants like Copilot to significantly increase productivity. Types of AI Assistants: The difference between general AI for everyone (like Copilot) and customized AI assistants tailored for specific company needs. Challenges of Implementation: Identifying users and their needs. Integrating the AI assistant with existing infrastructure (legacy and non-legacy systems). Ensuring scalability to other departments and use cases. Addressing governance and compliance, particularly GDPR and the EU AI Act. These considerations include data anonymization, masking, location, and usage. An Example Use Case: An AI assistant developed for a large bank's HR department to handle employee requests and save HR personnel time. Looking Ahead to 2025: The prediction that 2025 will be the year of agentic AI. Julia anticipates the increasing complexity of AI systems, with multiple models interacting and integrating with each other. AI Literacy in Organizations: Julia stresses the need for AI literacy across all levels of an organization, from employees to C-level decision-makers, suggesting training and leveraging online resources. This includes fostering trust in AI, facilitating familiarity with AI tools and resources, reducing fear of job displacement, and empowering informed decision-making about AI use cases. Ethical Considerations: The importance of understanding ethical issues related to AI, something that is often overlooked but crucial to consider from both technical and user perspectives. Julia's Current Pet Project: A GitHub repository focused on personal training on InstructLab, a methodology co-developed by Red Hat and IBM for fine-tuning smaller LLMs. This project stems from Julia's enthusiasm for open-source models and their potential for cost-effective and efficient AI solutions in production. Key Takeaways: Implementing AI assistants is a complex process requiring careful planning and consideration of various factors. AI literacy is essential for organizations to maximize the benefits of AI and address potential challenges. Ethical considerations are crucial in AI development and deployment. Open-source models and fine-tuning offer promising avenues for cost-effective and efficient AI solutions.