AI_dev: Open Source GenAI & ML Summit Europe 2025

Unlock the secrets to scaling production RAG! This case study dives into our successful deployment of a low-latency RAG chatbot built on Milvus, handling over 10 million vector records. Discover how we managed a 12-18 node Kubernetes cluster powered by cost-effective AWS Graviton CPUs and aggressively used Spot Instances to stay within budget while achieving high accuracy.

Trallie is an open-source framework backed by the NGI Search Consortium that leverages the power of large language models (LLMs) to reimagine information extraction (IE) with or without user guidelines. Instead of relying on labelled examples or copious amounts of training on your data collection, Trallie requires very few or no representative examples of the data to convert into a structured format. It has three core objectives:

Generative AI workloads have introduced a new class of infrastructure challenges—massive model sizes, unpredictable bursty inference, tight latency budgets, and soaring GPU costs. While teams invest heavily in model compression and distillation, they often overlook the other half of the equation: the ML pipeline.

In our talk, we present a practical use-case within the 8RA (https://www.8ra.com/) program, the IPCEI covering Cloud and Infrastructure Services (CIS), built on top open-source software. Our use-case exploits both ML and GenAI to support scenarios within Multi-Provider Cloud Edge Continuum, cloud infrastructure federation, automated service request and allocation of applications to edge sites properly selected. We design an architecture whose core is a set of AI agents, which collaborate to accomplish different tasks, ranging from application deployment file generation up to site prediction for application deployment, according to service requests taken as input by users in natural language. Specifically, our architecture currently includes several agents: LANE (Language-to-Action Neural Engine), responsible for processing user requests expressed in natural Language; Placement, providing the optimal edge site recommendation for each application, ensuring maximum energy efficiency once deployed. It leverages an AI-driven approach to predict energy consumption and optimize resource allocation, improving the sustainability of cloud services.

This issue has plagued Kubernetes for nearly 8 years: K8s issue #52757. The challenge of flexibly sharing GPUs across multiple containers is particularly prominent in AI scenarios, where inference tasks are typically short-lived. As a result, resource utilization becomes a critical concern.

Red Hat integrates and supports both kServe and vLLM in its MLOps Platform, OpenShift AI. In addition, Red Hat's engineers actively contribute on kServe and vLLM upstream projects everyday.

AI is evolving beyond the cloud—and into the browser. This talk explores how recent advances in Web AI are making it possible to run powerful machine learning models for language, vision, and personalization directly on the client side. Leveraging technologies like WebGPU and WebAssembly, frameworks such as ONNX.js and WebLLM are enabling real-time, private, and serverless inference right where users are. We'll demonstrate how this shift improves latency, enhances user privacy, and removes infrastructure barriers—bringing scalable, intelligent experiences to the open web. This talk will promotes open-source, privacy-first AI that reduces cloud dependency, lowers entry barriers, and encourages ethical, accessible AI development across devices.

In my session, I will discuss the topic of emotion estimation from facial expressions using AI and computer vision.

This lightning talk introduces a practical and visionary approach to building scalable agentic AI systems using Google’s new ADK SDK and deploying them on Kubernetes. Attendees will learn how to design autonomous AI agents with dynamic workflows, hybrid memory (BM25 + vector), and real-time decision-making capabilities. Through a live code walkthrough, I’ll demonstrate how to rapidly build a modular agent using ADK and scale it using cloud-native infrastructure patterns. This talk is ideal for AI engineers, DevOps professionals, and open source contributors looking to bring production-grade agentic intelligence to their systems.

Discover how Back Market uses a Python GenAI application to automate the identification of Personally Identifiable Information (PII) in GitHub.

In the rapidly evolving landscape of artificial intelligence, deploying AI models consistently and securely remains a significant challenge. Ramalama addresses this by merging the robustness of containerization with the intricacies of AI workloads. By default, Ramalama runs AI models in isolated environments, enhancing security and ensuring consistent performance across various platforms. This session will delve into Ramalama's architecture, demonstrating how it simplifies AI deployments, abstracts underlying complexities, and promotes scalability. Attendees will gain insights into integrating Ramalama into their workflows, making AI deployment a straightforward and 'boring' - yet highly efficient - process.

In this presentation, we would like to introduce our open-source AI project: ‍Mozilla.ai Blueprints.