Google Cloud Introduces Always-On Memory Agent for Continuous LLM Context

Google Cloud has released a new reference implementation called the Always-On Memory Agent, designed to provide continuous memory and context for large language models (LLMs). This agent, part of Google Cloud’s generative-ai repository, addresses the common limitation of AI agents that lose context after processing a request, offering a persistent memory solution built on Google ADK (Agent Development Kit) and Gemini 3.1 Flash-Lite.

Unlike traditional methods that often rely on Retrieval Augmented Generation (RAG) and vector databases for memory, this agent operates without them. Instead, it employs an LLM to read, process, and write structured memory directly into a SQLite database. This design prioritizes low latency and cost efficiency, making it suitable for continuous background operations.

Core Architecture and Functionality

The Always-On Memory Agent functions as a lightweight background process, running 24/7. Its architecture is managed by an orchestrator that directs requests to three specialized sub-agents: Ingest, Consolidate, and Query. Each sub-agent is equipped with specific tools for managing the memory store.

The IngestAgent is responsible for handling incoming content. It leverages Gemini’s multimodal capabilities to extract key information such as summaries, entities, topics, and importance scores from various data types. This structured information is then stored in the memories table. The agent supports 27 file types across five categories, including text, images, audio, video, and PDFs, allowing users to simply drop files into an inbox folder for automatic processing.

The ConsolidateAgent operates on a timed schedule, by default every 30 minutes. Its role is to review unconsolidated memories, identify connections between them, and then generate synthesized summaries and key insights. This continuous consolidation process allows the agent to build new understanding even when idle, without requiring explicit prompts for new information.

Finally, the QueryAgent is designed to answer questions by reading all stored memories and consolidated insights. It synthesizes a response and, importantly, cites the memory IDs used as sources, enhancing transparency and traceability of information.

Bypassing Traditional RAG and Embeddings

A significant departure from current LLM memory solutions is the agent’s complete avoidance of vector databases and embeddings. While RAG approaches retrieve relevant information from a knowledge base to augment LLM responses, and embeddings represent data in a high-dimensional vector space for similarity searches, the Always-On Memory Agent takes a different route. It treats memory as an active, running process where an LLM directly manages and structures information, rather than merely retrieving it. This continuous consolidation of knowledge aims to provide a more durable and evolving context for LLM applications.

Key Takeaways

Feature	Description
Memory Mechanism	LLM reads, processes, and writes structured memory to SQLite; no vector database or embeddings.
Operating Model	Runs 24/7 as a lightweight background process on Google ADK and Gemini 3.1 Flash-Lite.
Agent Structure	Orchestrator manages three sub-agents: Ingest, Consolidate, and Query.
Consolidation	Consolidates memories and generates insights every 30 minutes, even when idle.

Implications for Indian Developers and Businesses

For developers and businesses in India, this new agent offers a compelling alternative for building AI applications requiring persistent and evolving LLM context. The ease of setup, combined with the low latency and cost-effectiveness of Gemini 3.1 Flash-Lite, could accelerate the development of more sophisticated and context-aware AI agents. Use cases could span various sectors, from customer service bots that remember past interactions to intelligent assistants that continuously learn from new data, and internal knowledge management systems that proactively consolidate information. The ability to ingest a wide range of file types also makes it highly adaptable for diverse data environments common in Indian enterprises.

The implementation is designed for straightforward deployment, requiring minimal setup for early-level engineers. It includes an HTTP API for easy integration and an optional Streamlit dashboard for management.

Source: MarkTechPost, https://www.marktechpost.com/2026/07/18/google-clouds-always-on-memory-agent-replaces-rag-and-embeddings-with-continuous-llm-consolidation-on-gemini-3-1-flash-lite/