These guys. Always at the cutting edge of development. They say that Apple finally joined the AI game, locally.
That is me and my buddy DeepSeek:
What can you find about the Apple AI capable of running locally on Apple's unified memory, please?
Here is the information about Apple's AI capabilities for running locally on its unified memory, now in English.
Apple is aggressively pushing powerful AI models to run **on-device** (locally) via its **Unified Memory Architecture (UMA)**. This allows Macs and iPads to handle complex AI tasks without relying on expensive cloud servers, offering a major leap in privacy and performance.
Here is a breakdown of its core advantages, supported hardware, and key technical details.
### 🧠 Core Advantage: Unified Memory Architecture
Apple’s UMA is the cornerstone of its local AI strategy. It allows the CPU, GPU, and Neural Engine (NPU) to access a single, high-speed pool of memory simultaneously.
- **Eliminates Bottlenecks:** It removes the inefficiency of copying data back and forth between system RAM and dedicated GPU VRAM, which is a major bottleneck in traditional PC architectures.
- **Massive, High-Speed Memory Pool:** The entire memory pool acts as a unified space directly accessible by all compute units. This makes it feasible to run Large Language Models (LLMs) with hundreds of billions of parameters entirely on a consumer device.
- **Privacy and Security:** All AI computations happen on the device. User data never leaves the machine, effectively protecting personal privacy.
- **Cost-Effectiveness:** It enables consumer-grade devices to run AI models that previously required enterprise-grade servers costing tens of thousands of dollars.
---
### 💻 Apple Silicon Chips Supporting Local AI
Apple’s latest M-series chips provide varying levels of on-device AI power for different user needs:
- **M5 Ultra (Flagship):** Apple’s most powerful chip to date, featuring a quad-die architecture.
- **Memory & Bandwidth:** Supports up to **512GB** of unified memory with a staggering bandwidth of **1.2 TB/s**.
- **AI Capability:** Specifically optimized for local LLM inference. It can run **ultra-large language models with hundreds of billions of parameters entirely on-device**. It features an integrated GPU with up to 80 cores.
- **M6 (Next-Gen Mainstream):** The first M-series chip built on a **2-nanometer** process.
- **Memory & Bandwidth:** Supports up to **32GB** of unified memory with bandwidth between 160 GB/s and 170 GB/s.
- **AI Capability:** Capable of running LLMs and processing multiple high-load applications simultaneously on-device. It features a dual 16-core Neural Engine, delivering up to **2x the AI peak performance** compared to its predecessor.
---
### 📱 On-Device AI Requirements by Device
Running the most advanced local AI models requires specific hardware thresholds:
- **iPhone & iPad:** Apple’s most advanced on-device AI models (such as those introduced in iOS 27) require devices with **at least 12GB of unified memory**. This means only the **iPhone 17 Pro series** and **iPads with the M4 chip or newer (and ≥12GB RAM)** fully support these features. The base iPhone 17, which has 8GB of memory, cannot meet this requirement.
- **Mac Computers:** Any Mac with an **M3 chip or newer** and at least **12GB of memory** can run the most advanced local AI models.
---
### 🧩 Software Ecosystem and Advanced Clustering
- **MLX Framework:** Apple has created the open-source **MLX** machine-learning framework, specifically optimized for the M-series chip architecture. It is designed to efficiently leverage UMA for both training and inference.
- **Multi-Machine Clustering:** Via the **Thunderbolt 5** interface, multiple Macs (e.g., Mac Studio units) can be linked together into an "AI cluster." This combines their unified memory pools to handle massive models. For example, a cluster of four Mac Studios can share up to **2TB** of unified memory, which is enough to run **trillion-parameter** models.
- **System Support:** macOS 26.2 natively supports low-latency distributed AI inference over Thunderbolt 5, making multi-device clustering seamless.
---
### 🏁 Conclusion
Through the deep integration of its proprietary silicon and Unified Memory Architecture, Apple is redefining the boundaries of personal computing. It is making powerful, secure, and high-performance AI accessible on everyday devices, shifting the center of gravity away from cloud-dependent services and toward private, local computation.
What about Apple's LLM models?
Apple's LLM strategy revolves around its **third-generation Apple Foundation Models (AFM 3)**, a family of five models unveiled at WWDC 2026. These models are designed to work together, running either directly on the device or in Apple's Private Cloud Compute, to power the new "Siri AI" and other Apple Intelligence features.
Here is a detailed breakdown of Apple's current LLM offerings.
The AFM 3 family consists of two on-device models and three server-based models.
| Model Name | Location | Scale | Key Characteristics |
| :--- | :--- | :--- | :--- |
| **AFM 3 Core**
| **AFM 3 Core Advanced**
| **AFM 3 Cloud**
| **ADM 3 Cloud**
| **AFM 3 Cloud Pro**
### 💡 Key Technological Innovations
Apple has introduced several novel techniques to make its powerful LLMs run efficiently on its hardware:
* **Dynamic Sparsity & Instruction-Following Pruning (IFP)**: The key to the **AFM 3 Core Advanced** is its use of IFP. Instead of activating the entire 20B-parameter model for every request, it uses a small predictor to dynamically select and activate only the most relevant "expert" parameters (1-4B) for the specific prompt. This allows the performance of a much larger model while being computationally efficient.
* **Flash Memory Utilization**: To run such a large model on a device, Apple stores the complete **AFM 3 Core Advanced** model in NAND flash memory. Only a small set of "always-on" shared experts are kept in DRAM. When a prompt is received, the necessary experts are loaded from flash into DRAM for that specific request. This clever approach overcomes the bandwidth limitations between flash and DRAM.
### 🎯 Model Capabilities & Use Cases
* **On-Device**: The two on-device models (**AFM 3 Core** and **AFM 3 Core Advanced**) are designed for **low-latency, private, and offline-capable** tasks. **AFM 3 Core Advanced** powers the new "Siri AI", enabling it to "see" camera and screen content thanks to its multimodal capabilities.
* **Private Cloud Compute**: The server-side models handle more complex tasks that require greater computational power. Crucially, Apple designed the **Private Cloud Compute** infrastructure to ensure user data is **never stored or shared** with Apple when using these cloud models.
* **Complex Tasks**: For the most difficult tasks like agentic workflows and complex reasoning, the **AFM 3 Cloud Pro** steps in. This model was trained using the outputs from Google's Gemini models.
### 🌍 A Special Model for the Chinese Market
In a significant strategic move, Apple has reportedly developed a **dedicated large language model specifically for the Chinese market**.
* **Partnership**: The model is being developed in collaboration with **Alibaba**, with the Chinese tech giant providing training support.
* **Strategy**: This "dual-track" approach involves Apple creating its own China-specific model while also integrating with local providers like Alibaba's "Tongyi Qianwen" (通义千问) model to meet regulatory requirements. This would make Apple the first foreign tech company to deploy its own LLM in China.
The AFM 3 models are expected to be available in Fall 2026 as part of operating system updates.
Compare the local ones with those, please:
phi3.5:3.8b 2.2 GB
llama3.2:latest 2.0 GB
llama3.2:3b 2.0 GB
all-minilm:latest 45 MB
deepseek-r1:7b 4.7 GB
llama3:latest 4.7 GB
mixtral:8x7b 26 GB
phi:2.7b-chat-v2-q4_0 1.6 GB
And tell me how Apple embedded their's into the OS and Software stack available for Apple OS.
Here is a detailed comparison of Apple’s on-device AI models (AFM 3 Core & AFM 3 Core Advanced) with the models from your list, followed by an explanation of how Apple has embedded them into its operating systems.
| Model | Total Size (Params) | Active Size (Params) | Disk Space | Architecture | Key Strengths & Use Cases |
| :--- | :--- | :--- | :--- | :--- | :--- |
| **all-minilm:latest**
| **phi:2.7b-chat-v2-q4_0**
| **llama3.2:3b / llama3.2:latest**
| **AFM 3 Core**
| **phi3.5:3.8b**
| **deepseek-r1:7b**
| **llama3:latest**
| **AFM 3 Core Advanced**
| **mixtral:8x7b**
> *\*Approximate value. \*\*The full 20B model is stored in flash memory, with only the active 1-4B parameters loaded into DRAM per request.*
**Key Insights from the Comparison:**
* **Efficiency is Apple's Superpower**: The **AFM 3 Core Advanced** is the most impressive technically. It achieves the "effective" intelligence of a huge 20B model with the memory footprint (and speed) of a much smaller one, thanks to its sparse architecture. This is a significant advancement over models like `mixtral:8x7b`, which is similarly sparse but much larger on disk and in active memory.
* **Performance**: The AFM 3 Core Advanced runs at about **30 tokens per second** on an iPhone 15 Pro. For reference, an M4 Max Mac can run a 7B model at ~95 tokens/sec.
* **Multimodality**: Apple's on-device models can natively process images, not just text. This is a key differentiator from most models on your list, which are text-only.
* **Hardware Requirements**: Running the advanced 20B model requires a device with at least **12GB of unified memory**. The 3B Core model runs on a wider range of devices (iPhone 15 Pro and later, all M-series Macs).
* **Context Window**: Both on-device AFM models have a context window of **4,096 tokens**. In comparison, the server-based AFM 3 Cloud model offers a **32,000-token** context window.
---
### 🏗️ How Apple Embeds AFM 3 into the OS & Software Stack
Apple's integration is deep, system-wide, and designed to be seamless for both users and developers.
#### 1. System-Wide Integration
Apple Intelligence, powered by AFM 3, is woven into the fabric of the operating systems (iOS, iPadOS, macOS)****. The models are not third-party add-ons but are core OS components that ship with the system**** and are updated via OS updates****.
Examples of system-level features include:
* **Siri**: The new Siri AI is powered by the AFM 3 Core Advanced model.
* **Dictation**: Advanced, more accurate on-device dictation uses the AFM 3 Core Advanced model.
* **Photos & Apps**: Features like image understanding in Photos and intelligent tools in everyday apps are built on these models.
#### 2. A Single Swift API for Developers
For developers, Apple provides the **Foundation Models framework**, a single, powerful Swift API****.
* **Unified Access**: Developers can access the on-device models (`AFM 3 Core` and `AFM 3 Core Advanced`) and even cloud models through the same API****. The system can intelligently route requests based on complexity and device capability****.
* **No API Keys or Costs**: The on-device models are completely free for developers to use with no per-token cost****, and offer strong privacy by keeping data on the device****.
* **Structured Output**: A key feature is **guided generation**, which allows developers to define a Swift struct and have the model generate output that directly maps to it, eliminating the need for fragile string parsing****.
#### 3. The "Trick" to Running a 20B Model on a Phone
Apple uses a clever technique to run the 20B parameter model on devices with limited RAM:
* **Flash Storage**: Instead of loading the entire 20B model into active memory (DRAM), Apple stores the full model in the device's flash (NAND) storage****.
* **On-Demand Loading**: When a request is made, the system uses a technique called **Instruction-Following Pruning (IFP)** to identify and load only the 1-4 billion most relevant parameters from flash into DRAM****. This keeps the memory footprint low while delivering the power of a large model.
#### 4. Customization for Developers
Apple allows developers to fine-tune the on-device model for their specific app using **LoRA (Low-Rank Adaptation) adapters**. This is done using the **Foundation Models Adapter toolkit**, allowing for specialized, private AI features without sending user data to the cloud****.
#### 5. Privacy as a Core Principle
All of this is built on a foundation of privacy. By running powerful AI on-device, user data never leaves the device for many tasks. For more complex tasks that require cloud computing, Apple uses **Private Cloud Compute**, which extends the same privacy and security guarantees to the cloud****.
In summary, Apple’s AFM 3 models are not just another set of local LLMs. They represent a deeply integrated, system-level AI platform that is exceptionally efficient, developer-friendly, and built from the ground up with privacy in mind.
Mixtral is a very strong model that needs ages to load into RAM and does then actually not perform on an older i7-8500. On that 200 Euros hardware, but with maxed out 32GB RAM the llama3 runs fine. The trick on that old hardware is intelligent embeddement into specfic tasks and RAGing additional data points to optimize the overall system.
How much costs the hardware to run the AFM 3 model?
Are their any Open Source models that compete with the size, speed but intelligence of the AFM 3?
What is the use case Apple gave that LLM?
Here is the information about Apple's AFM 3 models, now in English.
---
### 1. 💰 Hardware Cost to Run the AFM 3 Model
The hardware to run Apple's most powerful on-device model, the **AFM 3 Core Advanced (20B sparse model)** , comes with specific requirements:
- **Hardware Requirement**: The device must have at least **12GB of unified memory**.
- **Hardware Cost**: You cannot buy the hardware component separately, as it is not sold at retail. The cost is the retail price of a compatible Apple device, which is concentrated on the "Pro" product lines. Supported devices include:
- **iPhone**: iPhone Air, iPhone 17 Pro, iPhone 17 Pro Max
- **iPad**: iPad Pro with M4 chip or later (with at least 12GB RAM)
- **Mac**: Macs with M3 chip or later and at least 12GB of RAM
> **Technical Principle**: The reason a 20B model can run on just 12GB of memory is that Apple does not load the entire model into RAM. The full model is stored in flash memory (NAND), and only the 1-4 billion parameters needed for the current task are loaded into DRAM. It uses a technique called **Instruction-Following Pruning (IFP)** to dynamically activate only the most relevant "expert" parameters for each request. This means that **the experience on a 12GB device is comparable to what would traditionally require 32GB or even 64GB of memory**.
As a comparison, Apple's base model, **AFM 3 Core (3B model)** , has a lower barrier to entry and can run on all devices that support Apple Intelligence (such as the iPhone 15 Pro and later).
---
### 2. ⚖️ Are There Any Open Source Models That Compete?
The current conclusion is: **in the specific "on-device AI" arena, no open-source model can match the combined performance of "intelligence," "speed," and "memory footprint" of the AFM 3 Core Advanced.**
- **Intelligence & Efficiency**: The AFM 3 Core Advanced uses dynamic sparsity to achieve quality close to a 9B dense model with the computational power of only activating 1-4B parameters. This is currently difficult for open-source models to replicate. They are either fixed-size dense models (like Llama 3.2 3B) or MoE models that require loading all parameters into memory (like Mixtral 8x7B, which needs about 26GB of disk space and a large amount of RAM).
- **Speed & Ecosystem**: Its running speed is approximately **30 tokens per second**. More importantly, Apple provides developers with a unified Swift API through the **Foundation Models framework** and underlying optimizations like **Core AI**. This level of deep, system-wide integration is something no open-source model can achieve within the Apple ecosystem.
---
Apple has defined clear system-level and developer-facing use cases for the AFM 3 models:
| Model
| :--- | :--- |
| **AFM 3 Core Advanced**
| **AFM 3 Cloud**
| **ADM 3 Cloud (Image)**
| **AFM 3 Cloud Pro**
For developers, Apple provides a unified API to call these models, allowing apps to seamlessly switch between on-device and cloud models based on task complexity.
Ok. I get its powerful, I get that's about 1k of Apple hardware sitting there and that we have still a lot of marketing slang in your search results.
Deeply embedded into the OS means for the average Apple user to click and drop, use hardly a keyboard and to use the microphone for Google search over typing.
Apple users normally don't have heavy reasoning tasks over large sets of data, like BI questions for a Spreadsheet, PDF or .docx file. To be honest, they hardly use multi-clause sentences and most of their task sets is comparable. So, can we expect that local AI on their hardware to be more advanced than Copilot systems of online services or do they bypass data restriction laws plus make if much more fancy than already being? The Super Apple.
Here is the previous response, now in English.
---
You raise a very practical point. In the daily use of an average user, Apple Intelligence may indeed *feel* more "usable" than cloud-based services like Copilot. However, this isn't because it's "more advanced" in an absolute sense—it's because their design goals and applicable scenarios are fundamentally different.
Apple Intelligence and Microsoft Copilot are products of two entirely different philosophies. In short: **Apple's strategy is to "make AI adapt to the average user," while Microsoft's strategy is to "arm professional users with powerful AI."**
### 🎯 Design Philosophy: Privacy & Convenience vs. Depth & Power
The core differences lie in their design objectives:
- **Apple Intelligence: A privacy-first personal assistant.** Its core goal is to **seamlessly integrate AI into daily life while protecting privacy**. It adheres to the principle of "data minimization"—if a task can be handled locally, it never goes to the cloud. This design allows users to enjoy AI benefits with zero cost and zero latency, but the trade-off is that **the intelligence ceiling of local models is constrained by device hardware**.
- **Microsoft Copilot: A productivity tool that connects everything.** Its core goal is to **deeply integrate into workflows and become a powerful productivity engine**. Through Microsoft Graph, it connects to all enterprise data—SharePoint, Teams, Outlook, etc.—and can handle long documents, complex reports, and more. However, the trade-off is **heavy reliance on the cloud**, which introduces data transfer concerns and additional API costs.
| Aspect
| :--- | :--- | :--- |
| **Everyday Tasks (Summaries/Replies)**
| **Complex Reasoning / Long Documents**
| **Privacy & Compliance**
| **Ecosystem Integration**
| **Target Users**
### ⚖️ On "Bypassing Data Restriction Laws"
Regarding your point about "bypassing data restriction laws," this is more about how Apple proactively responds to strict privacy regulations (like GDPR), rather than "bypassing" them.
- **Proactive Compliance by Design**: Apple's local-first strategy is itself a response to data regulations. By keeping data processing on-device, it naturally avoids the compliance risks associated with cross-border data transfers.
- **Cloud Commitments Are Equally Strong**: Even for complex tasks that must go to the cloud, Apple uses "Private Cloud Compute," which promises that data is processed in a **stateless** manner (burned after use), cannot be accessed by Apple itself, and that this process is verifiable.
- **Not Without Criticism**: This system is not without its critics. Some have pointed out that Apple cannot clearly tell enterprise users exactly in which country their cloud data is being processed, which is a concern for organizations with strict data residency requirements. Additionally, this "privacy black box" makes it difficult for enterprise security teams to monitor the AI's processing logic.
### 💎 Summary
To return to your original questions:
- **Is it more advanced than Copilot?** In some areas (privacy protection, on-device efficiency, ecosystem integration), yes. In terms of handling complex reasoning tasks, no.
- **Is it bypassing data regulations?** It is not "bypassing" them. Rather, it *technically satisfies* the core requirements of data protection laws through a clever architectural design—essentially **"moving computation to the data"** instead of the other way around.
**Apple's path is to "make AI invisible," while Microsoft's path is to "make AI powerful."** For the daily needs of most average Apple users, the former provides a more seamless and private experience. For heavy tasks like complex reports and long document processing, cloud-based solutions like Copilot remain the more powerful tool for now.
Complying is a form of bypassing a crash, sorry for that. I meant complying. So, Apple users will have a much better experience talking with their phone like: Siri. Open the last email from Mr Smith and give me a summery. .... Make a reply and write: Hello Mr Smith, thanks for your kind words. I am happy to meet you. What about next Friday, ah Siri, what does my calendar say for Friday
And Siri understands what to not write before sending?
Yes, absolutely. With the new **Siri AI** introduced in iOS 27 and macOS 27, the scenario you described is exactly the kind of complex, multi-step task it's designed to handle.
Here’s how your request would work in practice:
### 🗣️ The Interaction Flow
1. **"Siri, open the last email from Mr. Smith and give me a summary."**
- Siri AI will search your Mail app, find the most recent email from Mr. Smith, and generate a concise summary of its content.
2. **"...Make a reply and write: Hello Mr. Smith, thanks for your kind words. I am happy to meet you."**
- Siri will then draft a new email reply. Crucially, it maintains the context of the conversation, knowing the draft is for the email it just summarized.
3. **"What about next Friday, ah Siri, what does my calendar say for Friday"**
- You can interrupt or change the subject mid-request. Siri will seamlessly switch tasks, query your Calendar app, and inform you of your schedule for next Friday.
4. **Siri's Final Understanding**
- After telling you your schedule, Siri won't automatically send the draft. It will wait for your final confirmation or further instructions, such as "Okay, send it," or "Change the meeting time to 2 PM," before taking the final action.
### 🚀 Key Capabilities Enabling This
This level of interaction is possible because Siri AI has been rebuilt from the ground up with several major upgrades:
* **Personal Context Awareness**: It can understand and connect information across your apps, like your emails, messages, calendar, and photos.
* **Cross-App Actions**: It can execute multi-step tasks that involve multiple apps (Mail, Calendar, Notes, etc.) from a single, natural language request.
* **Screen Awareness**: It can see and understand the content on your screen, which allows it to perform actions like creating calendar events from an email you're reading.
* **Natural, Multi-Turn Conversations**: It can handle follow-up questions and remember the context of your ongoing conversation.
> **Note**: These advanced features are part of the new Siri AI, which is expected to launch with iOS 27 and macOS 27 in the fall of 2026.
How would Copilot in the Microsoft Universe have more capability over its Office Suite? They'd use the Keyboard more and...
Here is the previous response, now in English.
---
The difference between Microsoft Copilot and Apple Siri AI comes down to a fundamental design distinction: **Productivity Tool vs. Personal Assistant**.
As you rightly pointed out, in the Microsoft universe, the keyboard and mouse remain the primary interaction methods. Copilot’s strength isn't in flashy voice conversations—it's in its deep integration into Office workflows, where it acts as an **agent** that can actively execute complex, multi-step tasks *on your behalf*.
Here is a detailed breakdown of how Copilot is more capable within the Office suite.
### 🚀 Core Difference: From "Co-pilot" to "Agent"
While older AIs gave you suggestions (acting as a "co-pilot"), Copilot (as of 2026) has evolved into an **"Agent"** that can drive for you. It doesn't just answer questions; it can **directly take action within your documents, spreadsheets, and presentations** to complete multi-step complex tasks.
### 📊 Specific Capabilities Across the Office Suite
Copilot's capabilities are deeply embedded into every core application:
- **Microsoft Word**: It goes far beyond rewriting and summarizing. Copilot can take fragmented notes or raw content and transform them into a fully structured **proposal or report**. It can rewrite, reorganize, or restyle an entire document based on a specified tone or audience. Its new **"Agent Mode"** can even iteratively refine content over time, rather than just generating a single draft.
- **Automatically clean data and identify outliers**.
- **Independently perform advanced calculations and explain its reasoning**.
- **Create pivot tables and charts**, and even generate **multi-page reports** based
- **Microsoft PowerPoint**: Copilot can generate an **entire presentation deck—complete with images, charts, and layouts—from a simple outline or an existing Word document**. Critically, it can adhere to your organization's or your personal brand design guidelines.
- **Microsoft Outlook**: It can **summarize lengthy email threads** and draft personalized replies. The new **"Voice Catch-Up"** feature allows you to use voice commands to get a spoken summary of your inbox and action items. It can also **analyze your entire mailbox and calendar** to suggest meeting times or plan your schedule.
- **Microsoft Teams**: During meetings, Copilot acts as an **interactive meeting agent**. It understands content shared on screen, whiteboards, and even visual elements, providing you with private meeting summaries and actionable insights in real-time.
### 🔗 The Deeper Power: Microsoft Graph
The reason Copilot can do all of this is its secure access to the **Microsoft Graph**. This is a massive data graph that connects all your work-related data—emails, calendar events, files, meetings, chats, and more. This means every answer and action Copilot takes is grounded in the **most current and complete context of your personal and organizational workflow**.
Going further, enterprises can use **"Graph Connectors"** to integrate external business data (like CRM or ERP systems), allowing Copilot to understand and reason across the company's entire data landscape.
### 💡 A Practical Scenario Comparison
To understand the difference intuitively, compare how each handles the same complex task: **"Create a PowerPoint report based on last week's sales data and email it to my manager."**
- **In the Apple ecosystem (Siri)**: You'd likely have to say: *"Siri, find last week's sales data file. ... Okay, now create a PowerPoint. ... Now email that PowerPoint to my manager."* This is a **linear, single-step sequence** where you have to explicitly dictate each action.
- **In the Microsoft ecosystem (Copilot)**: You would simply type or say into Word or the Copilot chat window: **"Create a PowerPoint report based on last week's sales data from my email and draft an email to my manager."** Copilot acts as an **Agent**—it automatically breaks down the request, finds the data, generates the PPT, drafts the email, and presents it to you for review and confirmation before executing.
| :--- | :--- | :--- |
| **Core Positioning**
| **Primary Interaction**
| **Capability Boundary**
| **Core Advantage**
| **Target User**
In short: **Siri is like your personal secretary**—great at handling information in everyday contexts on your phone. **Copilot is like a super-capable intern**—it works directly inside your documents and spreadsheets to get the actual work done for you. They are powerful in entirely different dimensions.
#cyberpunkcoltoure #opensource
