Engineering at Meta

09/01/2026

We’re releasing MetaRoCE through the Open Compute Project (OCP), making our complete specification, reference software implementation, and compliance test suite freely available to the broader tech industry.

Training and serving frontier AI models requires ultra-fast, reliable networks that transfer data between GPUs without wasting valuable compute cycles. To solve this at scale, we designed MetaRoCE, a clean-sheet RDMA transport protocol purpose-built specifically for AI workloads on commodity Ethernet.

By contributing this to OCP, we are enabling hardware vendors, network operators, and developers to adopt, implement, and build the future of AI infrastructure together.

Watch the full technical deep dive from the At Scale Networking Conference: https://engineering.fb.com/2026/08/24/networking-traffic/metaroce-rdma-transport-ai-ethernet/

08/31/2026

When embedding tables make up >99% of model parameters, GPUs lose up to 20% of compute throughput just managing AllReduce and AllToAll collectives.

MTIA 300, the first of Meta’s family of in-house training and inference accelerators optimized for training ranking and recommendation models, fixes this by co-designing hardware and networking from scratch:

1️⃣ On-package NICs: 12x custom 800 Gbps RDMA NICs (1.2 TB/s I/O) bypass the host PCIe bus entirely.

2️⃣ Offloaded ex*****on: 16 dedicated RISC-V Message Engines handle network ops, keeping compute degradation under 0.5%.

3️⃣ Compiled HCCL: Integrates directly with torch.compile and PyTorch c10d for host-free ex*****on.

The result: 3.9x faster communication time on a 150B-parameter production recommendation model.

See the technical deep-dive from the At Scale: Networking Conference: https://engineering.fb.com/2026/08/24/networking-traffic/mtia-300-meta-training-chip-built-in-nics/

08/06/2026

Meta has developed and deployed a new multi-layer sequence learning architecture for ads recommendation systems.

This architecture is built for production at scale as a resource efficient, multi-stage system that boasts high upstream-to-online transfer ratios.

📈 Our latest research paper, “Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation,” contains a comprehensive scaling law study — analyzing model width, depth, information density, and sequence length — and how that drives ad performance growth.

👉 Read the full paper to learn more! https://engineering.fb.com/2026/08/05/ml-applications/from-user-sequences-to-scaling-laws-a-multi-stage-architecture-for-metas-ads-ranking/

08/04/2026

Meta's Generative Ads Recommendation Model (GEM), the engine powering ads across Instagram and Facebook, is now training at LLM-scale across thousands of the latest-generation GPUs.

Training a massive recommendation model like GEM brings unique challenges. Unlike typical LLMs, GEM uses a hybrid architecture with trillions of sparse embedding parameters and billions of dense parameters, plus highly variable data characteristics. AI infrastructure built for standard LLM training doesn’t just transfer over.

So, how did our engineering team crack the code? Through hardware/software co-design, focusing on two key innovations:

⚡ Compute Efficiency
We achieved this through a custom-built recommendation kernel library featuring Jagged Flash Attention, Generalized Dot-Product Attention, and BlockAttention. We also implemented mixed ultra-low precision training (including MXFP8 attention and MLP) purposely designed to exploit the latest GPU architectures.

🌐 Scaling Efficiency
To scale effectively, the team deployed topology-aware 5D parallelism with Streaming Multiprocessor-free collectives. By combining 2D FSDP + Expert Parallelism for dense parameters and Fully Sharded 2D Model Parallelism for sparse parameters—co-designed with Meta's network hierarchy—we significantly reduced communication overhead.

🏆 The Result
Over the past 12 months, we successfully doubled GEM's end-to-end training efficiency to reach 20–25% Model FLOPs Utilization, all while scaling total training FLOPs by 4x.

Check out the full technical breakdown on the Meta Engineering Blog here: https://engineering.fb.com/2026/08/03/ml-applications/training-gem-at-llm-scale-meta-ads-recommendation-foundation-model/

07/23/2026

At Meta's scale, a few milliseconds of kernel scheduling latency can impact millions of ad decisions per second.

When we upgraded our Linux kernel across the ads serving fleet, the new default scheduler introduced a latency regression that reduced the number of ads we could rank in time. We needed a solution.

Enter sched_ext: an open-source, BPF-based framework we co-developed that lets engineers write and deploy custom scheduling policies without touching the kernel itself.

The results speak for themselves:�
📉 28% reduction in P99 tail latency�
⚡ 3.28 megawatts in power savings�
📈 1.1% increase in ads ranked

Now, sched_ext is upstream in Linux v6.12. That means any team — at a hyperscaler, cloud provider, or embedded systems company — can build workload-specific scheduling without forking the kernel.

Read the full deep dive: https://engineering.fb.com/2026/07/13/ml-applications/modernizing-the-meta-ads-service-with-an-open-source-kernel-scheduler/

07/22/2026

AI compute performance roughly triples every two years, but storage performance growth has been far more modest. The result? Storage bottlenecks and GPU stalls during large-scale model training.

To maximize GPU utilization and research velocity across hundreds of exabyte-scale storage clusters, we overhauled Meta’s BLOB-storage architecture specifically for modern AI workloads.

Key Architectural Innovations:

1️⃣ Unified Metadata Subsystem: Collapsed multi-layer metadata into a single flat schema backed by ZippyDB, enabling O(1) path-to-storage address lookups.

2️⃣ Zero-Proxy Dataplane: Replaced proxy servers with a fat client SDK that streams bytes directly from Tectonic storage servers to GPU hosts—reducing latency while remaining within strict datacenter power budgets.

3️⃣ Distributed Hotspot Caching: Leveraged spare GPU host RAM as a distributed data cache (achieving an average 80% hit rate) alongside a 1–2 ms read-plan metadata cache.

4️⃣ Planet-Scale Tiered Storage: Applied operating system memory-hierarchy concepts to regional data loading—utilizing GPU host RAM/flash as L1/L2 caches and regional disaggregated flash as an L3 cache.

5️⃣ Deep Prefetching: Added explicit prefetch() APIs and automatic TTL/LRU eviction policies to eliminate cross-region transfer delays.

These updates eliminate storage-induced GPU stalls, add zero overhead on top of our foundational Tectonic block layer, and cut dataset ingestion times for researchers from hours down to minutes.

Read the full engineering deep dive on the blog: https://engineering.fb.com/2026/07/01/data-infrastructure/metas-ai-storage-blueprint-at-scale/

07/21/2026

We’ve modernized Facebook Groups Search with a hybrid retrieval architecture to help users more reliably discover and validate community knowledge.

By moving beyond traditional keyword matching, this system processes natural language intent to improve relevance and engagement without an increase in error rates.

Here is how we built it:
1️⃣ Parallel Retrieval: We decoupled query processing into a lexical path (Unicorn inverted index) and a semantic path using a 12-layer, 200M-parameter Search Semantic Retriever (SSR).

2️⃣ Advanced Ranking: Candidates are merged in the ranking stage using a Multi-Task Multi-Label (MTML) supermodel to optimize for multiple engagement objectives simultaneously.

3️⃣ Automated Evaluation: We utilize Llama 3 as an automated judge to grade search results against queries, allowing us to validate quality at scale and capture nuance without human labeling bottlenecks.

➡️ Read the engineering blog and paper to learn more:

We’ve fundamentally transformed Facebook Groups Search to help people more reliably discover, sort through, and validate community content that’s most relevant to them. We’ve adopted a new hybrid r…

07/01/2026

This year marks Meta’s 10th consecutive year as a sponsor of the Python Software Foundation!

Python is the most heavily used programming language across Meta’s engineering stack. It serves as the architectural backend for core products like Instagram and Threads, and acts as the foundation for our cutting-edge AI research and infrastructure.

Our decade-long technical investment focuses on ensuring the open-source ecosystem remains performant, secure, and sustainable. Key areas of our ongoing collaboration and contribution include:

1️⃣ Core Maintenance: Meta engineers serve as core maintainers of the language, directly authoring new features and Python Enhancement Proposals.

2️⃣ Open-Source Tooling: We build open-source Python developer tools to help developers write better quality, more performant Python, including projects like Pyrefly (a fast type checker and language server).

3️⃣ Community Growth: Our support helps fund educational programs and community events like PyCon US and PyLadies to grow the community and foster new talent.

Read our full reflection on our long-term commitment to Python: https://engineering.fb.com/2026/06/30/open-source/10-years-of-metas-commitment-to-python/

06/30/2026

The battery inside your AI glasses has to fit in the temple arm — a space so narrow that traditional smartphone batteries simply won't work.

On the latest episode of the Meta Tech Podcast, engineers Karthik Kadirvel and Myuran Kanga explain how Meta's battery team reimagined what a battery could be, moving from conventional rectangular cells to a custom steel-can design that adapts to the unique form factor of AI glasses.

The result? A 40% improvement in battery life, plus entirely new supply chains and global certification processes built from the ground up.

🎧 Listen now: https://engineering.fb.com/2026/06/23/production-engineering/how-meta-built-ultra-narrow-batteries-for-ai-glasses-meta-tech-podcast/

06/25/2026

How do you build privacy controls when your data is ambiguous?

A field called "age" might describe a person or a cache expiration value in an infrastructure pipeline. At Meta's scale, these ambiguities exist across millions of assets.

To solve for this, we built a hybrid approach to asset classification that:
1️⃣ Build a rich context before asking a model to reason.
2️⃣ Use LLMs to handle ambiguity, cold start, and novelty.
3️⃣ Keeps human-reviewed labels separate from model-generated recommendations
4️⃣ Distills stable behavior into deterministic rules for routine enforcement

The key insight: the end goal isn't AI everywhere. It's a system that can learn from ambiguous signals while moving enforcement toward logic that's low-latency, reproducible, and auditable.

See the full technical breakdown from our blog here: https://engineering.fb.com/2026/06/25/security/privacy-aware-infrastructure-in-the-ai-native-era-an-asset-classification-case-study/