Planet Mozilla

This essay was originally published on The Sidebar, Mozilla’s Substack.

There’s a moment Wiandi Vreeswijk knows well. After tending virtual crops in “Stardew Valley” and chatting with other players on Discord, he types: “I can’t anymore. I have to go lay in bed.” It’s the fatigue from long COVID setting in — and almost always, the replies roll in: “Oh yeah, me too.”

Cozy games like “Stardew Valley,” where players complete simple-yet-satisfying tasks like running a farm, have become a safe space for Wiandi.

A video game developer in the Netherlands, Wiandi led an active lifestyle until he got COVID-19 in 2023. He’s had long COVID since then, which makes it difficult for him to go out with friends and family, or to play some of the more intense video games he once enjoyed. That shift led him to explore a different kind of play, one that emphasizes comfort and connection.

For players like Wiandi who are seeking a slower-paced environment, cozy games offer an easy and welcoming entry point. The genre has seen a 57% increase in online mentions in just one year, as more people seek out calm and connection online.

A space to gather

Wiandi recently started a community on the gaming platform Steam where players with chronic conditions can congregate. Since then, he’s set up a public “Stardew Valley” server where people can drop into a shared game, “farm” and chat as they please. The group currently has 56 members, while a corresponding Discord chat has hundreds of participants. 

“It’s been bigger than I expected,” says Wiandi, who initially started with a Dutch community before expanding internationally. “I noticed that a lot of people wanted to join.”

For Wiandi, the best part of building this online community is connecting with others who can relate to what he’s going through. The virtual world allows him to meet people he otherwise wouldn’t have. He can simply fire off a message on Discord.

Bonding over shared comfort

Laura Dale, an accessibility consultant for video games and the author of “Uncomfortable Labels: My Life as a Gay Autistic Trans Woman,” often plays cozy games to connect with her neurodivergent friends. She says they’ll regularly chat online while building towns in “Animal Crossing” on their respective Nintendo Switches, a group activity that began during COVID lockdowns but is still going strong.

As someone who sometimes finds social chatter challenging, especially when there’s no obvious topic to discuss (like a video game), playing cozy games gave Laura a relaxing way to maintain and build her relationships.

“I found a lot of solace in playing ‘Animal Crossing’ with friends online,” Laura says. “Having this shared activity gave us a safe topic of conversation.”

Each person doesn’t even need to be playing the same game, as long as they share the same cozy vibe.

“We’re all doing different cozy game activities, but we’re doing them together,” Laura adds.

The appeal of low-pressure play

From “Tetris” to “Elden Ring,” there’s a video game out there for everyone. So what is it about cozy games that attracts players like Laura and Wiandi?

For Wiandi, the answer is obvious: Cozy games have a low barrier to entry.

“Even my mom could learn it in like a day,” he says of “Stardew Valley.” “That’s the charm of a cozy game. When you start, even if you have no experience, you don’t feel overwhelmed by any of the mechanics or the visuals. Everything has to be super easy and minimal — the user experience, the music, the sound effects, even the gameplay.”

Many of the defining characteristics of cozy games, like their leisurely pace, mean they’re more accessible to a wider range of gamers. There’s less pressure to press buttons quickly or get the timing of an attack perfectly right. Titles like “Stardew Valley” and “Animal Crossing” also offer a fixed top-down camera angle, which makes them a better option for some gamers who experience motion sickness, compared to the disorienting camera movement of a first-person shooter. In general, cozy games are also less likely to feature the kind of visual overstimulation that’s common in other genres and can be an issue for players with epilepsy or autism. 

For some players with conditions that affect memory or focus, like ADHD, these games are also designed to quickly remind you what task you were in the middle of during your last session. In “Animal Crossing,” for example, there’s almost always a non-player character nearby ready to explain (or re-explain) what you need to do next if you need it.

“All of these things lend themselves to a wide range of neurodiverse players being able to more safely assume that a cozy game is going to be accessible in a way that you can’t assume as easily with other genres,” Laura says.

Accessibility beyond cozy labels

Cozy games aren’t a perfect fit for every type of disability — there’s no one-size-fits-all solution for such a wide and disparate community.

For Grant Stoner, a games journalist who primarily covers physical accessibility in the industry, cozy games aren’t necessarily more or less accessible than any other genre. Stoner was diagnosed with Spinal Muscular Atrophy type II at 13 months old, and his muscles have weakened over time as a result; he relies on customizable settings and hardware to play most games. There are still some limitations to the types of games he can play, however, including some titles that fall under the cozy genre. 

“Depending on what your disability is, cozy games can be either very overwhelming or very secure safe spaces,” Grant says.

He adds that one of the benefits of a cozy game is the “routines that keep people grounded.” He played an earlier version of “Animal Crossing” for the Nintendo DS handheld, but found that some of the tasks in the latest iteration, for the Nintendo Switch, were too exhausting for him. And anyway, it’s not his video game genre of choice.

“I don’t really like cozy games,” Stoner says. “Not that I think there’s anything wrong with them, it’s just not my genre.” He prefers “intense action games” but still sees the value of titles like “Stardew Valley.” “They have a purpose in the industry.”

Making room for more players

Laura praises the cozy game genre for the many ways it caters to neurodivergent players, but she also recognizes that there’s plenty of room for improvement. One easy fix is in the way these games direct your attention. For plenty of video games, audio cues are a crucial way to convey information, but Laura often plays with the sound off to avoid overstimulation. She’s noticed that some titles are starting to use other methods to achieve the same result.

“I appreciate when cozy games offer visual flashes on screen to communicate information you otherwise need to hear,” Laura says. “Little details like that aren’t always designed for autistic players, but can still be useful.”

For Grant, the most important thing is that the video game industry doesn’t try to shoehorn certain players into a specific type of game or focus its accessibility efforts on just one genre.

“The disabled experience is so individualistic and so vast,” he says. “It’s not fair to the disabled community to say definitively that one [genre] is more accessible than the other.”

As a video game developer himself, Wiandi has plenty of opinions on how to make cozy games better. But for now, he’s just happy to have this new community. 

The small “Stardew Valley” server Wiandi built continues to show how simple interactions in calm digital spaces can create genuine bonds. Players come and go as they please, planting virtual seeds, raising pixelated animals and sharing small triumphs in a chat filled with mutual understanding. 

For Wiandi, the ability to play a game like “Stardew Valley” with other people who are experiencing something similar to him has been empowering.

“It makes me feel good,” he says. “It’s awesome.”

Übernimm die Kontrolle über dein Internet.

Firefox herunterladen

The post Cozy games: A slower pace, and a place to belong  appeared first on The Mozilla Blog.

Reporting to Mozilla President, Mark Surman, Raffi will be part of the team that coordinates efforts across our whole family of organizations. He will work alongside the existing bench of technical leaders including Anthony Enzor-DeMeo (GM / Firefox), Bobby Holley (CTO / Firefox), and Ryan Sipes (Thunderbird).

As AI and technology development becomes increasingly concentrated in the hands of a few, Krikorian joins Mozilla at a moment of urgency and opportunity. It is a pivotal time to shape a different future  — one where we build AI we can trust, use with agency, and understand — rather than accept a future defined by opacity and control. Krikorian will lead Mozilla’s efforts to develop trustworthy and open source AI, ensuring Mozilla is inventing and building technology that pushes us all in the right direction. 

“We need to make sure AI and the internet belong to all of us, not just a few big companies,” said Mark Surman, President of Mozilla. “Raffi is the right person to lead that charge — creative, innovative and passionate about responsible technology. He puts wind in our sails, accelerating Mozilla’s mission of building technology based on the values in our Manifesto.”

Krikorian brings a record of impact across sectors to Mozilla, with roles spanning tech, politics, media and philanthropy. Raffi joins us from having been the CTO at Emerson Collective, where he led efforts to bring technologists into sectors like education, the environment, immigration, and economic mobility — and to help people in those sectors see themselves as technologists. He also hosted the “Technically Optimistic” podcast and Substack, exploring technology’s impact on society.

Prior to that, he was the first CTO of the US’s Democratic National Committee, where he used data, technology, and digital security to support the election processes of Democratic candidates up and down the ballot; Director of Uber’s Advanced Technologies Center, where he led the development and rollout of the first passenger-carrying self-driving car fleet; and Vice President of Platform Engineering at Twitter, where he managed and built Twitter’s global infrastructure. 

Krikorian has served on the Mozilla Foundation Board of Directors since 2023, on Mozilla.ai’s Board since 2024, and on the Mozilla.org Board since its inception.

“We don’t just need critiques on how AI is being built — we need real, working alternatives,” said Krikorian. “Mozilla is one of the few places that can actually do that. It’s built for this moment. I’m excited to join the team and help shape technology that reflects values we care about — transparency, openness, participation, and trust.”

The post Mozilla welcomes Raffi Krikorian as Chief Technology Officer appeared first on The Mozilla Blog.

This essay was originally published on The Sidebar, Mozilla’s Substack.

Trigger warning: Discussion of pregnancy loss.

Open up TikTok on any given day and you’ll find confident young women openly discussing their periods. They might recognise a low mood is due to the luteal phase or feel amazing because they are ovulating. They acknowledge the importance of self-care and that everybody is different. Periods are normal, but there’s no such thing as a normal period!

For me, a peri-menopausal woman who was surprised by my period every month until trying to get pregnant well into my 30s, this self-awareness and desire of young women to understand their own bodies and cycles is a source of great feminist pride. It is welcome progress from the shame and secrecy traditionally surrounding menstruation education.

A craving for knowledge and understanding has made cycle tracking apps very popular with women and girls from a young age. “I didn’t know about tracking apps until I was 17”, says Gen Z’er Sara*. Some users are keen to share cycle information with their partner and friends to encourage understanding around mood swings, or share funny stories about loud notifications alerting all to the consistency of their discharge today.

Cycle tracking apps are certainly having a moment. There are hundreds available on app stores, from simple calendars to full hormone testing. One study found that the top three cycle tracking apps dominating the market were downloaded 250 million times globally.

Often framed as simple period prediction tools that aid or avoid pregnancy, most usage of cycle tracking apps relates to everyday mental and physical wellbeing. By logging a wide range of symptoms and events, women and girls can track mood, energy levels, manage health conditions and work with their bodies. Apps can provide health and nutrition information as well as recommendations for exercise and meditation.

Cycle tracking apps are also the poster girl of what is going wrong with women’s rights today. The dark side of cycle tracking apps are just that — the tracking. The two-fold threat of increased reproductive surveillance and the sale of sensitive data for advertising is in danger of outweighing the benefits of trusting these apps with information about our bodies and state of mind.

It has been three years since the overturning of Roe v. Wade, which ended the federal protection for abortion in the U.S. Many experts at the time urged women to delete cycle tracking apps in the face of the threat of tracking data being used to “prove” an abortion via a missed period. But is this actually what women are doing? Or, are they finding alternative solutions, workarounds, and new ways to engage with these tools?

Apps that were ahead of the game in terms of a good privacy commitment and track record saw a benefit. Ana Ramirez, co-executive director of Euki, a cycle tracking app and information service remembers, “Our largest download surges were immediately after Roe fell, when Euki received media coverage as a standout privacy period tracker…”

Amy Thompson, founder of the Moody Month app focused on improving mental health observed, “Moody, like much of the industry, saw users delete apps post-Roe v. Wade, but increased media coverage also drove new downloads.”

So while many did delete, tracking apps were not abandoned entirely. Ana explains, “People are hungry for a way to track their sexual and reproductive health without giving up their privacy. In this political climate, period trackers aren’t just tools — they’re lifelines.”

Concerns reached the U.K. where increased surveillance, investigation and prosecution of women suspected of “illegal” abortions after pregnancy loss added to a climate of fear, mistrust and misinformation. Azure*, 16, suspected of the app she uses, “It’s dodgy and sends information to the government. It tracks to see if you’ve had an illegal abortion or something.”

Chella Quint, founder of Period Positive and author of Own Your Period saw increased awareness of data privacy risks reaching many U.K. Gen Z’ers in the wake of Roe v. Wade being overturned which affected their use of cycle tracking apps, “Some people now use workarounds like avoiding login, using a Notes app instead, or tracking their cycles in a paper diary. Others — both adults and children — are aware of the risks but still find the apps too useful to give up.”

Rose*, 19, agreed “I think most [people] are aware of the issues surrounding privacy and data collection…it’s just a bit overwhelming and I’ve kind of just accepted that it will happen in one way or another.”

High-profile cases in the U.S. related to data breaches and apps selling or sharing customer data with third parties also led users to seek out privacy respecting alternatives, which pushed companies to raise the bar for stricter privacy standards and win back user trust.

For 20-year-old Ella* based in the U.K., a concern over the sale of data by an app in the U.S. prompted a switch of trackers to another one based in Europe that promoted its privacy credentials. Another popular reason to switch is discovering the company behind an app is owned by men — female-founded companies are preferred. While switching apps is common, it is a source of frustration as the data cannot be ported to another app and years of tracking data can be lost. In a competitive market, some apps have capitalised on this by offering to port all the data from another app if the user sends screenshots of calendars.

At a bare minimum cycle tracking apps should not be selling data. But the difference between selling and sharing gets blurry. If it’s in the cloud, it’s being shared with the cloud provider. If advertising on social media, it’s being shared with the platform. Synching with a wearable? It’s being shared. It’s almost impossible to get around this in the digital economy. And the law barely keeps up as Amy Thompson from Moody Month observed, “While anonymisation/pseudonymisation frameworks exist in major privacy laws, enforcement and implementation standards often lag behind evolving privacy risks.”

It can be difficult to comprehend why anyone would be so interested in this info in the first place. As Nat*, 42 from the UK said, “who cares if all the data leaks, who cares if anyone knows my periods, when I have sex, when I go swimming etc?”

A report by the Minderoo Centre for Democracy and Technology outlines the value of knowing if someone is trying to become pregnant as pregnancy is a life event that drastically changes shopping habits. Cycle-based advertising seeks to tailor adverts based on menstrual cycle phases, suggesting that hormone fluctuations can make people susceptible to products at different times, such as clothing and cosmetics in the first half of the cycle when women might be ovulating and feeling good. Either way, someone is commodifying aspects of your body while you are in the process of trying to understand it.

Some may not care deeply about this. For those more likely to be surveilled or face barriers to care — such as people of colour, young people, people on welfare, those living in restrictive U.S. states, menstruating people who do not identify as cis women — they are more guarded about their privacy and what happens to their data. Ana Ramirez from Euki believes these are the people who should be at the forefront in developer’s minds, “Power lies in designing with — not just for — the people most often dismissed as ‘edge cases.’ Centering privacy starts with centering those most impacted.”

Women have always been watched but that is not going to stop us trying to understand what is going on with our bodies. We’ll find workarounds and seek out high privacy standards. We may track our periods for fertility reasons, but there is so much more to it than that. Which makes sense because we, as women and human beings, are much more than that.

*Names have been changed

Lucy Purdon is the founder of Courage Everywhere, a consultancy advising organisations on the responsible development of technology to advance human rights, democracy and gender justice. Lucy has provided strategic advice, policy development and original research for organisations ranging from tech startups to the United Nations. She has worked in civil society roles for over 13 years, including as a Senior Tech Policy Fellow at Mozilla Foundation. She writes the weekly newsletter The Prompt, analysing the latest tech news and trends.

Übernimm die Kontrolle über dein Internet.

Firefox herunterladen

Join us in sharing your act of daily defiance using #dailydefiance and tagging @firefox on social channels to be a part of our movement and remind the world that everyday choices do matter, and we should all choose boldly.

Firefox will be sharing some of the entries with our community and creating mini-challenges for people to follow along. Join us on Instagram, TikTok, Threads, Bluesky and Substack.

The post Young people are outsmarting period tracking apps appeared first on The Mozilla Blog.

This year, we’ve built on that foundation and delivered major performance improvements and mobile support, ensuring more Firefox users benefit from privacy without compromise.

Introducing DoH for Android

After bringing encrypted DNS protection to millions of desktop users, we’re now extending the same to mobile. Firefox users who have been waiting for DoH on Android can now turn it on and browse with the same privacy protections as on their desktops.

Starting with this week’s release of Firefox 143 for Android, users can choose to enable DoH in Firefox on their mobile devices by selecting “Increased Protection” DoH configuration. Performance testing with Firefox DoH partners is currently underway. If DoH is as fast as we expect, we plan to enable it by default for Android users in certain regions, similar to desktop users. Until then, these configuration options provide you the choice to opt in early.

<figcaption class="wp-element-caption">Enable DoH in Firefox on Android</figcaption>

DoH performance breakthroughs in 2025

DNS resolution speed is critical to the browsing experience — when web pages involve multiple DNS queries, the speed difference compounds and can cause page loads to be slow. Since we first rolled out DoH in Canada, we’ve worked closely with CIRA for reliability and performance measurements. Through our strong collaboration with them and their technology partner Akamai, Firefox DoH lookups are now 61% faster year-to-date for the 75th percentile.

With these performance improvements, DoH resolution time is now within a millisecond or two of native DNS resolution. This is a big win because Firefox users in Canada now get the privacy of encrypted DNS with no performance penalty.

Although the investigation and analysis started with the desire to improve DoH in Firefox, the benefits didn’t end there. Our collaboration also improved CIRA DoH performance for many of its DNS users, including Canadian universities, as well as other DNS providers relying on CIRA’s or Akamai’s server implementations.

This is a win not just for Firefox users, but for the many other users around the globe.

Robust privacy on your terms

We have always approached DoH with an emphasis on transparency, user choice, and strong privacy safeguards. Firefox gives users meaningful control over how their DNS traffic is handled: Users can opt out, choose their own resolver, or adjust DoH protection levels, and Firefox makes it clear what DoH is doing and why it matters.

Firefox enforces strict requirements for DNS resolvers before trusting them with your browsing. Not every DNS provider can become a DoH provider in Firefox — only those that meet and attest to Mozilla’s rigorous Trusted Recursive Resolver (TRR) policy through a legally binding contract.

Prioritizing your privacy and speed

Our work with DoH this year shows what’s possible when privacy and performance go hand-in-hand. We’ve proven that encrypted DNS can be fast, reliable, and available on desktop and Android. Just as importantly, we’ve shown that partnerships grounded in open standards and accountability can deliver benefits not only to Firefox users but to the wider internet.

As we look forward, our commitment stays the same: Privacy should be the default, speed should never be a compromise, and the web should remain open and accessible to everyone. Choosing Firefox means choosing a browser that is built for you and for a better internet.

Übernimm die Kontrolle über dein Internet.

Firefox herunterladen

The post Firefox DNS privacy: Faster than ever, now on Android appeared first on The Mozilla Blog.

How it works

Whether you just want the recipe, need to know something fast, or want to see if a long read is worth the time, Shake to Summarize gives you the key takeaways in seconds. To activate it, you can:

• Shake your device.
• Tap the thunderbolt icon in the address bar.
• Or, from the menu, tap three dots > Summarize Page.

The feature works on webpages with fewer than 5,000 words. (Learn more about content you can summarize here.) 

Here’s an example of a summary:

If you have an iPhone 15 Pro or later with iOS 26+, the summary is created on your device using Apple Intelligence. On other devices with earlier iOS versions, the page text is sent securely to Mozilla cloud-based AI, which creates the summary and sends it back. 

Rollout starts Sept. 9

Shake to Summarize starts rolling out this week in the U.S. for English-language Firefox iOS users, then expands from there. 

You’ll see a prompt the first time you come across content that can be summarized, and you can turn the feature on or off in settings anytime.

Summarize a page on desktop

You can also summarize pages in Firefox on desktop with your choice of chatbot provider:

• Select Summarize Page at the bottom of the chatbot sidebar.
• Or hold down Control while you click, then choose Ask [chatbot name] > Summarize Page.

See more information here.

Designed for user choice

Sometimes you want the whole story. Sometimes you just need the highlights. Firefox gives you both and leaves the choice to you. 

Let us know what you think once you give it a try.

Take control of your internet

Download Firefox

The post On Firefox for iOS, summarize a page with a shake or a tap appeared first on The Mozilla Blog.

Last week, Judge Amit Mehta issued a ruling in the remedies proceedings of the U.S. v. Google LLC search case. Among the issues addressed, one key aspect that stood out for us was the court’s ruling on Google’s search agreements. 

The Court ordered changes to Google’s search agreements to give browsers more flexibility. Under the court’s decision, Google cannot restrict browsers from defaulting to or offering different search engines / generative AI services. They can also not prevent browsers from promoting these services.   

Crucially, the Court considered but ultimately rejected a proposed ban on search payments to small, independent browsers like Firefox. If the ban had been enforced, it would have made it harder for independent browsers to compete, effectively reducing competition in the browser market. 

In his reasoning, Judge Mehta cited Mozilla’s testimony, recognizing that banning payments to smaller browsers would harm innovation, competition, and consumers, and would threaten the pro-competitive role of Mozilla in the ecosystem. Ensuring that Mozilla’s Gecko — the only independent browser engine left — can continue to compete with Google and Apple is vital for the future of the open web.

The court also required a range of data sharing remedies — narrowing the scope of those proposed by the Department of Justice and State Attorneys General, while broadening their access. As Mozilla has discovered first-hand through previous antitrust cases and the implementation of the EU Digital Markets Act, ensuring that such remedies are effective in restoring competition requires careful attention and monitoring. Careful thought must also be given to protecting user privacy and security.

It will also be critical to ensure that these data remedies avoid simply transferring power from one tech giant to another — particularly given the focus on facilitating greater search competition through AI providers.  

This balance is something we’ve stressed throughout the trial. True competition in search starts with a healthy marketplace, one where small and large companies can compete on merit, where consumers have choice, and where the best new products, features, and ideas have a chance. 

As this case continues to unfold, one thing won’t change: Mozilla’s commitment to an internet that’s open, accessible, and built for the public good. We’ve historically met market and regulatory shifts with creativity and care. Each moment has helped us grow and discover new ways to live out our mission, and we’re invigorated about the path forward. 

The post Defending an open web: What the Google search ruling means for the future appeared first on The Mozilla Blog.

This post explains how we accelerated inference by replacing the default onnxruntime‑web that powers Transformers.js with its native C++ counterpart that now lives inside Firefox.

Where we started

Transformers.js is the JavaScript counterpart to Hugging Face’s Python library. Under the hood it relies on onnxruntime‑web, a WebAssembly (WASM) build of ONNX Runtime.

A typical inference cycle:

1. Pre‑processing in JavaScript (tokenization, tensor shaping)
2. Model execution in WASM
3. Post‑processing back in JavaScript

Even with warm caches, that dance crosses multiple layers. The real hotspot is the matrix multiplications, implemented with generic SIMD when running on CPU.

Why plain WASM wasn’t enough

WASM SIMD is great, but it can’t beat hardware‑specific instructions such as NEON on Apple Silicon or AVX‑512 on modern Intel chips.

Firefox Translations (uses Bergamot) already proves that diving to native code speeds things up: it uses WASM built‑ins which are small hooks that let WASM call into C++ compiled with those intrinsics. The project, nicknamed gemmology, works brilliantly.

We tried porting that trick to ONNX, but the huge number of operators made a one‑by‑one rewrite unmaintainable. And each cold start still paid the JS/WASM warm‑up tax.

Switching to ONNX C++

Transformers.js talks to ONNX Runtime through a tiny surface. It creates a session, pushes a Tensor, and pulls a result. It makes it simple to swap the backend without touching feature code.

Our steps to achieve this were:

1. Vendor ONNX Runtime C++ directly into the Firefox tree.
2. Expose it to JavaScript via a thin WebIDL layer.
3. Wire Transformers.js to the new backend.

From the perspective of a feature like PDF alt‑text, nothing changed, it still calls await pipeline(…). Underneath, tensors now go straight to native code.

Integration of ONNX Runtime to the build system

Upstream ONNX runtime does not support all of our build configuration, and it’s a large amount of code. As a consequence we chose not to add it in-tree. Instead, a configuration flag can be used to provide a compiled version of the ONNX runtime. It is eventually automatically downloaded from Taskcluster (where we build it for a selection of supported configuration) or provided by downstream developers. This provides flexibility while not slowing down our usual build and requiring low maintenance.

Building ONNX on Taskcluster required some configuration changes and upstream patches. The goal was to find a balance between speed and binary size, while being compatible with native code requirements from the Firefox repo. 

Most notably:

• Building without exception and RTTI support required some patches upstream
• Default build configuration is set to MinSizeRel, compilation uses LTO

The payoff

Because the native backend is a drop‑in replacement, we can enable it feature by feature and gather real‑world numbers. Early benchmarks shows from 2 to 10 × faster inference, with zero WASM warm‑up overhead.

For example, the Smart Tab Grouping topic suggestion, which can be laggy on first run, is now quite snappy, and this is the first feature we gradually moved to this backend for Firefox 142.

The image to text model used for PDF.js alt-text feature also benefited from this change. On the same hardware the latency went from from 3.5s to 350ms.

What’s next

We’re gradually rolling out this new backend to additional features throughout the summer, so all capabilities built on Transformers.js can take advantage of it. 

And with the C++ API at hand, we’re planning to tackle a few long‑standing pain points, and enable GPU.

Those changes will ship in our vendored ONNX Runtime and offer us the best possible performance for Transformers.js-based features in our runtime in the future.

1. DequantizeLinear goes multi‑threaded

The DequantizeLinear operation is single‑threaded and often dominated inference time. While upstream work recently merged an improvement (PR #24818), we built a patch to spread the work across cores, letting the compiler auto‑vectorize the inner loops. The result is an almost linear speedup, especially on machines with many cores.

2. Matrix transposition goes multi-threaded

Similarly, it is typical to have to transpose very large (multiple dozen megabytes) matrices when performing an inference task. This operation was done naively with nested for loops. Switching to a multi-threaded cache-aware tiled transposition scheme, and leveraging SIMD allowed to take advantage of modern hardware and speed up this operation by a supra-linear factor, typically twice the number of threads allocated to this task, for example a 8x speedup using 4 threads.

This can be explained by the fact that the naive for loop was auto-vectorized, but otherwise did poor usage of CPU caches.

3. Caching the compiled graph

Before an inference can run, ONNX Runtime compiles the model graph for the current platform. On large models such as Qwen 2.5 0.5B this can cost up to five seconds every launch. 

We can cache the compiled graph separately from the weights on the fly, shaving anywhere from a few milliseconds to the full five seconds.

4. Using GPUs

Currently, we’ve integrated only CPU-based providers. The next step is to support GPU-accelerated ONNX backends, which will require more effort. This is because GPU support demands additional sandboxing to safely and securely interact with the underlying hardware.

Conclusion

What is interesting about this migration is the fact that we could improve performance that much, while migrating features gradually, and all that in complete isolation, without having to change any feature code.

While the speed ups are already visible from a UX standpoint, we believe that a lot of improvement can and will happen in the future, further improving the efficiency of the ML-based features, and making them more accessible to a wider audience.

Have ideas, questions or bug reports? Ping us on Discord in the firefox-ai channel (https://discord.gg/TBZXDKnz) or file an issue on Bugzilla, we’re all ears.

The post Speeding up Firefox Local AI Runtime appeared first on The Mozilla Blog.