The browser has evolved into the de facto operating system for business applications, processing and storing vast amounts of sensitive data, like passwords, authentication tokens, and session cookies. Consumer browsers take a limited approach to securing this data, either leaving it unencrypted or only encrypting it when stored to disk.

The Island Enterprise Browser was designed to protect users' sensitive data both on disk and in memory - including the physical memory. This physical memory attack vector poses a particularly dangerous risk to enterprises: when end-user devices fall into the wrong hands, whether compromised by malware or accessed by malicious users, attackers can steal secrets stored in the browser's memory using memory dump tools.

In this article, I'll share how we resolved sensitive data leaks in browser memory - exploring kernel-level memory acquisition technique, and sensitive memory leak detection approaches - ultimately developing a two-pronged solution combining targeted memory deallocation hooks with an automated validation framework.‍

Our Initial Discovery

One of our first findings was that even after a plain-text cookie was used, it remained in memory unencrypted rather than being securely encrypted or zeroed out. We began our investigation by focusing on HTTP-only cookies, since these cookies are inaccessible to JavaScript. Our basic approach was to encrypt the cookies as soon as they were received from the server and decrypt them only when they needed to be sent back. The rest of Chromium's code would treat the encrypted cookies as a black box, which reduced the amount of changes we needed to make to support cookie encryption.

There are windows of time where the encrypted cookies need to be decrypted for use (for example, when sending them to the server) and we must wipe the plain cookies from memory after we're done with them. To make sure this approach was bulletproof, each time we decrypted a cookie we wrapped the result in a SensitiveString class, which securely zeroes the memory upon destruction. This lowered the likelihood of obtaining a plain cookie if an attacker captured a memory snapshot.

Following this first effort, we ran the browser through different attack scenarios in our security labs, where we continuously test the security mechanisms our browser provides for protecting enterprise organizations. We encountered a unique challenge - our findings report indicated that if an attacker used a physical memory dump tool like Winpmem, they could locate cookies stored in plain text by dumping the machine's physical memory and searching for the cookie name. This was surprising, since we had wrapped every usage of unencrypted cookies with SensitiveString - in theory, there shouldn't have been any plain cookies left.

At the moment, all we knew was that even after a plain-text cookie is deleted, it remains in memory in an unencrypted state, rather than being securely encrypted or zeroed out. What follows is the journey we took to address this issue. 

So, What Are Our Options?

Considering this analysis, we considered three potential approaches:

1. Find all the leaks - identify and analyze every location where plain cookies remained to understand their origins and causes. However, this process was like searching for a needle in a haystack. Additionally, since the Island browser is based on Chromium, a massive project with numerous contributors, the challenge was even greater – even if we managed to fix all identified leaks, there’s no guarantee that new code wouldn't reintroduce the same issue.
2. Leverage a kernel driver to protect the Island browser process from physical memory dumps. This approach would prevent attackers from dumping the machine's memory to capture Island's sensitive data, such as cookies, even if they remained stored in memory as plain text.
3. Leverage a kernel driver to block drivers that expose physical memory access. This approach had significant limitations. We are not an antivirus company, and even if we were, maintaining a comprehensive blocklist would be extremely challenging. Such lists are often incomplete and prone to gaps. Furthermore, in enterprise environments, blocking drivers can result in compatibility issues, especially when dealing with hardware-based drivers that unintentionally expose physical memory access.

After ruling out Option 3 due to its numerous drawbacks, we considered Option 2, which involved preventing tools like Winpmem from dumping the machine's physical memory by analyzing and mitigating the specific techniques they employ. To assess the feasibility of this approach, we needed to dive deep into the internals of Winpmem to evaluate how difficult it would be to exclude Island from memory dumps.

Winpmem Explained

Winpmem is a forensic memory acquisition tool designed to capture the physical memory from Windows systems, primarily used in digital forensics and incident response. It operates through a kernel driver to ensure reliable and consistent memory acquisition. Winpmem employs several techniques for memory acquisition, all of which needed to be addressed to prevent dumping sensitive data from Island. If any technique was overlooked, attackers could exploit it to successfully dump physical memory.

One of Winpmem's stronger techniques is "PTE remapping", which involves direct manipulation of x86's virtual memory structures. To understand this technique, let's quickly review the basics of x86 virtual memory:

Virtual memory is a memory management system that combines physical memory and disk space to create the illusion of a large, continuous address space for processes. This enables efficient multitasking, memory isolation, and the execution of applications larger than the available RAM.

The system relies on page tables - hierarchical data structures maintained by the operating system - to map virtual memory addresses to their corresponding physical locations. Each entry in a page table, known as a Page Table Entry (PTE), contains essential information such as the physical frame address, access permissions, and status flags (e.g., present, dirty, or no-execute).

The memory management unit (MMU) uses the page table during address translation to ensure secure and transparent access to memory. This process also supports advanced features like paging, shared memory, and memory protection while allowing the system to handle page faults and allocate memory efficiently.

On x64 Windows, the memory paging mechanism is managed by four specific tables:

• Page Map Level 4 (PML4)
• Page Directory Pointer Table (PDPT)
• Page Directory Table (PDT)
• Page Table (PT)

These tables work in a hierarchical structure, where each table contains indexes that direct to the start of the next level of paging structures. Each of these structures has 512 entries. The indexes are known as Page Frame Numbers (PFN), the last PFN leads to the physical address.

In general, the operating system creates the tables and places the physical address of the PML4 into the CR3 register. This is how the CPU determines the address of the first table. Below is a simplified (partial) flow of the address translation process:

You might be wondering why the CR3 register appears early in the illustration. This is because the diagram simplifies the actual address translation process. In practice, CR3 holds the base address of the PML4 table and is implicitly used by the CPU to begin virtual-to-physical address translation. TLBs (Translation Lookaside Buffers) store cached translations that are keyed by the CR3 value, ensuring that translations remain isolated per process. On a TLB miss, the CPU walks the full page table hierarchy, and using the final PTE’s PFN (Page Frame Number), it computes the corresponding physical address.

Now that we’ve covered the foundational concepts, let’s dive into PTE remapping.

PTE Remapping Explained

PTE remapping involves changing the PFN (Page Frame Number) of a PTE (Page Table Entry) to the physical address we want to scan and then fetching the data. This process involves iterating through the tables and modifying the final data structure involved in address translation. While certain preparations are required for implementation, this is the core concept behind PTE remapping.

The implementation of PTE remapping is typically divided into two parts. First, a custom PTE controlled by the driver must be located, created, or crafted. Then, its PFN is modified.

Let’s take a look at winpmem PTE remapping implementation. As mentioned earlier, the first step in this process is to identify a PTE that Winpmem can manipulate. The relevant function for this is setupBackupForOriginalRouguePage.

The function is well-documented, but here’s a quick overview: Its primary purpose is to prepare the PTE remapping. The output parameter, PPTE_METHOD_DATA, will contain the necessary information required for the remapping process.

Winpmem creates a custom section that acts as the "magic page." This section is defined with nopage and nocache attributes, which are crucial because Winpmem modifies the Page Frame Number (PFN) under the memory manager. These attributes ensure that the section is neither cached nor paged out.

You might wonder about handling large or huge pages. Winpmem assumes they will be split into 4K pages, but this creates a dependency: the virt_find_pte function will fail if the Page Table Entry (PTE) is not 4K. The virt_find_pte function reads the CR3 register and iterates over the page tables until it locates the final PTE. Once the PTE value is retrieved, Winpmem backs up the current PFN value, disables caching for this specific entry (since the PFN will change continuously during scanning), and finally flushes the corresponding entry in the TLB to remove the current cache entry.

The next part of the implementation is relatively straightforward. A physical address to be read is chosen, the PTE is updated accordingly, and the virtual memory address now points to the target physical address, allowing its contents to be read.

Given how PTE remapping works, how can we address it? Running Island inside an enclave isn’t feasible, and developing hypervisors to handle EPT violations is not cost-effective. While PTE remapping involves manipulating low-level structures, even with kernel-level protection, it would be excessive to implement just to avoid this issue.

To sum up, we see Winpmem doesn't use any special OS APIs here, but instead it directly utilizes low level CPU structures. Because of this, there's no simple way to exclude Island's memory from the dump, even if we have our own kernel driver.

This led us to pivot to option No. 1 - plugging the leaks by ensuring all unencrypted cookies were properly erased after usage.

Hunting For Leaks

To address the issue, our first step was identifying the source of the leaks. We began examining the cookie handling flow and discovered that cookies were often processed in plaintext. This pointed to overlooked areas in the code - particularly those involving direct manipulation of cookie strings. As a result, even though we had made substantial progress in clearing the decrypted plain text cookies when necessary, it became clear that our efforts weren’t enough.

In Figure 5 above, the cookie remains in memory as plain text, and string concatenation creates a temporary string that could be exposed (given the context of handling sensitive plain text data). Even if we erased the string after sending the cookie, these kinds of functions would need to be carefully managed to prevent leaks.

To prove this theory, we created a wrapper string that overrides the default deallocator to print the string being freed. As expected, the cookie was deallocated in plain text.

At this point, I realized that manually identifying these leaks wasn't practical. Mapping out all the functions and analyzing the entire project would be highly inefficient.

The Black Box Method

Then, I had an idea: what if we approached this as a black-box issue? This was just my third task at Island, so it was quite a leap into the deep Chromium waters.
I initially thought to map all flows dealing with cookies – but this was not cost-effective. Instead, I decided to take an HTTP-only cookie that would be easy to locate in a memory dump, such as SSID=<cookie-value> from my Google account. I reproduced the issue at the start of the task by logging into my Google account, dumping the entire physical memory using Winpmem, and searching for SSID= in the dump.

To streamline the process, I created an executable that injects a DLL into all Island processes. This DLL hooks into all Windows deallocation-related methods. Initially, I focused on functions like VirtualFreeEx, GlobalReAlloc, GlobalFree, LocalReAlloc, LocalFree, HeapReAlloc, and HeapFree.

Before each deallocation, I retrieved the size of the allocation and checked for the presence of the cookie. To avoid false positives, it was not enough to simply verify that the cookie contained "SSID="; I also needed to ensure it does not contain "SSID=ILIMEC". This is because, if the cookie is encrypted, the "ILIMEC" prefix indicates that the data after the prefix is encrypted using Island's encryption mechanism. In other words, if a deallocation contains "SSID=" but the value does not start with "ILIMEC", we trigger an int3 breakpoint. This allows us to capture the context and investigate the potential leak.

I started with a clean PC running Island, configured cookie encryption, and launched the injector to inject the hooking DLL into all Island processes. Then, I attached a debugger and logged into my Google account. To my surprise, it worked – but not quite as I expected. While I hit some breakpoints, nearly all of them originated from the same place: PartitionAlloc.
This was progress. We have a lead. But now the question is, what exactly is PartitionAlloc?

PartitionAlloc is a memory allocator designed and used by Chromium. You can read more about its design and rationale here. 

It turns out, Chromium implemented their own memory allocator, which explains why the breakpoints were coming from PartitionAlloc. However, this allocator doesn't necessarily free memory immediately when a string is deallocated, which is why the call stack didn’t match my expectations. To address this, we applied the same approach as before. We modified the PartitionAlloc deallocation API in the source code to include our previously mentioned cookie scanning mechanism and ran the same test. And the results were remarkable: Over 20 breakpoints were identified, scattered across various locations and processes in the project, and we were able to fix all of them. After performing another Winpmem scan, I found no leftover cookies. Mission accomplished.

We developed an efficient method to identify plain cookie leaks, especially when context is needed for analysis. But we needed to be sure the fix worked consistently and wouldn’t break in future versions of Chromium. There were several approaches to testing it. Here’s an overview of the one we decided to implement.

Automating In-Memory Cookie Encryption Validation

Let’s go step by step:

1. The browser initiates an HTTP GET request to an Island test web server.
2. The server generates a random cookie and, in addition, creates a xored version of the cookie. The first character of the xored cookie represents the XOR key, while the remaining characters form the xored cookie itself. You might be wondering why the xored cookie is necessary. If the cookie weren’t xored and the server simply sent back two cookies, one plain and the other HTTP-only (and therefore encrypted), the plain cookie would contaminate the entire Windows machine. Once the cookie exists in unencrypted form within memory, multiple instances of it would be scattered across physical memory.
3. The server responds to the client with a 200-OK status and includes a Set-Cookie header that contains both cookies.
4. The Island browser processes the Set-Cookie header and encrypts the cookie in memory. From this point, the cookie will not exist in plain text in the physical memory of the Windows machine, except when temporarily decrypted for use. Once decrypted, any traces of the cookie in memory are erased, unless a memory leak occurs. However, the xored cookie remains in plain text because it is not sent as an HttpOnly cookie.
5. The automation fetches the cookies from the browser, it gets the encrypted version of the cookie and the xored cookie.
6. The automation initiates web requests from the browser to the server, ensuring the cookie is included. Simultaneously, it deploys a custom memory scanner. The memory scanner gets the xored cookie and not the “decrypted” version because it will scan the physical memory and if we pass the scanner the plain cookie it will find the plain cookie in the physical memory the test won’t be valid.

‍

Initially, we used Winpmem to create a dump file of the physical memory. However, dumping 32GB of memory and writing it to a file on disk proved to be a slow operation, which is unacceptable for our CI pipeline at Island. After spending two hours reviewing Winpmem and its signed driver, I discovered that the driver itself doesn’t write data to a file; the user-mode client handles that. So instead, we developed our own Winpmem client, MemoryPatternSearcher, which loads the Winpmem driver, communicates with the device object created by the kernel driver, and starts by fetching a structure named WINPMEM_MEMORY_INFO.

Reading the entire physical memory can cause your PC to crash for various reasons, the most common being memory-mapped I/O (MMIO). Winpmem overcomes this issue by utilizing the well-known, undocumented, exported, ntoskrnl kernel function MmGetPhysicalMemoryRanges to identify memory ranges that are safe to read, thereby avoiding a BSOD.

MemoryPatternSearcher accepts two command-line arguments: the first is the XOR key, and the second is the xored pattern itself. Within MemoryPatternSearcher, we don’t directly use the XOR key to decrypt the cookie all at once. Instead, we extract each character in plaintext individually. By doing so, the machine remains uncontaminated, and cookie encryption tests can be executed without requiring a machine restart. Since the data must be compared locally, the memory scanner client and the additional XORed cookie are essential.

Over time, it identified several changes in the area where the test began to fail, and the first technique made it easy to understand the causes for these failures.

With these two techniques combined, we successfully provided enterprises with an effective solution for protecting their cookies from attackers using advanced memory dumping tools.

Thanks for sticking with me until the end. This is just a small glimpse into life as a software engineer at Island. So feel free to check out the Island Internals blog for more Island adventures.

—-

‍

The Island Enterprise Browser fundamentally transforms how the world’s leading organizations work by embedding enterprise-grade IT, security, network controls, data protections, app access, and productivity enhancements directly into the browser itself, enabling secure access to any application, protecting sensitive data, streamlining IT operations, and delivering a superior end-user experience while actually boosting productivity.

To learn more about how we're reimagining the enterprise workspace from the browser up, start here.

If you’re interested in building something that’s changing everything, check out our open positions here.