How do I use memory mapped files with FileChannel.map()?

Using FileChannel.map() allows you to map a region of a file directly into memory. This creates a MappedByteBuffer, which acts like a bridge between your application’s memory and the file on disk. The operating system handles the actual reading and writing in the background, making it extremely efficient for large files.

Here is how you can use it for both reading and writing.

1. Reading from a Memory-Mapped File

To read, open the channel with StandardOpenOption.READ and use MapMode.READ_ONLY.

import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class MemoryMappedExample {
    public void readMappedFile(Path path) throws IOException {
        try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
            long size = channel.size();

            // Map the entire file for reading
            MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, size);

            // Access data directly from memory
            while (buffer.hasRemaining()) {
                byte b = buffer.get();
                // Process byte...
            }
        }
    }
}

2. Writing to a Memory-Mapped File

To write, you must open the channel with both READ and WRITE options (even if you only intend to write) and use MapMode.READ_WRITE.

public void writeMappedFile(Path path) throws IOException {
    // Files must be opened for both READ and WRITE to use MapMode.READ_WRITE
    try (FileChannel channel = FileChannel.open(path, 
            StandardOpenOption.READ, 
            StandardOpenOption.WRITE, 
            StandardOpenOption.CREATE)) {

        long size = 1024 * 1024; // Map 1MB
        MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_WRITE, 0, size);

        // Writing to the buffer automatically writes to the file
        buffer.putInt(12345);
        buffer.put("Hello Memory!".getBytes());

        // Force changes to storage to ensure they are written to disk
        buffer.force();
    }
}

Key Considerations

  • Map Modes:
    • READ_ONLY: Any attempt to modify the buffer results in a ReadOnlyBufferException.
    • READ_WRITE: Changes to the buffer are eventually propagated to the file.
    • PRIVATE: “Copy-on-write” mode. Changes are local to the buffer and not saved to the file.
  • Size Limits: On 32-bit JVMs, you cannot map more than 2GB at once because of address space limits. On 64-bit systems, you can map much larger regions, but a single MappedByteBuffer is still limited to Integer.MAX_VALUE bytes (approx 2GB). To handle larger files, you must create multiple mappings.
  • Performance: Memory mapping is most beneficial for large files accessed frequently or randomly. For small, sequential reads, standard BufferedInputStream might be simpler and just as fast.
  • Unmapping: Java does not provide an explicit “unmap” method. The mapping remains until the MappedByteBuffer object is garbage collected. Closing the FileChannel does not unmap the file.

How do I use FileChannel for efficient file IO?

Using FileChannel from the java.nio.channels package is a powerful way to perform high-performance file operations. It allows for advanced features like memory-mapped files and direct transfer between channels, which are often much faster than traditional stream-based I/O.

Here are the most efficient ways to use FileChannel.

1. Fast File Copying with transferTo or transferFrom

This is arguably the most efficient way to copy files. It uses “zero-copy” technology, where the operating system transfers data directly from the file system cache to the target channel without copying it into application memory (the heap).

package org.kodejava.nio;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.nio.channels.FileChannel;
import java.io.IOException;
import java.io.File;

public class FastCopy {
    public static void copyFile(File source, File dest) throws IOException {
        try (FileChannel sourceChannel = new FileInputStream(source).getChannel();
             FileChannel destChannel = new FileOutputStream(dest).getChannel()) {

            long position = 0;
            long count = sourceChannel.size();

            // Transfer data directly between channels
            sourceChannel.transferTo(position, count, destChannel);
        }
    }
}

2. Reading/Writing with ByteBuffer

FileChannel works with ByteBuffer. For maximum efficiency, use Direct Buffers (ByteBuffer.allocateDirect()). Direct buffers are allocated outside the standard JVM heap, allowing the OS to perform I/O operations directly on the memory.

package org.kodejava.nio;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class EfficientRead {
    public void readWithBuffer(Path path) throws IOException {
        try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
            // Use a direct buffer for better performance with OS I/O
            ByteBuffer buffer = ByteBuffer.allocateDirect(1024 * 8); // 8KB

            while (channel.read(buffer) != -1) {
                buffer.flip(); // Prepare buffer for reading

                // Process the data...
                // while(buffer.hasRemaining()) { System.out.print((char) buffer.get()); }

                buffer.clear(); // Prepare buffer for writing (reading from channel)
            }
        }
    }
}

3. Memory-Mapped Files (MappedByteBuffer)

For very large files, memory mapping is often the fastest approach. It maps a region of the file directly into virtual memory. The OS handles loading the data from disk as you access it.

package org.kodejava.nio;

import java.io.IOException;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.StandardOpenOption;

public class MemoryMappedExample {
    public void mapLargeFile(Path path) throws IOException {
        try (FileChannel channel = FileChannel.open(path, StandardOpenOption.READ)) {
            long size = channel.size();
            // Map the entire file into memory
            MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_ONLY, 0, size);

            if (buffer.hasRemaining()) {
                // You can access data like an array without calling read()
                byte firstByte = buffer.get(0);
            }
        }
    }
}

Key Tips for Efficiency:

  • Use try-with-resources: FileChannel implements AutoCloseable. Always ensure it is closed to release file locks and native resources.
  • Direct Buffers: Use ByteBuffer.allocateDirect() if the buffer is long-lived or used for heavy I/O, but remember that allocating/deallocating them is more expensive than heap buffers.
  • File Locks: FileChannel provides lock() and tryLock() methods, which are useful for synchronizing file access between different JVM processes.
  • StandardOpenOption: When opening a channel via FileChannel.open(), use specific options like READ, WRITE, CREATE, or SPARSE to hint at your intentions to the OS.

How do I read last n characters from a file?

In the following post you will learn how to read last n characters from a file. The JDK 7 introduces a new SeekableByteChannel interface which allows its implementation to change the position and the size of the byte channel. One of its implementation is the FileChannel class (java.nio.channels.FileChannel).

The FileChannel class make it possible to get hold the current position of where we are going to read from or write to a file. The code snippet below shows you how you can read the last 1000 characters from a log file.

package org.kodejava.io;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.nio.file.StandardOpenOption;

public class FileReadLastNCharacters {
    public static void main(String[] args) {
        // Defines the path to the log file and creates a ByteBuffer.
        Path logPath = Paths.get("C:/tools/apache-tomcat-10.0.11/logs/catalina.out");
        ByteBuffer buffer = ByteBuffer.allocate(1024);

        try {
            // Creates FileChannel and open the file channel for read access.
            FileChannel channel = FileChannel.open(logPath, StandardOpenOption.READ);

            // Read a sequence of bytes from the channel into the buffer starting
            // at given file position, which is the channel size - 1000. Because
            // we are going to read the last 1000 characters from the file.
            channel.read(buffer, channel.size() - 1000);
            System.out.println("Characters = " + new String(buffer.array()));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

The steps in the code snippet above are:

  • Get the path to the log file.
  • Create a ByteBuffer, a buffer where the read bytes to be transferred.
  • Using the FileChannel.open() method we open a file to be read and return a FileChannel object.
  • The read() method of the FileChannel reads a sequence of bytes from the channel and transfer them to the given buffer starting and the position defined by channel.size() - 1000. This method returns the number of bytes read, possible zero, or -1 if the given position is greater than or equal to the file’s current size.
  • Print out the buffered string.

How do I read data from a buffer into channel?

In this example you’ll see how to read data from buffer using FileChannel.write() method call. Reading from a buffer means that you are writing data into the channel object. In the snippet below the data from our dummy buffer will be read and written into the result.txt file.

package org.kodejava.io;

import java.io.File;
import java.io.FileOutputStream;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

public class BufferRead {
    public static void main(String[] args) throws Exception {
        FileChannel channel = null;

        try {
            // Define an output file and create an instance of FileOutputStream
            File file = new File("result.txt");
            FileOutputStream fos = new FileOutputStream(file);

            // Create a dummy ByteBuffer which value to be read into a channel.
            ByteBuffer buffer = ByteBuffer.allocate(256);
            buffer.put(new byte[]{65, 66, 67, 68, 69, 70, 71, 72, 73, 74});

            // Change the buffer from writing mode to reading mode.
            buffer.flip();

            // Gets the channel from the FileOutputStream object and read the
            // data available in buffer using channel.write() method.
            channel = fos.getChannel();
            int bytesWritten = channel.write(buffer);
            System.out.println("written : " + bytesWritten);
        } finally {
            if (channel != null && channel.isOpen()) {
                channel.close();
            }
        }
    }
}

How do I use a FileChannel to read data into a Buffer?

The example below show how to use a FileChannel to read some data into a Buffer. We create a FileChannel from a FileInputStream instance. Because a channel reads data into buffer we need to create a ByteBuffer and set its capacity. Read data from a channel into buffer using the FileChannel.read() method.

To read out the data from the buffer we need to flip the buffer first using the Buffer.flip() method. The method will change the buffer from writing-mode into reading-mode. After the entire buffer is read clear the buffer using the clear() method call.

package org.kodejava.io;

import java.io.FileInputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.FileChannel;

public class FileRead {
    public static void main(String[] args) {
        String path = "D:/Temp/source.txt";
        try (FileInputStream fis = new FileInputStream(path);
             FileChannel fileChannel = fis.getChannel()) {

            ByteBuffer buffer = ByteBuffer.allocate(64);

            int bytesRead = fileChannel.read(buffer);
            while (bytesRead != -1) {
                buffer.flip();

                while (buffer.hasRemaining()) {
                    System.out.print((char) buffer.get());
                }

                buffer.clear();
                bytesRead = fileChannel.read(buffer);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}