How do I read large files with streams?

By I Wayan Saryada in IO 0 Comment

Reading large files in Java efficiently is best achieved by using Stream-based APIs that process the file line-by-line or chunk-by-chunk. This prevents loading the entire file into memory (preventing OutOfMemoryError).

Here are the most common and efficient ways to do this:

1. Using Files.lines() (Recommended)

This is the most modern and idiomatic way in Java. It returns a Stream<String> where each element is a line from the file. It reads the lines lazily, meaning it only keeps a small portion of the file in memory at any given time.

Important: Always use a try-with-resources block to ensure the file handle is closed.

package org.kodejava.nio;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.stream.Stream;

public class LargeFileReader {
    public static void main(String[] args) {
        Path path = Paths.get("D:/large-file.txt");

        try (Stream<String> lines = Files.lines(path)) {
            lines.filter(line -> line.contains("Error")) // Example processing
                    .forEach(System.out::println);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

2. Using BufferedReader.lines()

If you already have a BufferedReader (for example, if you’re dealing with a specific character encoding), you can use its .lines() method. This also returns a lazy stream.

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

try (BufferedReader br = new BufferedReader(new FileReader("large-file.txt"))) {
    br.lines()
      .map(String::toLowerCase)
      .forEach(line -> {
          // Process each line here
      });
} catch (IOException e) {
    e.printStackTrace();
}

3. Using Scanner (For Tokens)

If you need to read tokens (like words or numbers) rather than full lines, Scanner is useful. However, it is generally slower than BufferedReader.

import java.util.Scanner;
import java.io.File;

try (Scanner scanner = new Scanner(new File("large-file.txt"))) {
    while (scanner.hasNextLine()) {
        String line = scanner.nextLine();
        // Process line
    }
} catch (IOException e) {
    e.printStackTrace();
}

Summary of Tips for Large Files:

  • Lazy Evaluation: Operations like filter and map on Java Streams are lazy. They don’t process the data until a terminal operation (like forEach or collect) is called.
  • Memory Efficiency: The Stream API ensures that you aren’t storing the whole file in a List<String>, which would quickly crash your app for multi-gigabyte files.
  • Parallelism: For huge files, you can use .parallel() on the stream. However, be careful as IO-bound tasks often don’t benefit much from parallel streams unless the processing logic per line is very heavy.

How do I use NIO Path.of() instead of Paths.get()?

By I Wayan Saryada in IO, Java 11 0 Comment

In Java 11 and later, Path.of() is the preferred way to create Path instances, effectively replacing Paths.get().

Here is how you can use it:

1. Basic Usage (Replacing Paths.get)

The syntax is almost identical. It accepts a string or a sequence of strings to join into a path.

package org.kodejava.nio;

import java.nio.file.Path;

public class PathExample {
    public static void main(String[] args) {
        // Using a single string
        Path path1 = Path.of("C:/logs/app.log");

        // Using multiple strings (varargs) to join paths
        Path path2 = Path.of("C:", "logs", "app.log");

        System.out.println(path2); // Outputs: C:\logs\app.log (on Windows)
    }
}

2. Working with URIs

Path.of() also has an overload that accepts a URI object, just like Paths.get(URI uri).

import java.net.URI;
import java.nio.file.Path;

Path pathFromUri = Path.of(URI.create("file:///C:/logs/app.log"));

Why use Path.of() instead of Paths.get()?

  • Cleaner API: Path is the primary interface. Path.of() keeps the logic within the interface itself rather than relying on a separate utility class (Paths).
  • Modern Standard: Paths.get() was introduced in Java 7 as a bridge. Java 11 introduced Path.of() as the modern, static factory method on the interface.
  • Consistency: Most modern Java APIs (like List.of(), Set.of()) use this naming convention.

How do I use Files.mismatch() to compare files?

By I Wayan Saryada in IO, Java 12 0 Comment

In Java, java.nio.file.Files.mismatch(Path, Path) is a powerful method introduced in Java 12 that allows you to compare the contents of two files efficiently. It returns the position of the first byte where the two files differ, or -1L if they are identical.

How to use Files.mismatch

Here is a basic example of how to implement it:

package org.kodejava.nio;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;

public class FileCompare {
    public static void main(String[] args) {
        Path path1 = Path.of("file1.txt");
        Path path2 = Path.of("file2.txt");

        try {
            long mismatch = Files.mismatch(path1, path2);

            if (mismatch == -1L) {
                System.out.println("Files are identical.");
            } else {
                System.out.println("Files differ at byte position: " + mismatch);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Key Behaviors to Keep in Mind:

  1. Return Values:
    • -1L: The files are identical (same size and same content).
    • A non-negative value: The index of the first byte that differs.
    • File Size Mismatch: If one file is a prefix of the other, it returns the size of the smaller file as the mismatch point.
  2. Performance: Files.mismatch is generally faster than manual byte-by-byte comparison because it uses optimized internal buffers.
  3. Same Path: If you pass the exact same Path object (or two paths that point to the same file via Files.isSameFile), it returns -1L immediately without reading the content.
  4. Exceptions: It throws an IOException if there’s an error reading the files or if one of the paths does not exist.

How do I read and write files with Files.readString() and Files.writeString()?

By I Wayan Saryada in IO, Java 11 0 Comment

In Java, Files.readString and Files.writeString (introduced in Java 11) are the most straightforward ways to handle small-to-medium-sized text files. They handle the opening, closing, and encoding for you in a single line of code.

Here is how you can use them:

1. Reading a File to a String

Files.readString(Path) reads the entire content of a file into a String. By default, it uses UTF-8 encoding.

package org.kodejava.nio;

import java.nio.file.Files;
import java.nio.file.Path;
import java.io.IOException;

public class ReadExample {
    public static void main(String[] args) {
        Path filePath = Path.of("example.txt");

        try {
            // Reads the whole file into a String using UTF-8
            String content = Files.readString(filePath);
            System.out.println(content);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

2. Writing a String to a File

Files.writeString(Path, CharSequence) writes text to a file. If the file doesn’t exist, it creates it. If it does exist, it overwrites it by default.

package org.kodejava.nio;

import java.nio.file.Files;
import java.nio.file.Path;
import java.io.IOException;
import java.nio.file.StandardOpenOption;

public class WriteExample {
    public static void main(String[] args) {
        Path filePath = Path.of("example.txt");
        String data = "Hello, Java developers!\nThis is a test.";

        try {
            // Overwrites the file with the string content
            Files.writeString(filePath, data);

            // To APPEND instead of overwrite, use StandardOpenOption:
            // Files.writeString(filePath, "\nMore data", StandardOpenOption.APPEND);

            System.out.println("File written successfully.");
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Key Points to Remember:

  • Memory Usage: Both methods load the entire file content into memory. Do not use them for very large files (e.g., gigabyte-sized logs), as they could cause an OutOfMemoryError.
  • Encoding: Both methods use UTF-8 by default. If you need a different encoding, you can pass a Charset as an additional argument:
    Files.readString(path, StandardCharsets.ISO_8859_1);
  • Exceptions: Both methods throw IOException, so they must be used within a try-catch block or a method that declares throws IOException.
  • Path API: Use Path.of("path/to/file") (Java 11+) or Paths.get("path/to/file") to create the Path object needed for these methods.

How do I debug concurrency issues effectively?

By I Wayan Saryada in Concurrency 0 Comment

Debugging concurrency issues (like deadlocks, race conditions, and thread starvation) can feel like chasing ghosts because they are often non-deterministic. Here’s a strategy to tackle them effectively using both design patterns and tools available in your environment.

1. Give Your Threads Meaningful Names

The default pool-1-thread-1 names are useless in a thread dump. By using a Custom Thread Factory, you can prefix threads based on their purpose (e.g., Email-Dispatcher-1, Database-Writer-2).

As shown in your project’s CustomThreadFactory.java:

// ... existing code ...
        @Override
        public Thread newThread(Runnable r) {
            Thread t = new Thread(r, namePrefix + threadNumber.getAndIncrement());
            t.setDaemon(daemon);
            t.setPriority(Thread.NORM_PRIORITY);
            return t;
        }
// ... existing code ...

This simple change makes logs and debugger views instantly readable.

2. Leverage IntelliJ IDEA’s Concurrency Tools

IntelliJ has built-in features specifically for multithreaded debugging:

  • Thread Selector: When hit at a breakpoint, use the dropdown in the Debug Tool Window to switch between threads and see their individual call stacks.
  • Breakpoint Suspend Policy: Right-click a breakpoint and change “Suspend” from All to Thread. This allows other threads to keep running while you inspect one, which is crucial for reproducing race conditions.
  • Async Stack Traces: Enable “Instrumenting agent” in Settings -> Build, Execution, Deployment -> Debugger -> Async Stack Traces. This stitches together stack traces across CompletableFuture or ExecutorService boundaries.

3. Analyze Thread Dumps

If your application “freezes,” it’s likely a deadlock.

  • Capture a Dump: In IntelliJ, use Process Console -> Tasks -> Attach Debugger or jstack <pid> from the terminal.
  • What to Look For: Look for threads in the BLOCKED state. Modern JVMs are quite good at detecting deadlocks and will explicitly list them at the bottom of the dump:
    Found one Java-level deadlock: ...

4. Logging with Context

Standard System.out.println is often not thread-safe or lacks context. Use a logging framework (like Logback, which is in your pom.xml) and include the thread name in your pattern:

<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>

5. Use Thread-Safe Decorators and Atomic Variables

Before reaching for synchronized blocks, see if you can use the java.util.concurrent utilities you already have in your project:

  • AtomicInteger / AtomicReference: For lock-free state updates (see your HighContentionCounter.java).
  • StampedLock: For high-performance optimistic reading (see StampedLockExample.java).
  • Semaphore: To throttle resource access and prevent starvation (see SemaphoreExample.java).

6. Stress Testing with jcrestress or Thread Interleaving

Sometimes code works 99% of the time. To find the 1% failure:

  1. Reduce Thread Sleep: Replace Thread.sleep() with CountDownLatch or Phaser to ensure threads hit a specific point at the same time.
  2. Looping: Wrap your test case in a loop that runs 10,000 times. Concurrency bugs often require specific CPU timing to trigger.

Pro-tip: If you suspect a race condition on a specific field, use a Field Watchpoint in IntelliJ. It will pause execution every time that specific variable is modified by any thread.