How to read text file contents line by line?

In the following code example you will learn how to read file contents line by line using the Apache Commons FileUtils.lineIterator() method. Reading file contents one line at a time, do some processing, and release it from memory immediately will lower the memory consumption used by your program.

The snippet below give you the basic usage of the FileUtils.lineIterator() method. You pass the file to read and the encoding to use. An Iterator of the lines in the file will be returned. Use the hasNext() method to see if there are lines to read from the iterator. The nextLine() method will give you the next line from the file.

When we finished with the iterator we need to close it using the LineIterator.close() or LineIterator.closeQuietly() method.

package org.kodejava.example.commons.io;

import org.apache.commons.io.FileUtils;
import org.apache.commons.io.LineIterator;

import java.io.File;
import java.net.URL;
import java.util.Objects;

public class ReadFileLineByLine {
    public static void main(String[] args) throws Exception {
        // Load file from resource directory.
        ClassLoader classLoader = ReadFileLineByLine.class.getClassLoader();
        URL url = Objects.requireNonNull(classLoader.getResource("data.txt"),
                "Resource could not be found.");

        File file = new File(url.getFile());
        LineIterator iterator = FileUtils.lineIterator(file, "UTF-8");
        try {
            while (iterator.hasNext()) {
                String line = iterator.nextLine();
                System.out.println("line = " + line);
            }
        } finally {
            LineIterator.closeQuietly(iterator);
        }
    }
}

In the example above we load the file from a resource directory. That’s why we use the ClassLoader.getResource() method. If you want to load a file from an absolute path you can simply create a File object and pass the absolute path to the file.

Maven Dependency

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.5</version>
</dependency>

Using DigestUtils.sha1hex() method to generate SHA-1 digest

In this example you’ll learn how to generate an SHA-1 digest using the Apache Commons Codec DigestUtils class. In the last two examples you’ve already seen how to generate the MD5 digest using the same library. Compared to the MD5 version the SHA-1 digest is known to be stronger to brute force attacks, but it slower to generate. The SHA-1 produces a 160 bit (20 byte) message digest while the MD5 produces only a 128 bit message digest (16 byte).

In the code snippet below we demonstrate three different ways to use the DigestUtils.sha1Hex() method. In the first method in the example, the byteDigest(), we calculates the digest from an array of byte data. Followed by the second method, the inputStreamDigest() where we calculate the digest of an InputStream object. And on the last method we call the overload version of the sha1Hex() method to calculate the digest of a string.

Let’s see the full code snippet.

package org.kodejava.example.commons.codec;

import org.apache.commons.codec.digest.DigestUtils;

import java.io.*;

public class SHAHashDemo {
    public static void main(String[] args) {
        SHAHashDemo demo = new SHAHashDemo();
        demo.byteDigest();
        demo.inputStreamDigest();
        demo.stringDigest();
    }

    /**
     * Calculates SHA-1 digest from byte array.
     */
    private void byteDigest() {
        System.out.println("SHAHashDemo.byteDigest");
        try {
            byte[] data = "The quick brown fox jumps over the lazy dog.".getBytes("UTF-8");
            String digest = DigestUtils.sha1Hex(data);
            System.out.println("Digest          = " + digest);
            System.out.println("Digest.length() = " + digest.length());
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    /**
     * Calculates SHA-1 digest of InputStream object.
     */
    private void inputStreamDigest() {
        System.out.println("SHAHashDemo.inputStreamDigest");
        String data = System.getProperty("user.dir") + "/target/classes/data.txt";
        File file = new File(data);
        try {
            InputStream is = new FileInputStream(file);
            String digest = DigestUtils.sha1Hex(is);
            System.out.println("Digest          = " + digest);
            System.out.println("Digest.length() = " + digest.length());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    /**
     * Calculate SHA-1 digest of a string / text.
     */
    private void stringDigest() {
        System.out.println("SHAHashDemo.stringDigest");
        String data = "This is just a simple data message for SHA digest demo.";
        String digest = DigestUtils.sha1Hex(data);
        System.out.println("Digest          = " + digest);
        System.out.println("Digest.length() = " + digest.length());
    }
}

When you run the code it will output the following result:

SHAHashDemo.stringDigest
Digest          = 4290d13ca383c2159c442d75355d83e310a2ea15
Digest.length() = 40
SHAHashDemo.inputStreamDigest
Digest          = b94d7f261acd677ae69f4244e5f894313a2cd559
Digest.length() = 40
SHAHashDemo.byteDigest
Digest          = 408d94384216f890ff7a0c3528e8bed1e0b01621
Digest.length() = 40