How do I do multipart upload using HttpClient?

This example demonstrates how to do multipart upload using the Apache HttpClient library. In this example we upload a single file. We start by creating an object of the file to be uploaded. The FileBody represent the binary body part of the file.

Next, prepare the HttpEntity object by create an instance of MultipartEntityBuilder. Add parts to this object, in this case we add the fileBody. We can add multiple part to this object as the name says. It can be string, file, etc as we do in a normal web form.

The build() method of the builder object finalize the entity creation and return us the HttpEntity object. To send / upload to server we create an HttpPost request and set the entity to be posted. Finally the execute() method of the HttpClient object send the multipart object to server.

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.File;
import java.io.IOException;

public class HttpPostMultipartExample {
    public static void main(String[] args) {
        File file = new File("data.zip");
        FileBody fileBody = new FileBody(file, ContentType.DEFAULT_BINARY);

        MultipartEntityBuilder builder = MultipartEntityBuilder.create();
        builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
        builder.addPart("file", fileBody);
        HttpEntity entity = builder.build();

        HttpPost request = new HttpPost("http://localhost:8080/upload");
        request.setEntity(entity);

        HttpClient client = HttpClientBuilder.create().build();
        try {
            client.execute(request);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

To receive the file on the server you can take a look at the servlet code in the following example: How do I create a web based file upload?.

Maven Dependencies

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.2</version>
</dependency>
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpmime</artifactId>
    <version>4.3.6</version>
</dependency>

How do I send POST request with a JSON body using the HttpClient?

The following code snippet show you how to send POST request with a JSON body using HttpClient. The payload in this example is a user information containing id, first_name and a last_name. We placed the payload in an object called StringEntity and also set its content type to ContentType.APPLICATION_FORM_URLENCODED.

On the other end called by this post request, data can be read for instance in a Java Servlet using the HttpServletRequest.getParameter() method. For example to read the JSON body send below we can call request.getParameter("data"). Which will give us the payload send using the HttpClient Post request.

Let’s jump into the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;

public class HttpPostJsonExample {
    public static void main(String[] args) throws Exception {
        String payload = "data={" +
                "\"username\": \"admin\", " +
                "\"first_name\": \"System\", " +
                "\"last_name\": \"Administrator\"" +
                "}";
        StringEntity entity = new StringEntity(payload,
                ContentType.APPLICATION_FORM_URLENCODED);

        HttpClient httpClient = HttpClientBuilder.create().build();
        HttpPost request = new HttpPost("http://localhost:8080/register");
        request.setEntity(entity);

        HttpResponse response = httpClient.execute(request);
        System.out.println(response.getStatusLine().getStatusCode());
    }
}

Maven Dependencies

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.2</version>
</dependency>

How do I sort file names by their extension?

To sort file names by their extension we can use the ExtensionFileComparator class from the Apache Commons IO library. This class provide a couple instances of comparator such as:

Comparator Description
EXTENSION_COMPARATOR Case sensitive extension comparator
EXTENSION_REVERSE Reverse case sensitive extension comparator
EXTENSION_INSENSITIVE_COMPARATOR Case insensitive extension comparator
EXTENSION_INSENSITIVE_REVERSE Reverse case insensitive extension comparator
EXTENSION_SYSTEM_COMPARATOR System sensitive extension comparator
EXTENSION_SYSTEM_REVERSE Reverse system sensitive path comparator

The following snippet show you how to use the first two comparators listed above.

package org.kodejava.example.commons.io;

import org.apache.commons.io.FilenameUtils;
import static org.apache.commons.io.comparator.ExtensionFileComparator.*;

import java.io.File;
import java.util.Arrays;

public class FileSortByExtension {
    public static void main(String[] args) {
        File file = new File(".");

        // Excludes directory in the list
        File[] files = file.listFiles(File::isFile);

        if (files != null) {
            // Sort in ascending order.
            Arrays.sort(files, EXTENSION_COMPARATOR);
            FileSortByExtension.displayFileOrder(files);

            // Sort in descending order.
            Arrays.sort(files, EXTENSION_REVERSE);
            FileSortByExtension.displayFileOrder(files);
        }
    }

    private static void displayFileOrder(File[] files) {
        System.out.printf("%-20s | %s%n", "Name", "Ext");
        System.out.println("--------------------------------");
        for (File file : files) {
            System.out.printf("%-20s | %s%n", file.getName(),
                    FilenameUtils.getExtension(file.getName()));
        }
        System.out.println("");
    }
}

The result of the code snippet:

Name                 | Ext
--------------------------------
README               | 
lipsum.doc           | doc
lipsum.docx          | docx
data.html            | html
contributors.txt     | txt
pom.xml              | xml

Name                 | Ext
--------------------------------
pom.xml              | xml
contributors.txt     | txt
data.html            | html
lipsum.docx          | docx
lipsum.doc           | doc
README               | 

Maven Dependencies

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.5</version>
</dependency>

How do I sort files and directories based on their size?

In this example you will learn how to sort files and directories based on their size. Using the Apache Commons IO we can utilize the SizeFileComparator class. This class provides some instances to sort file size such as:

Comparator Description
SIZE_COMPARATOR Size comparator instance – directories are treated as zero size
SIZE_REVERSE Reverse size comparator instance – directories are treated as zero size
SIZE_SUMDIR_COMPARATOR Size comparator instance which sums the size of a directory’s contents
SIZE_SUMDIR_REVERSE Reverse size comparator instance which sums the size of a directory’s contents

Now let’s jump to the code snippet below:

package org.kodejava.example.commons.io;

import org.apache.commons.io.FileUtils;

import java.io.File;
import java.util.Arrays;

import static org.apache.commons.io.comparator.SizeFileComparator.*;

public class FileSortBySize {
    public static void main(String[] args) {
        File dir = new File(".");
        File[] files = dir.listFiles();

        if (files != null) {
            // Sort files in ascending order based on file size.
            System.out.println("Ascending order.");
            Arrays.sort(files, SIZE_COMPARATOR);
            FileSortBySize.displayFileOrder(files, false);

            // Sort files in descending order based on file size
            System.out.println("Descending order.");
            Arrays.sort(files, SIZE_REVERSE);
            FileSortBySize.displayFileOrder(files, false);

            // Sort files in ascending order based on file / directory
            // size
            System.out.println("Ascending order with directories.");
            Arrays.sort(files, SIZE_SUMDIR_COMPARATOR);
            FileSortBySize.displayFileOrder(files, true);

            // Sort files in descending order based on file / directory
            // size
            System.out.println("Descending order with directories.");
            Arrays.sort(files, SIZE_SUMDIR_REVERSE);
            FileSortBySize.displayFileOrder(files, true);
        }
    }

    private static void displayFileOrder(File[] files, boolean displayDirectory) {
        for (File file : files) {
            if (!file.isDirectory()) {
                System.out.printf("%-25s - %s%n", file.getName(),
                        FileUtils.byteCountToDisplaySize(file.length()));
            } else if (displayDirectory) {
                long size = FileUtils.sizeOfDirectory(file);
                String friendlySize = FileUtils.byteCountToDisplaySize(size);
                System.out.printf("%-25s - %s%n", file.getName(),
                        friendlySize);
            }
        }
        System.out.println("------------------------------------");
    }
}

In the code snippet above we use a couple method from the FileUtils class such as the FileUtils.sizeOfDirectory() to calculate the size of a directory and FileUtils.byteCountToDisplaySize() to create human readable file size.

The result of the code snippet:

Ascending order.
.editorconfig             - 389 bytes
kodejava.iml              - 868 bytes
pom.xml                   - 1 KB
------------------------------------
Descending order.
pom.xml                   - 1 KB
kodejava.iml              - 868 bytes
.editorconfig             - 389 bytes
------------------------------------
Ascending order with directories.
.editorconfig             - 389 bytes
src                       - 851 bytes
kodejava.iml              - 868 bytes
pom.xml                   - 1 KB
apache-commons-example    - 8 KB
hibernate-example         - 29 KB
.idea                     - 85 KB
------------------------------------
Descending order with directories.
.idea                     - 85 KB
hibernate-example         - 29 KB
apache-commons-example    - 8 KB
pom.xml                   - 1 KB
kodejava.iml              - 868 bytes
src                       - 851 bytes
.editorconfig             - 389 bytes

Maven Dependencies

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.5</version>
</dependency>

How to read text file contents line by line?

In the following code example you will learn how to read file contents line by line using the Apache Commons FileUtils.lineIterator() method. Reading file contents one line at a time, do some processing, and release it from memory immediately will lower the memory consumption used by your program.

The snippet below give you the basic usage of the FileUtils.lineIterator() method. You pass the file to read and the encoding to use. An Iterator of the lines in the file will be returned. Use the hasNext() method to see if there are lines to read from the iterator. The nextLine() method will give you the next line from the file.

When we finished with the iterator we need to close it using the LineIterator.close() or LineIterator.closeQuietly() method.

package org.kodejava.example.commons.io;

import org.apache.commons.io.FileUtils;
import org.apache.commons.io.LineIterator;

import java.io.File;
import java.net.URL;
import java.util.Objects;

public class ReadFileLineByLine {
    public static void main(String[] args) throws Exception {
        // Load file from resource directory.
        ClassLoader classLoader = ReadFileLineByLine.class.getClassLoader();
        URL url = Objects.requireNonNull(classLoader.getResource("data.txt"),
                "Resource could not be found.");

        File file = new File(url.getFile());
        LineIterator iterator = FileUtils.lineIterator(file, "UTF-8");
        try {
            while (iterator.hasNext()) {
                String line = iterator.nextLine();
                System.out.println("line = " + line);
            }
        } finally {
            LineIterator.closeQuietly(iterator);
        }
    }
}

In the example above we load the file from a resource directory. That’s why we use the ClassLoader.getResource() method. If you want to load a file from an absolute path you can simply create a File object and pass the absolute path to the file.

Maven Dependency

<dependency>
    <groupId>commons-io</groupId>
    <artifactId>commons-io</artifactId>
    <version>2.5</version>
</dependency>

Why do I get NotOLE2FileException: Invalid header signature error?

When I was creating the code snippet for the How do I replace text in Microsoft Word document using Apache POI? I got the following error when running the snippet . As an additional information, the code snippet for the Kodejava website is written as a Maven project.

org.apache.poi.poifs.filesystem.NotOLE2FileException: Invalid header signature; read 0xE011BDBFEFBDBFEF, expected 0xE11AB1A1E011CFD0 – Your file appears not to be a valid OLE2 document

This error message was produce when the code trying to open a Word document. The Apache POI reports that the Word document has an invalid header signature, not a valid OLE2 document. Comparing the original document with the document under maven’s target directory I found out that the file size was different. This could mean that something alter the document and corrupt the header.

After doing some googling I find out that this error is due to maven resource filtering. The maven resource filtering process cause the Word document header corrupt during the copy phase. The solution to this problem is to make sure that the filtering process is set to false. The pom.xml of the maven project should be altered to have the following configuration.

<build>
    <resources>
        <resource>
            <directory>src/main/resources</directory>
            <filtering>false</filtering>
        </resource>
    </resources>
</build>

How do I replace text in Microsoft Word document using Apache POI?

The code snippet below show you how you can replace string in Microsoft Word document using the Apache POI library. The class below have three method, the openDocument(), saveDocument() and replaceText().

The routine for replacing text is implemented in the replaceText() method. This method take the HWPFDocument, the String to find and the String to replace it as parameters. The openDocument() opens the Word document. When the text replacement is done the Word document will be saved by the saveDocument() method.

This code snippet requires the following library dependency to work:

<dependency>
    <groupId>org.apache.poi</groupId>
    <artifactId>poi</artifactId>
    <version>3.13</version>
</dependency>
<dependency>
    <groupId>org.apache.poi</groupId>
    <artifactId>poi-scratchpad</artifactId>
    <version>3.13</version>
</dependency>

And here is the complete code snippet. It will replace every o characters with 0 character in the source document, the lipsum.doc and save the result in a new document called new-lipsum.doc.

package org.kodejava.example.poi;

import org.apache.poi.hwpf.HWPFDocument;
import org.apache.poi.hwpf.usermodel.CharacterRun;
import org.apache.poi.hwpf.usermodel.Paragraph;
import org.apache.poi.hwpf.usermodel.Range;
import org.apache.poi.hwpf.usermodel.Section;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.URL;

public class WordReplaceText {
    public static final String SOURCE_FILE = "lipsum.doc";
    public static final String OUTPUT_FILE = "new-lipsum.doc";

    public static void main(String[] args) throws Exception {
        WordReplaceText instance = new WordReplaceText();
        HWPFDocument doc = instance.openDocument(SOURCE_FILE);
        if (doc != null) {
            doc = instance.replaceText(doc, "o", "0");
            instance.saveDocument(doc, OUTPUT_FILE);
        }
    }

    private HWPFDocument replaceText(HWPFDocument doc, String findText, String replaceText) {
        Range r = doc.getRange();
        for (int i = 0; i < r.numSections(); ++i) {
            Section s = r.getSection(i);
            for (int j = 0; j < s.numParagraphs(); j++) {
                Paragraph p = s.getParagraph(j);
                for (int k = 0; k < p.numCharacterRuns(); k++) {
                    CharacterRun run = p.getCharacterRun(k);
                    String text = run.text();
                    if (text.contains(findText)) {
                        run.replaceText(findText, replaceText);
                    }
                }
            }
        }
        return doc;
    }

    private HWPFDocument openDocument(String file) throws Exception {
        URL res = getClass().getClassLoader().getResource(file);
        HWPFDocument document = null;
        if (res != null) {
            document = new HWPFDocument(new POIFSFileSystem(
                    new File(res.getPath())));
        }
        return document;
    }

    private void saveDocument(HWPFDocument doc, String file) {
        try (FileOutputStream out = new FileOutputStream(file)) {
            doc.write(out);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Using DigestUtils.sha1hex() method to generate SHA-1 digest

In this example you’ll learn how to generate an SHA-1 digest using the Apache Commons Codec DigestUtils class. In the last two examples you’ve already seen how to generate the MD5 digest using the same library. Compared to the MD5 version the SHA-1 digest is known to be stronger to brute force attacks, but it slower to generate. The SHA-1 produces a 160 bit (20 byte) message digest while the MD5 produces only a 128 bit message digest (16 byte).

In the code snippet below we demonstrate three different ways to use the DigestUtils.sha1Hex() method. In the first method in the example, the byteDigest(), we calculates the digest from an array of byte data. Followed by the second method, the inputStreamDigest() where we calculate the digest of an InputStream object. And on the last method we call the overload version of the sha1Hex() method to calculate the digest of a string.

Let’s see the full code snippet.

package org.kodejava.example.commons.codec;

import org.apache.commons.codec.digest.DigestUtils;

import java.io.*;

public class SHAHashDemo {
    public static void main(String[] args) {
        SHAHashDemo demo = new SHAHashDemo();
        demo.byteDigest();
        demo.inputStreamDigest();
        demo.stringDigest();
    }

    /**
     * Calculates SHA-1 digest from byte array.
     */
    private void byteDigest() {
        System.out.println("SHAHashDemo.byteDigest");
        try {
            byte[] data = "The quick brown fox jumps over the lazy dog.".getBytes("UTF-8");
            String digest = DigestUtils.sha1Hex(data);
            System.out.println("Digest          = " + digest);
            System.out.println("Digest.length() = " + digest.length());
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        }
    }

    /**
     * Calculates SHA-1 digest of InputStream object.
     */
    private void inputStreamDigest() {
        System.out.println("SHAHashDemo.inputStreamDigest");
        String data = System.getProperty("user.dir") + "/target/classes/data.txt";
        File file = new File(data);
        try {
            InputStream is = new FileInputStream(file);
            String digest = DigestUtils.sha1Hex(is);
            System.out.println("Digest          = " + digest);
            System.out.println("Digest.length() = " + digest.length());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    /**
     * Calculate SHA-1 digest of a string / text.
     */
    private void stringDigest() {
        System.out.println("SHAHashDemo.stringDigest");
        String data = "This is just a simple data message for SHA digest demo.";
        String digest = DigestUtils.sha1Hex(data);
        System.out.println("Digest          = " + digest);
        System.out.println("Digest.length() = " + digest.length());
    }
}

When you run the code it will output the following result:

SHAHashDemo.stringDigest
Digest          = 4290d13ca383c2159c442d75355d83e310a2ea15
Digest.length() = 40
SHAHashDemo.inputStreamDigest
Digest          = b94d7f261acd677ae69f4244e5f894313a2cd559
Digest.length() = 40
SHAHashDemo.byteDigest
Digest          = 408d94384216f890ff7a0c3528e8bed1e0b01621
Digest.length() = 40

Generating MD5 digest from File or InputStream object

In the previous example you can see how to calculate the MD5 digest from a text or a string. We are using the Apache Commons Codec library and use the DigestUtils.md5Hex() method to generate the MD5. I’ve mention in that post that we can also generate the MD5 digest of a byte array and InputStream object. In the example below you’ll see an example to generate the MD5 digest of a text data stored in a file.

package org.kodejava.example.commons.codec;

import org.apache.commons.codec.digest.DigestUtils;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;

public class MD5FileHashDemo {
    public static void main(String[] args) {
        try {
            // Define the data file path and create an InputStream object.
            String data = System.getProperty("user.dir") + "/target/classes/data.txt" ;
            File file = new File(data);
            InputStream is = new FileInputStream(file);

            // Calculates the MD5 digest of the given InputStream object.
            // It will generate a 32 characters hex string.
            String digest = DigestUtils.md5Hex(is);
            System.out.println("Digest = " + digest);
            System.out.println("Length = " + digest.length());
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

The first thing we need to do is to add the import statement to our class to import the org.apache.commons.codec.digest.DigestUtils class. The DigestUtils.md5Hex() method define as a static method so that we don’t have to create an instance of DigestUtils class before we can use it. Before we create the InputStream object we define the path to our data file. I am creating the code snippet using Apache Maven as the build tools. That’s why you see my path the the data file is under the target/classes directory, because that directory is the default location where Maven will place the compiled classes and resource file. Next we create the File object from the defined path followed by creating the InputStream object of the File.

To generate the digest we can simply pass the instance of the InputStream object into the DigestUtils.md5Hex() method. And if there is no error occurred during the process we will get a 32 character of hex string as the output. One last thing that you have to do is the catch the possible exception thrown by the method. So we add the try-catch block and print the error stack trace to help us identify any error.

And here is the an example output generated by the code snippet above:

Digest = bafb2ec738d6de85170073c80a1008ad
Length = 32

How do I calculate the MD5 digest of a string?

In this post you will learn how to calculate the MD5 digest of a string. One of the most commonly use for this functionality in an application is to secure a password. For security reasons, you will never want to store user passwords in the application database in a plain text. There are other more secure hash algorithm out there such as the SHA-1 (Secure Hash Algorithm) but the MD5 message digest offer a faster implementation compared to SHA-1.

You can calculates the MD5 digest using the message digest library provided by the Java Standard Edition library, but the Apache Commons Codec library gives you a simple and easy to use API for generating the MD5 hash. On the example below you will see how to generate to hash to secure a password and also to generate the hash for a longer text. The text is what you might want to send to a friend via an email. To make sure that your friend receive your original message without any modification you can send the generated hash in separated email that can be use to verify the message.

package org.kodejava.example.commons.codec;

import org.apache.commons.codec.digest.DigestUtils;

public class MD5HashDemo {
    public static void main(String[] args) {
        // Calculates the MD5 digest for the password text and returns
        // the value as a 32 character hex string.
        String password = "s3cretw0rd**";
        String digest = DigestUtils.md5Hex(password);

        // Prints the plain text password, the digest and the length of
        // the digest.
        System.out.println("Password        = " + password);
        System.out.println("Password Digest = " + digest);
        System.out.println("Length          = " + digest.length());

        // Calculates the MD5 digest for the long texts.
        String md5 = "The MD5 message-digest algorithm is a formerly " +
                "widely used cryptographic hash function that produces " +
                "a 128-bit (16-byte) hash value. Specified in RFC 1321, " +
                "MD5 has been utilized in a wide variety of security " +
                "applications, and is also commonly used to check data " +
                "integrity. MD5 was designed by Ron Rivest in 1991 to " +
                "replace an earlier hash function, MD4. An MD5 hash value " +
                "is typically expressed as a hexadecimal number, 32 " +
                "digits long.";
        String fingerprint = DigestUtils.md5Hex(md5);

        // Prints the text, the fingerprint and the length of the digest /
        // fingerprint.
        System.out.println("Text        = " + md5);
        System.out.println("Fingerprint = " + fingerprint);
        System.out.println("Length      = " + fingerprint.length());
    }
}

As can be seen in the code example above, we use the DigestUtils.md5Hex() method to generate the MD5 digest. This class is part of the Apache Commons Codec under theĀ org.apache.commons.codec.digest package. The method is a static method so we don’t need to create an instance of the class before we can utilize the method. The md5Hex() method takes a string argument and produce a 32 characters hex string. The length will always 32 characters regardless the length of the processed text / string.

Besides accepting a string argument, the overload version of the DigestUtils.md5Hex() method can accept an array of byte or a java.io.InputStream object as the argument.

Here is an example of the output produces:

Password        = s3cretw0rd**
Password Digest = 203c603a7330ab3ea032f4b9f140cf95
Length          = 32
Text        = The MD5 message-digest algorithm is a formerly widely used cryptographic hash function that produces a 128-bit (16-byte) hash value. Specified in RFC 1321, MD5 has been utilized in a wide variety of security applications, and is also commonly used to check data integrity. MD5 was designed by Ron Rivest in 1991 to replace an earlier hash function, MD4. An MD5 hash value is typically expressed as a hexadecimal number, 32 digits long.
Fingerprint = 3434955eecba50de487890d493ebaddd
Length      = 32