How do I get HTTP headers using HttpClient HEAD request?

The HTTP HEAD method is used for reading the headers information of a resource returned when accessing it using the HTTP GET method. Such request can be done before deciding to download a large resource to save bandwidth. The response to a HEAD method should not have a body, in the code below we use the HttpResponse.BodyHandlers.discarding(), which is a response body handler that discards the the response body.

In the code snippet below we start by creating an instance of HttpClient, in this example we use the HttpClient.newBuilder().build() method. After creating the HttpClient we create the HttpRequest object. We set the HTTP method to HEAD by calling the method method() and pass a string “HEAD” as the method name and HttpRequest.BodyPublishers.noBody() a request body publisher which sends no request body.

The next step in the code below is to send the request and get the response headers from the HttpResponse object using the headers() method. The map() method of the HttpHeaders object give us a key-values of the headers returned by the server.

package org.kodejava.example.httpclient;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpHeaders;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

public class HeadRequestExample {
    public static void main(String[] args) throws Exception {
        HttpClient client = HttpClient.newBuilder().build();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://google.com"))
            .method("HEAD", HttpRequest.BodyPublishers.noBody())
            .build();

        HttpResponse<Void> response = client.send(request,
            HttpResponse.BodyHandlers.discarding());

        // Returns an unmodifiable multi map view of this HttpHeaders.
        // The map contains key of string, with list of strings as
        // its value.
        HttpHeaders headers = response.headers();
        headers.map().forEach((key, values) ->
            System.out.printf("%s = %s%n", key, values));
    }
}

Here are the HTTP headers we got and printed out to the console screen:

:status = [301]
alt-svc = [quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000]
cache-control = [public, max-age=2592000]
content-length = [220]
content-type = [text/html; charset=UTF-8]
date = [Wed, 22 Apr 2020 14:41:49 GMT]
expires = [Fri, 22 May 2020 14:41:49 GMT]
location = [https://www.google.com/]
server = [gws]
x-frame-options = [SAMEORIGIN]
x-xss-protection = [0]

How do I read website content using HttpClient?

The HTTP Client API can be used to request HTTP resources over the network. This new API was introduced as a new API in Java 11. It supports HTTP/1.1 and HTTP/2 and also support both synchronous and asynchronous programming models. The code snippet below show you how to use the new API to read the content of a website page.

In the code below we start by creating a new instance of HttpClient using the newHttpClient() static method. This is equivalent to calling newBuilder().build(). This give us an instance of HttpClient with default settings like using the “GET” request method the as the default. Then we create an HttpRequest object using the newBuilder() method, set the request URI and call the build() method to build the HttpRequest object.

Next we send the request by calling the send() method of the HttpClient object. This will sends the given request, blocking if necessary to get the response. The returned HttpResponse object contains the response status, headers, and body as handled by given response body handler.

package org.kodejava.example.httpclient;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;

public class ReadWebsiteContent {
    public static void main(String[] args) throws Exception {
        // Creates HttpClient object with default configuration.
        HttpClient httpClient = HttpClient.newHttpClient();

        // Creates HttpRequest object and set the URI to be requested, 
        // when not defined the default request method is the GET request.
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://httpie.org/hello"))
            .GET()
            .build();

        // Sends the request and print out the returned response.
        HttpResponse<String> response = httpClient.send(request,
            HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));

        System.out.println("Status Code: " + response.statusCode());
        System.out.println("Headers    : " + response.headers().toString());
        System.out.println("Body       : " + response.body());
    }
}

Here is the content of the website that we read using the code snippet above:

Status Code: 200
Headers    : java.net.http.HttpHeaders@2d299ad6 { {:status=[200], cf-cache-status=[DYNAMIC], cf-ray=[5875b78d5df2eb00-LAX], cf-request-id=[023d710c5b0000eb00b738f200000001], content-length=[116], content-type=[text/x-rst;charset=utf-8], date=[Tue, 21 Apr 2020 08:25:53 GMT], etag=["234b9a1fe19f125356a5396c8cc72d54493a2eef"], expect-ct=[max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"], server=[cloudflare], set-cookie=[__cfduid=d5bdb6d828be3bb85d0f1f4c2ff81041c1587457553; expires=Thu, 21-May-20 08:25:53 GMT; path=/; domain=.httpie.org; HttpOnly; SameSite=Lax]} }
Body       : 

Hello, World! 👋
~~~~~~~~~~~~~~~~

Thank you for trying out HTTPie 🥳

I hope this will become a friendship.

How do I do multipart upload using HttpClient?

This example demonstrates how to do multipart upload using the Apache HttpClient library. In this example we upload a single file. We start by creating an object of the file to be uploaded. The FileBody represent the binary body part of the file.

Next, prepare the HttpEntity object by create an instance of MultipartEntityBuilder. Add parts to this object, in this case we add the fileBody. We can add multiple part to this object as the name says. It can be string, file, etc as we do in a normal web form.

The build() method of the builder object finalize the entity creation and return us the HttpEntity object. To send / upload to server we create an HttpPost request and set the entity to be posted. Finally the execute() method of the HttpClient object send the multipart object to server.

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.HttpMultipartMode;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.File;
import java.io.IOException;

public class HttpPostMultipartExample {
    public static void main(String[] args) {
        File file = new File("data.zip");
        FileBody fileBody = new FileBody(file, ContentType.DEFAULT_BINARY);

        MultipartEntityBuilder builder = MultipartEntityBuilder.create();
        builder.setMode(HttpMultipartMode.BROWSER_COMPATIBLE);
        builder.addPart("file", fileBody);
        HttpEntity entity = builder.build();

        HttpPost request = new HttpPost("http://localhost:8080/upload");
        request.setEntity(entity);

        HttpClient client = HttpClientBuilder.create().build();
        try {
            client.execute(request);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

To receive the file on the server you can take a look at the servlet code in the following example: How do I create a web based file upload?.

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>
<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpmime/4.5.9/httpmime-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpmime</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central
Maven Central

How do I send POST request with a JSON body using the HttpClient?

The following code snippet show you how to send POST request with a JSON body using HttpClient. The payload in this example is a user information containing id, first_name and a last_name. We placed the payload in an object called StringEntity and also set its content type to ContentType.APPLICATION_FORM_URLENCODED.

On the other end called by this post request, data can be read for instance in a Java Servlet using the HttpServletRequest.getParameter() method. For example to read the JSON body send below we can call request.getParameter("data"). Which will give us the payload send using the HttpClient Post request.

Let’s jump into the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.HttpClientBuilder;

public class HttpPostJsonExample {
    public static void main(String[] args) throws Exception {
        String payload = "data={" +
                "\"username\": \"admin\", " +
                "\"first_name\": \"System\", " +
                "\"last_name\": \"Administrator\"" +
                "}";
        StringEntity entity = new StringEntity(payload,
                ContentType.APPLICATION_FORM_URLENCODED);

        HttpClient httpClient = HttpClientBuilder.create().build();
        HttpPost request = new HttpPost("http://localhost:8080/register");
        request.setEntity(entity);

        HttpResponse response = httpClient.execute(request);
        System.out.println(response.getStatusLine().getStatusCode());
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I send an HTTP POST request?

The following code snippet will show you how to send an HTTP post request using the Apache HttpComponents. We will send a post request to http://localhost:8080/register, with some parameters to the request using the NameValuePair object.

To pass this parameters to the HTTP post request we create an instance of UrlEncodedFormEntity and pass a list of NameValuePair as the arguments. And before executing the request we set this entity object to the HttpPost.setEntity() method.

Let’s see the code below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.util.EntityUtils;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

public class HttpPostExample {
    public static void main(String[] args) {
        HttpClient client = HttpClientBuilder.create().build();
        HttpPost post = new HttpPost("http://localhost:8080/register");

        // Create some NameValuePair for HttpPost parameters
        List<NameValuePair> arguments = new ArrayList<>(3);
        arguments.add(new BasicNameValuePair("username", "admin"));
        arguments.add(new BasicNameValuePair("firstName", "System"));
        arguments.add(new BasicNameValuePair("lastName", "Administrator"));

        try {
            post.setEntity(new UrlEncodedFormEntity(arguments));
            HttpResponse response = client.execute(post);

            // Print out the response message
            System.out.println(EntityUtils.toString(response.getEntity()));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I execute HTTP Get request?

Below is an example using HttpClient library to retrieve the content of a website using Http Get request. We start by creating an instance of DefaultHttpClient class. We then define an Http get request by create an instance of HttpGet class and specify the Url of a website we are going to retrieve.

To execute the request we call the HttpClient.execute() method and pass the HttpGet as the arguments. This execution return an HttpResponse object. From this response object we can read the content of the website by accessing the getEntity().getContent() which will give us an InputStream to the content.

For more detail let’s see the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

public class HttpGetExample {
    public static void main(String[] args) {
        HttpClient client = HttpClientBuilder.create().build();
        HttpGet request = new HttpGet("https://kodejava.org");

        try {
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();
            if (entity != null) {
                try (InputStream stream = entity.getContent()) {
                    BufferedReader reader =
                            new BufferedReader(new InputStreamReader(stream));
                    String line;
                    while ((line = reader.readLine()) != null) {
                        System.out.println(line);
                    }
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I get entity ContentType in HttpClient?

This code snippet show you how to get the content type of a result of executing an Http Get request. The ContentType can be obtained by using ContentType.getOrDefault() method and passing an HttpEntity as the arguments. The HttpEntity can be obtained from the HttpResponse object.

From the ContentType object we can get the mime-type by calling the getMimeType() method. This method will return a string value. To get the charset we can call the getCharset() method which will return a java.nio.charset.Charset object.

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.entity.ContentType;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.IOException;
import java.nio.charset.Charset;

public class EntityContentType {
    public static void main(String[] args) {
        EntityContentType demo = new EntityContentType();
        demo.getContentType("http://www.google.com");
        demo.getContentType("http://www.google.com/images/srpr/logo3w.png");
    }

    public void getContentType(String url) {
        try {
            HttpClient client = HttpClientBuilder.create().build();
            HttpGet request = new HttpGet(url);
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();
            showContentType(entity);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private void showContentType(HttpEntity entity) {
        ContentType contentType = ContentType.getOrDefault(entity);
        String mimeType = contentType.getMimeType();
        Charset charset = contentType.getCharset();

        System.out.println("MimeType = " + mimeType);
        System.out.println("Charset  = " + charset);
    }
}

The output of our code snippet are:

MimeType = text/html
Charset  = ISO-8859-1
MimeType = image/png
Charset  = null

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I get HTTP response body as a string?

This example show you how to get HTTP response body as a string when using the Apache HttpComponents library. To get the response body as a string we can use the EntityUtils.toString() method. This method read the content of an HttpEntity object content and return it as a string. The content will be converted using the character set from the entity object.

Let’s see the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;

import java.io.IOException;

public class EntityAsString {
    public static void main(String[] args) {
        HttpClient client = HttpClientBuilder.create().build();
        HttpGet request = new HttpGet("https://kodejava.org");

        try {
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();

            // Read the contents of an entity and return it as a String.
            String content = EntityUtils.toString(entity);
            System.out.println(content);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I add query string to HttpMethod object?

To send query string information in HTTP GET command you either using passing a simple string or an array of NameValuePair object into the HttpMethod‘s setQueryString() method. We also need to encode the parameter values before passing it to the method.

package org.kodejava.example.commons.httpclient;

import org.apache.commons.httpclient.*;
import org.apache.commons.httpclient.util.URIUtil;
import org.apache.commons.httpclient.methods.GetMethod;

import java.io.IOException;

public class SendingQueryParameter {
    public static void main(String[] args) {
        HttpClient client = new HttpClient();
        HttpMethod method = new GetMethod("http://localhost:8080/hello.jsp");

        try {
            // Set query string information for accessing the page using a
            // simple string information.
            method.setQueryString(URIUtil.encodeQuery("catid=10&page=1"));
            client.executeMethod(method);

            // Other cleaner alternative is to use the NameValuePair object to
            // define the parameters for a HTTP GET method.
            NameValuePair param1 = new NameValuePair("catid", URIUtil.encodeQuery("20"));
            NameValuePair param2 = new NameValuePair("page", URIUtil.encodeQuery("2"));
            NameValuePair[] params = new NameValuePair[] {param1, param2};
            method.setQueryString(params);
            client.executeMethod(method);
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            method.releaseConnection();
        }
    }
}

Maven Dependencies

<!-- http://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar -->
<dependency>
    <groupId>commons-httpclient</groupId>
    <artifactId>commons-httpclient</artifactId>
    <version>3.1</version>
</dependency>

How do I setting a proxy for HttpClient?

In this example you will see how to configure proxy when using the Apache Commons HttpClient library.

package org.kodejava.example.commons.httpclient;

import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HostConfiguration;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;
import org.apache.commons.httpclient.methods.GetMethod;

import java.io.IOException;

public class HttpGetProxy {
    private static final String PROXY_HOST = "proxy.host.com";
    private static final int PROXY_PORT = 8080;

    public static void main(String[] args) {
        HttpClient client = new HttpClient();
        HttpMethod method = new GetMethod("https://kodejava.org");

        HostConfiguration config = client.getHostConfiguration();
        config.setProxy(PROXY_HOST, PROXY_PORT);

        String username = "guest";
        String password = "s3cr3t";
        Credentials credentials = new UsernamePasswordCredentials(username, password);
        AuthScope authScope = new AuthScope(PROXY_HOST, PROXY_PORT);

        client.getState().setProxyCredentials(authScope, credentials);

        try {
            client.executeMethod(method);

            if (method.getStatusCode() == HttpStatus.SC_OK) {
                String response = method.getResponseBodyAsString();
                System.out.println("Response = " + response);
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            method.releaseConnection();
        }
    }
}

Maven Dependencies

<!-- http://repo1.maven.org/maven2/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar -->
<dependency>
    <groupId>commons-httpclient</groupId>
    <artifactId>commons-httpclient</artifactId>
    <version>3.1</version>
</dependency>