How do I get HTTP headers using HttpClient HEAD request?

By Wayan Saryada in HTTP Client 0 Comment

The HTTP HEAD method is used for reading the headers information of a resource returned when accessing it using the HTTP GET method. Such request can be done before deciding to download a large resource to save bandwidth. The response to a HEAD method should not have a body, in the code below we use the HttpResponse.BodyHandlers.discarding(), which is a response body handler that discards the the response body.

In the code snippet below we start by creating an instance of HttpClient, in this example we use the HttpClient.newBuilder().build() method. After creating the HttpClient we create the HttpRequest object. We set the HTTP method to HEAD by calling the method method() and pass a string “HEAD” as the method name and HttpRequest.BodyPublishers.noBody() a request body publisher which sends no request body.

The next step in the code below is to send the request and get the response headers from the HttpResponse object using the headers() method. The map() method of the HttpHeaders object give us a key-values of the headers returned by the server.

package org.kodejava.example.httpclient;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpHeaders;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

public class HeadRequestExample {
    public static void main(String[] args) throws Exception {
        HttpClient client = HttpClient.newBuilder().build();

        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://google.com"))
            .method("HEAD", HttpRequest.BodyPublishers.noBody())
            .build();

        HttpResponse<Void> response = client.send(request,
            HttpResponse.BodyHandlers.discarding());

        // Returns an unmodifiable multi map view of this HttpHeaders.
        // The map contains key of string, with list of strings as
        // its value.
        HttpHeaders headers = response.headers();
        headers.map().forEach((key, values) ->
            System.out.printf("%s = %s%n", key, values));
    }
}

Here are the HTTP headers we got and printed out to the console screen:

:status = [301]
alt-svc = [quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,h3-T050=":443"; ma=2592000]
cache-control = [public, max-age=2592000]
content-length = [220]
content-type = [text/html; charset=UTF-8]
date = [Wed, 22 Apr 2020 14:41:49 GMT]
expires = [Fri, 22 May 2020 14:41:49 GMT]
location = [https://www.google.com/]
server = [gws]
x-frame-options = [SAMEORIGIN]
x-xss-protection = [0]

How do I read website content using HttpClient?

By Wayan Saryada in HTTP Client 0 Comment

The HTTP Client API can be used to request HTTP resources over the network. This new API was introduced as a new API in Java 11. It supports HTTP/1.1 and HTTP/2 and also support both synchronous and asynchronous programming models. The code snippet below show you how to use the new API to read the content of a website page.

In the code below we start by creating a new instance of HttpClient using the newHttpClient() static method. This is equivalent to calling newBuilder().build(). This give us an instance of HttpClient with default settings like using the “GET” request method the as the default. Then we create an HttpRequest object using the newBuilder() method, set the request URI and call the build() method to build the HttpRequest object.

Next we send the request by calling the send() method of the HttpClient object. This will sends the given request, blocking if necessary to get the response. The returned HttpResponse object contains the response status, headers, and body as handled by given response body handler.

package org.kodejava.example.httpclient;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;

public class ReadWebsiteContent {
    public static void main(String[] args) throws Exception {
        // Creates HttpClient object with default configuration.
        HttpClient httpClient = HttpClient.newHttpClient();

        // Creates HttpRequest object and set the URI to be requested, 
        // when not defined the default request method is the GET request.
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://httpie.org/hello"))
            .GET()
            .build();

        // Sends the request and print out the returned response.
        HttpResponse<String> response = httpClient.send(request,
            HttpResponse.BodyHandlers.ofString(StandardCharsets.UTF_8));

        System.out.println("Status Code: " + response.statusCode());
        System.out.println("Headers    : " + response.headers().toString());
        System.out.println("Body       : " + response.body());
    }
}

Here is the content of the website that we read using the code snippet above:

Status Code: 200
Headers    : java.net.http.HttpHeaders@2d299ad6 { {:status=[200], cf-cache-status=[DYNAMIC], cf-ray=[5875b78d5df2eb00-LAX], cf-request-id=[023d710c5b0000eb00b738f200000001], content-length=[116], content-type=[text/x-rst;charset=utf-8], date=[Tue, 21 Apr 2020 08:25:53 GMT], etag=["234b9a1fe19f125356a5396c8cc72d54493a2eef"], expect-ct=[max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"], server=[cloudflare], set-cookie=[__cfduid=d5bdb6d828be3bb85d0f1f4c2ff81041c1587457553; expires=Thu, 21-May-20 08:25:53 GMT; path=/; domain=.httpie.org; HttpOnly; SameSite=Lax]} }
Body       : 

Hello, World! 👋
~~~~~~~~~~~~~~~~

Thank you for trying out HTTPie 🥳

I hope this will become a friendship.

How do I execute HTTP Get request?

By Wayan Saryada in Apache HttpComponents 0 Comment

Below is an example using HttpClient library to retrieve the content of a website using Http Get request. We start by creating an instance of DefaultHttpClient class. We then define an Http get request by create an instance of HttpGet class and specify the Url of a website we are going to retrieve.

To execute the request we call the HttpClient.execute() method and pass the HttpGet as the arguments. This execution return an HttpResponse object. From this response object we can read the content of the website by accessing the getEntity().getContent() which will give us an InputStream to the content.

For more detail let’s see the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;

public class HttpGetExample {
    public static void main(String[] args) {
        HttpClient client = HttpClientBuilder.create().build();
        HttpGet request = new HttpGet("https://kodejava.org");

        try {
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();
            if (entity != null) {
                try (InputStream stream = entity.getContent()) {
                    BufferedReader reader =
                            new BufferedReader(new InputStreamReader(stream));
                    String line;
                    while ((line = reader.readLine()) != null) {
                        System.out.println(line);
                    }
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I get entity ContentType in HttpClient?

By Wayan Saryada in Apache HttpComponents 0 Comment

This code snippet show you how to get the content type of a result of executing an Http Get request. The ContentType can be obtained by using ContentType.getOrDefault() method and passing an HttpEntity as the arguments. The HttpEntity can be obtained from the HttpResponse object.

From the ContentType object we can get the mime-type by calling the getMimeType() method. This method will return a string value. To get the charset we can call the getCharset() method which will return a java.nio.charset.Charset object.

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.entity.ContentType;
import org.apache.http.impl.client.HttpClientBuilder;

import java.io.IOException;
import java.nio.charset.Charset;

public class EntityContentType {
    public static void main(String[] args) {
        EntityContentType demo = new EntityContentType();
        demo.getContentType("http://www.google.com");
        demo.getContentType("http://www.google.com/images/srpr/logo3w.png");
    }

    public void getContentType(String url) {
        try {
            HttpClient client = HttpClientBuilder.create().build();
            HttpGet request = new HttpGet(url);
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();
            showContentType(entity);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private void showContentType(HttpEntity entity) {
        ContentType contentType = ContentType.getOrDefault(entity);
        String mimeType = contentType.getMimeType();
        Charset charset = contentType.getCharset();

        System.out.println("MimeType = " + mimeType);
        System.out.println("Charset  = " + charset);
    }
}

The output of our code snippet are:

MimeType = text/html
Charset  = ISO-8859-1
MimeType = image/png
Charset  = null

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central

How do I get HTTP response body as a string?

By Wayan Saryada in Apache HttpComponents 4 Comments

This example show you how to get HTTP response body as a string when using the Apache HttpComponents library. To get the response body as a string we can use the EntityUtils.toString() method. This method read the content of an HttpEntity object content and return it as a string. The content will be converted using the character set from the entity object.

Let’s see the code snippet below:

package org.kodejava.example.httpclient;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.util.EntityUtils;

import java.io.IOException;

public class EntityAsString {
    public static void main(String[] args) {
        HttpClient client = HttpClientBuilder.create().build();
        HttpGet request = new HttpGet("https://kodejava.org");

        try {
            HttpResponse response = client.execute(request);
            HttpEntity entity = response.getEntity();

            // Read the contents of an entity and return it as a String.
            String content = EntityUtils.toString(entity);
            System.out.println(content);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Maven Dependencies

<!-- https://search.maven.org/remotecontent?filepath=org/apache/httpcomponents/httpclient/4.5.9/httpclient-4.5.9.jar -->
<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.9</version>
</dependency>

Maven Central