How do I use HttpClient with JSON parsing?

To use HttpClient with JSON parsing in Java, the most common approach is to combine the standard java.net.http.HttpClient with a JSON library like Jackson or Gson.

While HttpClient doesn’t have a built-in JSON parser, you can map the response body string to a Java object.

Using Jackson (Recommended)

Jackson is a powerful and widely-used library in the Java ecosystem. Here is how you can fetch a JSON response and parse it into a POJO (Plain Old Java Object).

First, define your data model:

public record Post(int userId, int id, String title, String body) {}

Then, use the HttpClient to fetch the data and ObjectMapper to parse it:

package org.kodejava.httpclient;

import com.fasterxml.jackson.databind.ObjectMapper;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

public class HttpClientJsonExample {
    public static void main(String[] args) {
        HttpClient client = HttpClient.newHttpClient();
        ObjectMapper mapper = new ObjectMapper();

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://jsonplaceholder.typicode.com/posts/1"))
                .header("Accept", "application/json")
                .build();

        try {
            // Send request and get response as a String
            HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());

            if (response.statusCode() == 200) {
                // Parse the JSON string into the Post object
                Post post = mapper.readValue(response.body(), Post.class);

                System.out.println("Post Title: " + post.title());
                System.out.println("Post Body: " + post.body());
            } else {
                System.err.println("Error: " + response.statusCode());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Tip: Custom BodyHandler

If you find yourself parsing JSON frequently, you can implement a custom HttpResponse.BodyHandler to handle the conversion automatically.

public static <T> HttpResponse.BodyHandler<T> asJson(Class<T> targetType) {
    return responseInfo -> HttpResponse.BodySubscribers.mapping(
            HttpResponse.BodySubscribers.ofString(StandardCharsets.UTF_8),
            body -> {
                try {
                    return new ObjectMapper().readValue(body, targetType);
                } catch (IOException e) {
                    throw new UncheckedIOException(e);
                }
            });
}

// Usage:
// HttpResponse<Post> response = client.send(request, asJson(Post.class));
// Post post = response.body();

Dependencies

Ensure you have the Jackson dependency in your pom.xml:

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.18.2</version>
</dependency>

This approach keeps your networking logic clean while leveraging the robustness of proven JSON libraries

How do I send async HTTP requests with HttpClient?

To send asynchronous HTTP requests in Java using the java.net.http.HttpClient, you use the sendAsync() method. This method returns a CompletableFuture<HttpResponse<T>>, allowing you to handle the response without blocking the main thread.

Here is a step-by-step example of how to implement this:

1. Basic Asynchronous GET Request

This example demonstrates how to fire a request and handle the result using thenAccept.

package org.kodejava.httpclient;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.concurrent.CompletableFuture;

public class AsyncRequestExample {
    public static void main(String[] args) {
        // 1. Create the HttpClient
        HttpClient client = HttpClient.newHttpClient();

        // 2. Build the HttpRequest
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create("https://jsonplaceholder.typicode.com/posts/1"))
                .GET()
                .build();

        // 3. Send the request asynchronously
        CompletableFuture<HttpResponse<String>> responseFuture =
                client.sendAsync(request, HttpResponse.BodyHandlers.ofString());

        // 4. Handle the response when it arrives
        responseFuture.thenAccept(response -> {
            System.out.println("Status Code: " + response.statusCode());
            System.out.println("Response Body: " + response.body());
        }).exceptionally(ex -> {
            System.err.println("Error occurred: " + ex.getMessage());
            return null;
        });

        // The program continues here immediately while the request is in flight
        System.out.println("Request sent! Doing other things...");

        // Optional: Block if you need to wait for the result before the program exits
        responseFuture.join();
    }
}

2. Chaining and Transforming Results

Because sendAsync returns a CompletableFuture, you can chain operations like extracting the body or converting JSON.

CompletableFuture<String> bodyFuture = client.sendAsync(request, HttpResponse.BodyHandlers.ofString())
        .thenApply(HttpResponse::body)       // Transform response to just the body
        .thenApply(String::toUpperCase);      // Further transform the string

bodyFuture.thenAccept(System.out::println);

Key Components

  • sendAsync(request, bodyHandler): The non-blocking counterpart to send().
  • HttpResponse.BodyHandlers: Defines how to handle the incoming data (e.g., ofString(), ofByteArray(), or ofFile()).
  • CompletableFuture: Provides methods like .thenApply() (map), .thenAccept() (consume), and .exceptionally() (error handling).

Best Practices

  • Reuse the Client: Don’t create a new HttpClient for every request. It’s designed to be long-lived and shared.
  • Executor Service: By default, HttpClient uses a default executor. For high-load applications, you can provide your own thread pool when building the client:
    HttpClient client = HttpClient.newBuilder()
                .executor(Executors.newFixedThreadPool(10))
                .build();
    
  • Join/Get: In a console application, use .join() or .get() at the very end to prevent the main method from finishing (and the JVM exiting) before the background thread completes.

How to Encode and Decode URLs in Java

In Java, you can encode and decode URLs using the java.net.URLEncoder and java.net.URLDecoder classes. These classes handle encoding and decoding in compliance with the application/x-www-form-urlencoded MIME type.
Here’s how you can encode and decode URLs:

Code Example

import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.net.URLDecoder;

public class URLEncoderDecoderExample {

    public static void main(String[] args) {
        try {
            // The String to be encoded
            String url = "https://example.com/query?name=John Doe&age=25";

            // Encoding URL
            String encodedUrl = URLEncoder.encode(url, "UTF-8");
            System.out.println("Encoded URL: " + encodedUrl);

            // Decoding URL
            String decodedUrl = URLDecoder.decode(encodedUrl, "UTF-8");
            System.out.println("Decoded URL: " + decodedUrl);

        } catch (UnsupportedEncodingException e) {
            e.printStackTrace(); // Handle exception if unsupported encoding is provided
        }
    }
}

Explanation:

  1. Encoding:
    • The URLEncoder.encode() method encodes special characters in the URL to make it safe for transmission over the network.
    • UTF-8 is typically used as the charset.
  2. Decoding:
    • The URLDecoder.decode() method decodes the string back to its original format.

Sample Output:

If the input is:

https://example.com/query?name=John Doe&age=25

After encoding:

https%3A%2F%2Fexample.com%2Fquery%3Fname%3DJohn+Doe%26age%3D25

After decoding:

https://example.com/query?name=John Doe&age=25

Notes:

  • Replace spaces with + in the encoded string. This is because spaces are not typically allowed in URLs, and encoding replaces them.
  • Use "UTF-8" because it’s the most widely used and supports all Unicode characters.

How to Resolve a Domain Name in Java

Here are common ways to resolve domain names in Java, from simplest to more advanced use cases.

Basic A/AAAA record lookup (IPv4/IPv6)

  • Uses the system resolver and OS DNS settings.
  • Returns all IPs (both IPv4 and IPv6 where available).
import java.net.InetAddress;
import java.net.UnknownHostException;

public class DnsLookup {
    public static void main(String[] args) {
        String host = "example.com";
        try {
            InetAddress[] addresses = InetAddress.getAllByName(host);
            for (InetAddress addr : addresses) {
                System.out.println(addr.getHostAddress());
            }
        } catch (UnknownHostException e) {
            System.err.println("DNS lookup failed: " + e.getMessage());
        }
    }
}

Notes for InetAddress:

  • No direct per-call timeout configuration (it relies on OS resolver timeouts).
  • Caching is controlled by security properties:
    • -Dnetworkaddress.cache.ttl=60 (seconds; -1 = forever; default often JVM-dependent)
    • -Dnetworkaddress.cache.negative.ttl=10
  • Prefer IPv6: -Djava.net.preferIPv6Addresses=true

Asynchronous lookups

import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.concurrent.CompletableFuture;

public class AsyncDns {
    public static CompletableFuture<InetAddress[]> resolve(String host) {
        return CompletableFuture.supplyAsync(() -> {
            try {
                return InetAddress.getAllByName(host);
            } catch (UnknownHostException e) {
                throw new RuntimeException(e);
            }
        });
    }
}

Reverse DNS (PTR)

  • Basic: addr.getHostName() may trigger reverse lookup (can be slow or cached).
InetAddress addr = InetAddress.getByName("93.184.216.34");
String reverse = addr.getHostName(); // may do a PTR lookup

Query specific DNS record types (MX, TXT, SRV, PTR) or specific DNS servers

Option 1: JNDI DNS (built-in, configurable)

import javax.naming.directory.*;
import javax.naming.*;
import java.util.Hashtable;

public class JndiDns {
    public static void main(String[] args) throws NamingException {
        String domain = "example.com";
        Hashtable<String, String> env = new Hashtable<>();
        env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.dns.DnsContextFactory");
        // Use specific DNS server(s) (optional)
        env.put(Context.PROVIDER_URL, "dns://8.8.8.8 dns://1.1.1.1");
        // Timeouts in milliseconds (optional)
        env.put("com.sun.jndi.dns.timeout.initial", "2000");
        env.put("com.sun.jndi.dns.timeout.retries", "1");

        DirContext ctx = new InitialDirContext(env);
        Attributes attrs = ctx.getAttributes(domain, new String[] {"MX", "TXT", "A"});
        Attribute mx = attrs.get("MX");
        if (mx != null) {
            for (int i = 0; i < mx.size(); i++) System.out.println("MX: " + mx.get(i));
        }
        Attribute txt = attrs.get("TXT");
        if (txt != null) {
            for (int i = 0; i < txt.size(); i++) System.out.println("TXT: " + txt.get(i));
        }
        Attribute a = attrs.get("A");
        if (a != null) {
            for (int i = 0; i < a.size(); i++) System.out.println("A: " + a.get(i));
        }
    }
}

Notes:

  • JNDI DNS supports MX, TXT, SRV, CNAME, PTR, etc.
  • You can set specific DNS servers via PROVIDER_URL.

Option 2: Use a dedicated DNS library (e.g., dnsjava)

  • Recommended for fine control, timeouts, EDNS, DNSSEC (if needed), or custom resolvers.

Maven Dependency:

<dependency>
    <groupId>dnsjava</groupId>
    <artifactId>dnsjava</artifactId>
    <version>3.6.3</version>
    <type>bundle</type>
</dependency>

Lookup A/AAAA with custom resolver and timeout:

import org.xbill.DNS.*;

public class DnsJavaExample {
    public static void main(String[] args) throws Exception {
        String domain = "example.com";
        Resolver resolver = new SimpleResolver("8.8.8.8");
        resolver.setTimeout(Duration.ofSeconds(2));
        Name name = Name.fromString(domain + ".");
        Record[] records = new Lookup(name, Type.A).run();
        if (records != null) {
            for (Record r : records) System.out.println(r.rdataToString());
        }
    }
}

SRV/TXT example:

import org.xbill.DNS.*;

Name srvName = Name.fromString("_sip._tcp.example.com.");
Record[] srv = new Lookup(srvName, Type.SRV).run();
if (srv != null) {
    for (Record r : srv) System.out.println(r.rdataToString());
}

Name txtName = Name.fromString("example.com.");
Record[] txt = new Lookup(txtName, Type.TXT).run();
if (txt != null) {
    for (Record r : txt) System.out.println(r.rdataToString());
}

Spring/Jakarta usage example (service component)

import org.springframework.stereotype.Service;
import java.net.InetAddress;

@Service
public class DnsService {
    public String[] resolve(String host) {
        try {
            return java.util.Arrays.stream(InetAddress.getAllByName(host))
                    .map(InetAddress::getHostAddress)
                    .toArray(String[]::new);
        } catch (Exception e) {
            return new String[0];
        }
    }
}

Practical tips

  • Retry logic: DNS failures are often transient. Consider simple retries with backoff when appropriate.
  • Validate input: Ensure the host is a valid hostname to avoid unnecessary exceptions.
  • Respect caching: Tune networkaddress.cache.ttl for your runtime environment to balance freshness and performance.
  • Split-horizon DNS: In containerized/cloud setups, behavior may differ between environments. Test where it runs.
  • Don’t hardcode IPs unless necessary; prefer hostnames to benefit from DNS-based failover.

How to Get Hostname and IP Address in Java

Here are the most common and reliable ways to get hostnames and IP addresses in Java (Java 21). Pick the approach that matches your runtime (desktop app, server app, behind proxy, etc.).

  1. Quick local host info
    • Good for simple cases, but can return 127.0.0.1 if your host isn’t configured in DNS/hosts.
    import java.net.InetAddress;
    
    public class LocalHostQuick {
        public static void main(String[] args) throws Exception {
            InetAddress local = InetAddress.getLocalHost();
            System.out.println("Host name: " + local.getHostName());
            System.out.println("Canonical host name: " + local.getCanonicalHostName());
            System.out.println("IP address: " + local.getHostAddress());
        }
    }
    
  2. Robust way: list network interfaces
    • Picks non-loopback, non-virtual, up interfaces; prefers IPv4 but supports IPv6.
    import java.net.Inet4Address;
    import java.net.InetAddress;
    import java.net.NetworkInterface;
    import java.util.ArrayList;
    import java.util.Enumeration;
    import java.util.List;
    
    public class LocalAddresses {
        public static void main(String[] args) throws Exception {
            List<InetAddress> addresses = new ArrayList<>();
            for (Enumeration<NetworkInterface> ifaces = NetworkInterface.getNetworkInterfaces(); ifaces.hasMoreElements(); ) {
                NetworkInterface nif = ifaces.nextElement();
                if (!nif.isUp() || nif.isLoopback() || nif.isVirtual()) continue;
    
                for (Enumeration<InetAddress> addrs = nif.getInetAddresses(); addrs.hasMoreElements(); ) {
                    InetAddress addr = addrs.nextElement();
                    if (addr.isLoopbackAddress() || addr.isLinkLocalAddress()) continue; // skip 127.0.0.1, fe80::
                    addresses.add(addr);
                }
            }
    
            // Prefer IPv4 for display
            addresses.stream()
                     .sorted((a, b) -> Boolean.compare(b instanceof Inet4Address, a instanceof Inet4Address))
                     .forEach(a -> System.out.println(a.getHostAddress() + " (" + a.getHostName() + ")"));
        }
    }
    
  3. DNS lookup: resolve a hostname to IPs
    • Useful to get IPs for a remote host or reverse lookup a specific IP.
    import java.net.InetAddress;
    
    public class ResolveHost {
        public static void main(String[] args) throws Exception {
            String host = "example.com"; // replace with your host
            InetAddress[] all = InetAddress.getAllByName(host);
            for (InetAddress inet : all) {
                System.out.println(host + " -> " + inet.getHostAddress());
            }
    
            // Reverse lookup of a specific IP
            InetAddress ip = InetAddress.getByName("203.0.113.10"); // placeholder IP
            System.out.println(ip.getHostAddress() + " reverse -> " + ip.getCanonicalHostName());
        }
    }
    
  4. In a Spring MVC/Jakarta web app
    • Getting the client IP (taking proxies into account) and server info. Utility to extract client IP (checks common proxy headers, then falls back):
    import jakarta.servlet.http.HttpServletRequest;
    import java.util.List;
    
    public class IpUtils {
        private static final List<String> IP_HEADER_CANDIDATES = List.of(
            "X-Forwarded-For",
            "X-Real-IP",
            "CF-Connecting-IP",
            "Fastly-Client-Ip",
            "True-Client-Ip",
            "X-Cluster-Client-Ip",
            "Forwarded",
            "Forwarded-For"
        );
    
        public static String getClientIp(HttpServletRequest request) {
            for (String header : IP_HEADER_CANDIDATES) {
                String value = request.getHeader(header);
                if (value != null && !value.isBlank() && !"unknown".equalsIgnoreCase(value)) {
                    // X-Forwarded-For can contain a list: client, proxy1, proxy2...
                    String first = value.split(",")[0].trim();
                    if (!first.isBlank()) return first;
                }
            }
            return request.getRemoteAddr();
        }
    }
    

Controller example:

import jakarta.servlet.http.HttpServletRequest;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

import java.net.InetAddress;
import java.util.Map;

@RestController
public class NetInfoController {

    @GetMapping("/net-info")
    public Map<String, String> netInfo(HttpServletRequest request) throws Exception {
        String clientIp = IpUtils.getClientIp(request);

        // Server info via servlet and InetAddress
        String serverIp = request.getLocalAddr();     // or request.getServerName()
        String serverHostName = InetAddress.getLocalHost().getHostName();

        return Map.of(
            "clientIp", clientIp,
            "serverIp", serverIp,
            "serverHostName", serverHostName
        );
    }
}

Notes and tips

  • getLocalHost may return 127.0.0.1 if your machine’s hostname isn’t resolvable. Enumerating NetworkInterface is more reliable.
  • For containers/Kubernetes, you may prefer:
    • The interface enumeration approach, or
    • Reading an environment variable like HOSTNAME (if set by the platform).
  • Reverse DNS (getCanonicalHostName) depends on network/DNS config and may be slow; cache if needed.
  • Always handle exceptions: UnknownHostException, SocketException.
  • When behind proxies/load balancers, only trust client-IP headers if your infrastructure sanitizes them; otherwise they can be spoofed.