How do I use Java 11 HttpClient with a proxy server?

To use Java 11’s HttpClient with a proxy server, you need to configure the proxy using the ProxySelector class. The ProxySelector allows you to specify a proxy through which HTTP requests should pass.

Here’s an example of how you can configure and use HttpClient with a proxy server:

Step-by-Step Example

Example with Proxy Configuration

package org.kodejava.net.http;

import java.io.IOException;
import java.net.*;
import java.net.http.*;
import java.util.List;

public class HttpClientWithProxyExample {

   public static void main(String[] args) {
      // Create a ProxySelector with a specified proxy server
      ProxySelector proxySelector = new ProxySelector() {
         @Override
         public List<Proxy> select(URI uri) {
            // Returning the proxy details
            return List.of(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.example.com", 8080)));
         }

         @Override
         public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
            System.err.println("Proxy connection failed: " + ioe.getMessage());
         }
      };

      // Build an HttpClient with the proxy selector
      HttpClient httpClient = HttpClient.newBuilder()
              .proxy(proxySelector)
              .build();

      // Create an HttpRequest
      HttpRequest request = HttpRequest.newBuilder()
              .uri(URI.create("https://example.com"))
              .GET()
              .build();

      try {
         // Send the request
         HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
         System.out.println("Response status code: " + response.statusCode());
         System.out.println("Response body: " + response.body());
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
}

Key Points in the Example:

  1. ProxySelector Implementation:
    • A custom ProxySelector is created, which is capable of specifying a proxy (Proxy.Type.HTTP) and its hostname (proxy.example.com) and port (8080).
  2. Build HttpClient:
    • The HttpClient is created using the HttpClient.newBuilder() API, and the proxy method is used to assign the custom ProxySelector.
  3. Send an HTTP Request:
    • Create a request using HttpRequest.newBuilder and send the request using HttpClient.send().

Example with Proxy Authentication

If your proxy server requires authentication, you can configure it using the Authenticator class:

package org.kodejava.net.http;

import java.io.IOException;
import java.net.*;
import java.net.http.*;
import java.util.List;

public class HttpClientWithProxyAuthExample {

   public static void main(String[] args) {
      // Set system-wide default authenticator
      Authenticator.setDefault(new Authenticator() {
         @Override
         protected PasswordAuthentication getPasswordAuthentication() {
            if (getRequestorType() == RequestorType.PROXY) {
               return new PasswordAuthentication("proxyUsername", "proxyPassword".toCharArray());
            }
            return null;
         }
      });

      // Create ProxySelector with proxy details
      ProxySelector proxySelector = new ProxySelector() {
         @Override
         public List<Proxy> select(URI uri) {
            return List.of(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.example.com", 8080)));
         }

         @Override
         public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
            System.err.println("Proxy connection failed: " + ioe.getMessage());
         }
      };

      // Build the HttpClient with the proxy selector and default authenticator
      HttpClient httpClient = HttpClient.newBuilder()
              .proxy(proxySelector)
              .authenticator(Authenticator.getDefault())
              .build();

      // Create an HTTP Request
      HttpRequest request = HttpRequest.newBuilder()
              .uri(URI.create("http://example.com"))
              .GET()
              .build();

      try {
         // Send HTTP request
         HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
         System.out.println("Response status code: " + response.statusCode());
         System.out.println("Response body: " + response.body());
      } catch (Exception e) {
         e.printStackTrace();
      }
   }
}

Key Points in Proxy Authentication:

  1. Authenticator Class:
    • Use Authenticator.setDefault() to set a global proxy authenticator. Override the getPasswordAuthentication() method to return the credentials for the proxy server.
  2. Proxy and Authentication Configuration:
    • Combine the ProxySelector and the Authenticator to handle both proxy selection and authentication credentials.

Additional Notes:

  • Replace "proxy.example.com" with your proxy server’s hostname and 8080 with the proxy’s port number.
  • Replace proxyUsername and proxyPassword with the credentials required for accessing the proxy server.
  • For HTTPS requests, ensure the proxy supports secure traffic.

This approach allows you to work with both HTTP and HTTPS requests through a proxy server.

How do I create a proxy server using ProxySelector and Socket in Java?

Creating a proxy server using ProxySelector and Socket in Java involves a few steps. ProxySelector is used to determine which proxy to use for connections, while Socket facilitates communication with the client and/or the target server behind the proxy.

Here’s a step-by-step guide:


1. Setup ProxySelector

A custom ProxySelector can be created to define the proxy behavior. The select(URI uri) method of ProxySelector decides which proxy should be used for a given URI.

2. Create a Server Socket

A ServerSocket will listen for incoming client requests. Once a client is connected, communication between client and the target server through the proxy is handled via sockets.

3. Communicate between the Client and Remote Server

You’ll use a Socket to connect to the remote server via the proxy. The data from the client is forwarded to the remote server, and the response from the remote server is sent back to the client.

Example Code:

package org.kodejava.net;

import java.io.*;
import java.net.*;
import java.util.ArrayList;
import java.util.List;

public class ProxyServerExample {

   public static void main(String[] args) {
      int localPort = 8888; // The port your proxy server will listen on

      try (ServerSocket serverSocket = new ServerSocket(localPort)) {
         System.out.println("Proxy server is running on port: " + localPort);

         // Custom ProxySelector
         ProxySelector.setDefault(new CustomProxySelector());

         while (true) {
            // Accept incoming client connections
            Socket clientSocket = serverSocket.accept();
            System.out.println("Accepted connection from: " + clientSocket.getRemoteSocketAddress());

            // Handle the connection in a separate thread
            new Thread(new ProxyHandler(clientSocket)).start();
         }
      } catch (IOException e) {
         e.printStackTrace();
      }
   }

   // Custom ProxySelector class
   static class CustomProxySelector extends ProxySelector {
      @Override
      public List<Proxy> select(URI uri) {
         List<Proxy> proxies = new ArrayList<>();
         // Specify your proxy details here
         proxies.add(new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.example.com", 8080)));
         return proxies;
      }

      @Override
      public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
         System.err.printf("Failed to connect to proxy: %s%n", sa);
      }
   }

   // ProxyHandler to manage client-server communication
   static class ProxyHandler implements Runnable {
      private final Socket clientSocket;

      public ProxyHandler(Socket clientSocket) {
         this.clientSocket = clientSocket;
      }

      @Override
      public void run() {
         try (InputStream clientIn = clientSocket.getInputStream();
              OutputStream clientOut = clientSocket.getOutputStream()) {

            BufferedReader clientReader = new BufferedReader(new InputStreamReader(clientIn));
            OutputStreamWriter clientWriter = new OutputStreamWriter(clientOut);

            // Read the first line from the client (usually HTTP request line)
            String clientRequestLine = clientReader.readLine();
            System.out.println("Client Request: " + clientRequestLine);

            if (clientRequestLine != null) {
               // Extract the target host and port from the client request
               String[] requestParts = clientRequestLine.split(" ");
               String targetHost = new URI(requestParts[1]).getHost();
               int targetPort = new URI(requestParts[1]).getPort() != -1 ? new URI(requestParts[1]).getPort() : 80;

               // Connect to the target server through the proxy
               List<Proxy> proxies = ProxySelector.getDefault().select(new URI(requestParts[1]));
               Proxy proxy = proxies.isEmpty() ? Proxy.NO_PROXY : proxies.get(0);

               System.out.println("Connecting to: " + targetHost + ":" + targetPort + " via " + proxy);

               // Connect to the target server
               Socket serverSocket;
               if (proxy.type().equals(Proxy.Type.DIRECT)) {
                  // A direct connection to the target host
                  serverSocket = new Socket(targetHost, targetPort);
               } else if (proxy.type().equals(Proxy.Type.HTTP)) {
                  // Connect through an HTTP proxy
                  InetSocketAddress proxyAddress = (InetSocketAddress) proxy.address();
                  serverSocket = new Socket(proxyAddress.getHostName(), proxyAddress.getPort());
               } else {
                  throw new IOException("Unsupported proxy type: " + proxy.type());
               }

               // Forward client's request to the server
               try (OutputStream serverOut = serverSocket.getOutputStream();
                    InputStream serverIn = serverSocket.getInputStream()) {

                  serverOut.write((clientRequestLine + "\r\n").getBytes());
                  serverOut.flush();

                  // Forward other client headers to the server
                  String line;
                  while ((line = clientReader.readLine()) != null && !line.isEmpty()) {
                     serverOut.write((line + "\r\n").getBytes());
                     serverOut.flush();
                  }
                  serverOut.write("\r\n".getBytes());
                  serverOut.flush();

                  // Read server response and forward to the client
                  byte[] buffer = new byte[8192];
                  int bytesRead;
                  while ((bytesRead = serverIn.read(buffer)) != -1) {
                     clientOut.write(buffer, 0, bytesRead);
                     clientOut.flush();
                  }
               }

               serverSocket.close();
            }

         } catch (Exception e) {
            e.printStackTrace();
         } finally {
            try {
               clientSocket.close();
            } catch (IOException e) {
               e.printStackTrace();
            }
         }
      }
   }
}

How It Works:

  1. ProxySelector:
    • CustomProxySelector determines which proxy to use for a particular URI.
    • Modify the proxy host and port in CustomProxySelector to suit your requirements.
  2. Proxy Server:
    • The proxy server listens for incoming client requests on a specified port using ServerSocket.
  3. Handling Requests:
    • A new thread is started for each incoming client connection to handle the client’s request and forward it to the target server via the proxy.
  4. Forwarding Traffic:
    • ProxyHandler reads data from the client, connects to the target server through the proxy (if required), forwards the data, and sends the response back to the client.

Key Considerations:

  • Validate input to ensure the proxy server cannot be exploited.
  • Add proper error handling/logging.
  • Consider using a library like Netty or Jetty for production-grade performance.
  • This example is a basic proxy that works for simple HTTP requests. For HTTPS, consider HTTP CONNECT tunneling.

This structure forms a good starting point for a custom proxy server in Java!

How do I use Proxy class to configure HTTP and SOCKS proxies in Java?

In Java, the Proxy class (available in the java.net package) allows you to configure and use HTTP or SOCKS proxies when making network connections. With this class, you can control the routing of requests through a proxy server.
Here’s an explanation and examples of how to use the Proxy class for both HTTP and SOCKS proxies:

1. Creating a Proxy Instance

To use a proxy, you create an instance of the Proxy class by specifying:
– A Proxy.Type (HTTP or SOCKS).
– An address or host of the proxy server using the InetSocketAddress.

Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.example.com", 8080));

For SOCKS proxies:

Proxy proxy = new Proxy(Proxy.Type.SOCKS, new InetSocketAddress("proxy.example.com", 1080));

2. Using Proxy with HttpURLConnection

When making HTTP requests using HttpURLConnection, you pass the created Proxy instance to the openConnection() method:

package org.kodejava.net;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.URL;

public class ProxyExample {
    public static void main(String[] args) {
        try {
            // Proxy configuration
            Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("proxy.example.com", 8080));

            // URL to connect to
            URL url = new URL("https://www.example.com");

            // Open connection with the proxy
            HttpURLConnection connection = (HttpURLConnection) url.openConnection(proxy);

            // Send GET request
            connection.setRequestMethod("GET");

            // Reading response
            BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
            }
            reader.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

3. Using Proxy with Socket Connections

The Proxy class can also be used with raw Socket or SocketChannel for SOCKS proxies:

package org.kodejava.net;

import java.io.InputStream;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.Socket;

public class SocksProxyExample {
    public static void main(String[] args) {
        try {
            // Create a SOCKS proxy
            Proxy proxy = new Proxy(Proxy.Type.SOCKS, new InetSocketAddress("socksproxy.example.com", 1080));

            // Connect to a host through the proxy
            Socket socket = new Socket(proxy);
            socket.connect(new InetSocketAddress("example.com", 80));

            // Send HTTP request manually
            OutputStream output = socket.getOutputStream();
            output.write("GET / HTTP/1.1\r\nHost: example.com\r\n\r\n".getBytes());
            output.flush();

            // Read response
            InputStream input = socket.getInputStream();
            int data;
            while ((data = input.read()) != -1) {
                System.out.print((char) data);
            }

            // Close the connection
            socket.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

4. Global Proxy Configuration

If you want to configure a proxy globally for all network connections, you can set system properties:
For HTTP proxy:

System.setProperty("http.proxyHost", "proxy.example.com");
System.setProperty("http.proxyPort", "8080");

For HTTPS proxy:

System.setProperty("https.proxyHost", "proxy.example.com");
System.setProperty("https.proxyPort", "8080");

For SOCKS proxy:

System.setProperty("socksProxyHost", "socksproxy.example.com");
System.setProperty("socksProxyPort", "1080");

To disable certain hosts (bypass proxy):

System.setProperty("http.nonProxyHosts", "localhost|127.0.0.1|*.example.com");

5. Proxy Authentication

If your proxy requires authentication, you need to configure an Authenticator:

package org.kodejava.net;

import java.net.Authenticator;
import java.net.PasswordAuthentication;

public class ProxyAuthenticator {
    public static void main(String[] args) {
        Authenticator.setDefault(new Authenticator() {
            @Override
            protected PasswordAuthentication getPasswordAuthentication() {
                return new PasswordAuthentication("username", "password".toCharArray());
            }
        });

        // Set the global proxy settings
        System.setProperty("http.proxyHost", "proxy.example.com");
        System.setProperty("http.proxyPort", "8080");

        // Make HTTP requests as usual
    }
}

Choosing Between Proxy and Global Configuration

  • Use the Proxy class if you only want specific connections to use a proxy.
  • Use global properties (System.setProperty) if the proxy should be used for all connections.

How to configure a proxy in Maven settings?

When we work behind a proxy server, we need to configure Maven to be able to connect to the internet. To enable proxy we can configure Maven settings.xml file, either in ${M2_HOME}/conf/settings.xml or ${user.home}/.m2/settings.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<settings>
    .
    .
    <proxies>
        <proxy>
            <id>my-proxy</id>
            <active>true</active>
            <protocol>http</protocol>
            <host>proxy.example.org</host>
            <port>8080</port>
            <username>username</username>
            <password>password</password>
            <nonProxyHosts>*.example.org|*.example.com</nonProxyHosts>
        </proxy>
    </proxies>
    .
    .
</settings>

The <proxy> element in the configuration above contains the information about the proxy server. These include information about the host, port, username and password. Set these elements to match your proxy server configuration.

How do I set a proxy for HttpClient?

In this example you will see how to configure proxy when using the Apache Commons HttpClient library.

package org.kodejava.commons.httpclient;

import org.apache.commons.httpclient.Credentials;
import org.apache.commons.httpclient.HostConfiguration;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.UsernamePasswordCredentials;
import org.apache.commons.httpclient.auth.AuthScope;
import org.apache.commons.httpclient.methods.GetMethod;

import java.io.IOException;

public class HttpGetProxy {
    private static final String PROXY_HOST = "proxy.host.com";
    private static final int PROXY_PORT = 8080;

    public static void main(String[] args) {
        HttpClient client = new HttpClient();
        HttpMethod method = new GetMethod("https://kodejava.org");

        HostConfiguration config = client.getHostConfiguration();
        config.setProxy(PROXY_HOST, PROXY_PORT);

        String username = "guest";
        String password = "s3cr3t";
        Credentials credentials = new UsernamePasswordCredentials(username, password);
        AuthScope authScope = new AuthScope(PROXY_HOST, PROXY_PORT);

        client.getState().setProxyCredentials(authScope, credentials);

        try {
            client.executeMethod(method);

            if (method.getStatusCode() == HttpStatus.SC_OK) {
                String response = method.getResponseBodyAsString();
                System.out.println("Response = " + response);
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            method.releaseConnection();
        }
    }
}

Maven Dependencies

<dependency>
    <groupId>commons-httpclient</groupId>
    <artifactId>commons-httpclient</artifactId>
    <version>3.1</version>
</dependency>

Maven Central