The code snippet below show you how to use the FileUtils.copyURLToFile(URL, File) method of the Apache Commons IO library to help you to copy the contents of a URL directly into a file.
package org.kodejava.commons.io;
import org.apache.commons.io.FileUtils;
import java.io.File;
import java.io.IOException;
import java.net.URL;
public class URLToFile {
public static void main(String[] args) {
try {
URL url = new URL("https://www.google.com");
File destination = new File("google.html");
// Copy bytes from the URL to the destination file.
FileUtils.copyURLToFile(url, destination);
} catch (IOException e) {
e.printStackTrace();
}
}
}
In the code example below we will extract information regarding the HTTP (Hypertext Transport Protocol) from the request object (HttpServletRequest). We will extract the protocol used (http / https), server name and its assigned port number. We will also read our application context path, servlet path, path info and the query string. If we combine all the previously mentioned information we will get something equals to the value returned by request.getRequestURL() method.
Let’s check the code snippet below to see what method of the HttpServletRequest class that we can call to get the information regarding the HTTP request object that we can collect.
package org.kodejava.net;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.URL;
import java.nio.charset.StandardCharsets;
public class URLDemo {
public static void main(String[] args) {
try {
// Creating a url object by specifying each parameter separately, including
// the protocol, hostname, port number, and the page name
URL url = new URL("https", "kodejava.org", 443, "/index.php");
// We can also specify the address in a single line
url = new URL("https://kodejava.org:443/index.php");
BufferedReader reader = new BufferedReader(
new InputStreamReader(url.openStream(), StandardCharsets.UTF_8));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
reader.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
You want to create a program that read a webpage content of a website page. The example below use the URL class to create a connection to the website. You create a new URL object and pass the URL information of a page. After the object created you can open a stream connection using the openStream() method of the URL object.
Next, you can read the stream using the BufferedReader object. This reader allows you to read line by line from the stream. To write it to a file create a writer using the BufferedWriter object and specify the file name where the downloaded page will be stored.
When all the content are read from the stream and stored in a file close the BufferedReader object and the BufferedWriter object at the end of your program.
package org.kodejava.net;
import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;
public class UrlReadPageDemo {
public static void main(String[] args) {
try {
URL url = new URL("https://kodejava.org");
BufferedReader reader = new BufferedReader(
new InputStreamReader(url.openStream(), StandardCharsets.UTF_8));
BufferedWriter writer = new BufferedWriter(
new FileWriter("data.html", StandardCharsets.UTF_8));
String line;
while ((line = reader.readLine()) != null) {
System.out.println(line);
writer.write(line);
writer.newLine();
}
reader.close();
writer.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
