How do I read or download web page content?

You want to create a program that read a webpage content of a website page. The example below use the URL class to create a connection to the website. You create a new URL object and pass the URL information of a page. After the object created you can open a stream connection using the openStream() method of the URL object.

Next, you can read the stream using the BufferedReader object. This reader allows you to read line by line from the stream. To write it to a file create a writer using the BufferedWriter object and specify the file name where the downloaded page will be stored.

When all the content are read from the stream and stored in a file close the BufferedReader object and the BufferedWriter object at the end of your program.

package org.kodejava.net;

import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;

public class UrlReadPageDemo {
    public static void main(String[] args) {
        try {
            URL url = new URL("https://kodejava.org");

            BufferedReader reader = new BufferedReader(
                    new InputStreamReader(url.openStream(), StandardCharsets.UTF_8));
            BufferedWriter writer = new BufferedWriter(
                    new FileWriter("data.html", StandardCharsets.UTF_8));

            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
                writer.write(line);
                writer.newLine();
            }

            reader.close();
            writer.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

Author
Recent Posts

Wayan

Founder at Kode Java

A programmer, runner and recreational diver, lives on the island of Bali, Indonesia. Programming in Java, Spring Framework, Hibernate / JPA, MySQL. To support me working on this project, buy me a cup of coffee, every little bit helps. Thank you.

Leave a ReplyCancel reply