How do I read or download web page content?

By Wayan in Core API, Networking 0 Comment

You want to create a program that read a webpage content of a website page. The example below use the URL class to create a connection to the website. You create a new URL object and pass the URL information of a page. After the object created you can open a stream connection using the openStream() method of the URL object.

Next, you can read the stream using the BufferedReader object. This reader allows you to read line by line from the stream. To write it to a file create a writer using the BufferedWriter object and specify the file name where the downloaded page will be stored.

When all the content are read from the stream and stored in a file close the BufferedReader object and the BufferedWriter object at the end of your program.

package org.kodejava.net;

import java.io.*;
import java.net.URL;
import java.nio.charset.StandardCharsets;

public class UrlReadPageDemo {
    public static void main(String[] args) {
        try {
            URL url = new URL("https://kodejava.org");

            BufferedReader reader = new BufferedReader(
                    new InputStreamReader(url.openStream(), StandardCharsets.UTF_8));
            BufferedWriter writer = new BufferedWriter(
                    new FileWriter("data.html", StandardCharsets.UTF_8));

            String line;
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
                writer.write(line);
                writer.newLine();
            }

            reader.close();
            writer.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
Wayan
Latest posts by Wayan (see all)

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.