How do I set fetch size for large queries?

To set the fetch size for large queries in Java using JDBC, you use the setFetchSize(int rows) method on a Statement or PreparedStatement object.

This gives the JDBC driver a hint as to the number of rows that should be fetched from the database when more rows are needed. This is particularly useful for large result sets to avoid loading everything into memory at once or to reduce the number of network round-trips.

Using JDBC Statement

Here is how you can apply it to a standard Statement:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;

// ... existing code ...
try (Connection conn = DriverManager.getConnection(url, user, password);
     Statement stmt = conn.createStatement()) {

    // Set the fetch size to 100 rows
    stmt.setFetchSize(100);

    try (ResultSet rs = stmt.executeQuery("SELECT * FROM very_large_table")) {
        while (rs.next()) {
            // Process rows
        }
    }
} catch (Exception e) {
    e.printStackTrace();
}

Important Considerations

  1. Driver Support: setFetchSize is a hint. Not all JDBC drivers honor this value in the same way.
  2. MySQL Specifics: By default, the MySQL driver fetches all rows into memory. To stream results (fetch row-by-row), you must set the fetch size to Integer.MIN_VALUE and use a forward-only, read-only result set:
    stmt.setFetchSize(Integer.MIN_VALUE);
    
  3. Oracle Specifics: Oracle has a default fetch size (usually 10). Increasing this to 100 or 500 can significantly improve performance for large queries.

  4. Memory vs. Network:
    • Small fetch size: Saves memory but increases network round-trips (slower).
    • Large fetch size: Reduces network round-trips (faster) but consumes more client-side memory.

Using Spring Data JPA / Jakarta EE

Since your project uses Spring Data JPA, you can also set the fetch size using the @QueryHints annotation on your repository methods:

import jakarta.persistence.QueryHint;
import org.springframework.data.jpa.repository.QueryHints;
import static org.hibernate.jpa.HibernateHints.HINT_FETCH_SIZE;

@QueryHints(value = @QueryHint(name = HINT_FETCH_SIZE, value = "100"))
List<User> findAllByStatus(String status);

How do I use auto-generated keys in JDBC?

To use auto-generated keys in JDBC (like an AUTO_INCREMENT primary key), you need to follow a three-step process: notify the statement you want the keys, execute the update, and then retrieve them from a special ResultSet.

Here is a practical example using PreparedStatement:

1. Prepare the Statement

When creating your PreparedStatement, you must pass the constant Statement.RETURN_GENERATED_KEYS to let the driver know you want the keys back.

String sql = "INSERT INTO users (username, email) VALUES (?, ?)";
try (PreparedStatement pstmt = connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) {
    pstmt.setString(1, "john_doe");
    pstmt.setString(2, "[email protected]");

    // ...
}

2. Execute and Retrieve

After calling executeUpdate(), use getGeneratedKeys() to fetch the IDs. Even if you only inserted one row, the keys are returned as a ResultSet because some databases support multiple generated keys per row or batch inserts.

int affectedRows = pstmt.executeUpdate();

if (affectedRows > 0) {
    try (ResultSet generatedKeys = pstmt.getGeneratedKeys()) {
        if (generatedKeys.next()) {
            long id = generatedKeys.getLong(1);
            System.out.println("Inserted record ID: " + id);
        }
    }
}

Complete Example

Based on standard JDBC practices, here is how the implementation usually looks:

package org.kodejava.jdbc;

import java.sql.*;

public class GetGeneratedKey {
    public static void main(String[] args) {
        String url = "jdbc:mysql://localhost:3306/your_db";
        String user = "root";
        String password = "password";

        String sql = "INSERT INTO authors (name) VALUES (?)";

        try (Connection conn = DriverManager.getConnection(url, user, password);
             PreparedStatement pstmt = conn.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) {

            pstmt.setString(1, "Herbert Schildt");
            pstmt.executeUpdate();

            // Retrieve the generated key
            try (ResultSet rs = pstmt.getGeneratedKeys()) {
                if (rs.next()) {
                    long generatedId = rs.getLong(1);
                    System.out.println("Generated ID: " + generatedId);
                }
            }
        } catch (SQLException e) {
            e.printStackTrace();
        }
    }
}

Key Points to Remember:

  • Constant: Don’t forget Statement.RETURN_GENERATED_KEYS. Without it, getGeneratedKeys() will return an empty result set or throw an exception depending on the driver.
  • Column Index: Usually, the generated key is in the first column of the returned ResultSet, so rs.getLong(1) is standard.
  • Database Support: Most modern databases (MySQL, PostgreSQL, SQL Server, Oracle) support this, though the internal mechanism (Sequences vs. Identity columns) varies.

How do I batch insert data with JDBC?

To batch insert data with JDBC, you typically use the addBatch() and executeBatch() methods. This is much more efficient than executing individual INSERT statements because it reduces the number of round-trips between your application and the database.

The most common and secure way to do this is with a PreparedStatement.

Batch Insert with PreparedStatement

Using PreparedStatement allows you to define a template query and then add multiple sets of parameters to a single batch.

package org.kodejava.jdbc;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;

public class JDBCBatchInsert {
    public static void main(String[] args) {
        String url = "jdbc:mysql://localhost:3306/your_db";
        String user = "user";
        String password = "password";

        String sql = "INSERT INTO employees (name, department) VALUES (?, ?)";

        try (Connection conn = DriverManager.getConnection(url, user, password)) {
            // 1. Disable auto-commit for better performance and transaction control
            conn.setAutoCommit(false);

            try (PreparedStatement pstmt = conn.prepareStatement(sql)) {

                // Add first record to batch
                pstmt.setString(1, "Alice");
                pstmt.setString(2, "Engineering");
                pstmt.addBatch();

                // Add second record to batch
                pstmt.setString(1, "Bob");
                pstmt.setString(2, "Marketing");
                pstmt.addBatch();

                // 2. Execute the batch
                int[] results = pstmt.executeBatch();

                // 3. Commit the transaction
                conn.commit();
                System.out.println("Batch executed. Rows affected: " + results.length);

            } catch (SQLException e) {
                // Rollback in case of error
                conn.rollback();
                e.printStackTrace();
            }
        } catch (SQLException e) {
            e.printStackTrace();
        }
    }
}

Key Considerations

  1. setAutoCommit(false): By default, JDBC commits every statement individually. Turning this off allows the entire batch to be treated as a single transaction, which significantly boosts performance.
  2. addBatch(): Adds the current set of parameters to the internal list of commands.
  3. executeBatch(): Sends all the gathered commands to the database. It returns an int[] where each element represents the update count for the corresponding command in the batch.
  4. Batch Size: For very large datasets (e.g., thousands of rows), don’t add everything to a single batch. Instead, execute the batch every 500–1000 rows to avoid memory issues:
    if (count % 1000 == 0) {
        pstmt.executeBatch();
        conn.commit(); // Optional: commit periodically
    }
    

Using Statement

While possible, using Statement.addBatch(String sql) is generally discouraged for inserts involving variables because it is vulnerable to SQL injection and harder for the database to optimize. Use PreparedStatement whenever possible.

How do I delete rows with JDBC?

To delete rows from a database using JDBC, you use the executeUpdate() method. This method is used for SQL statements that modify data (like DELETE, INSERT, or UPDATE) and returns an integer representing the number of rows affected.

While you can use a simple Statement, it is highly recommended to use a PreparedStatement to prevent SQL injection and handle parameters safely.

Example: Deleting a Row with PreparedStatement

Here is a typical implementation using the try-with-resources pattern to ensure database connections are closed properly:

package org.kodejava.jdbc;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;

public class DeleteRowExample {
    public static void main(String[] args) {
        String url = "jdbc:mysql://localhost:3306/your_database";
        String user = "username";
        String password = "password";

        // SQL query with a placeholder (?) for the ID
        String sql = "DELETE FROM users WHERE id = ?";

        try (Connection conn = DriverManager.getConnection(url, user, password);
             PreparedStatement pstmt = conn.prepareStatement(sql)) {

            // Set the value for the placeholder
            int idToDelete = 101;
            pstmt.setInt(1, idToDelete);

            // Execute the delete operation
            int rowsDeleted = pstmt.executeUpdate();

            if (rowsDeleted > 0) {
                System.out.println("Successfuly deleted " + rowsDeleted + " row(s).");
            } else {
                System.out.println("No record found with the specified ID.");
            }

        } catch (SQLException e) {
            e.printStackTrace();
        }
    }
}

Key Steps:

  1. Prepare the SQL: Use the DELETE syntax. Use ? as placeholders for dynamic values.
  2. Create a PreparedStatement: Call connection.prepareStatement(sql).
  3. Bind Parameters: Use setter methods like setInt(), setString(), or setLong() to provide values for the placeholders (indices start at 1).
  4. Execute Update: Call executeUpdate(). It returns the count of deleted rows.
  5. Handle Exceptions: Wrap the code in a try-catch block to handle SQLException.

Which method should I use?

  • executeUpdate(): Use this for DELETE statements. It returns the number of rows removed.
  • Statement: Use only for static SQL with no user input.
  • PreparedStatement: Always preferred for security and performance when using variables in your WHERE clause.

How do I insert rows with JDBC?

To insert rows into a database using JDBC, you typically use the executeUpdate(String sql) method of a Statement or PreparedStatement object.

Here are the two primary ways to do it:

1. Using PreparedStatement (Recommended)

This is the standard approach because it prevents SQL Injection and is more efficient for repeated inserts.

package org.kodejava.jdbc;

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;

public class InsertRecordExample {
    public static void main(String[] args) {
        String url = "jdbc:mysql://localhost:3306/kodejava";
        String user = "root";
        String password = "password";

        String sql = "INSERT INTO users (username, email) VALUES (?, ?)";

        try (Connection conn = DriverManager.getConnection(url, user, password);
             PreparedStatement pstmt = conn.prepareStatement(sql)) {

            // Set the values for the placeholders (?)
            pstmt.setString(1, "john_doe");
            pstmt.setString(2, "[email protected]");

            // executeUpdate returns the number of rows affected
            int rowsInserted = pstmt.executeUpdate();
            if (rowsInserted > 0) {
                System.out.println("A new user was inserted successfully!");
            }

        } catch (SQLException e) {
            e.printStackTrace();
        }
    }
}

2. Using Statement

Use this only for simple, static SQL queries where no user input is involved.

// ... existing code ...
    try (Connection conn = DriverManager.getConnection(url, user, password);
         Statement stmt = conn.createStatement()) {

        String sql = "INSERT INTO users (username, email) VALUES ('jane_doe', '[email protected]')";
        int rows = stmt.executeUpdate(sql);

        System.out.println("Rows affected: " + rows);
    } catch (SQLException e) {
        e.printStackTrace();
    }
// ... existing code ...

Key Takeaways:

  • executeUpdate(): Unlike executeQuery() (which returns a ResultSet), executeUpdate() returns an int representing how many rows were added, changed, or deleted.
  • Try-with-resources: Always wrap your Connection, Statement, or PreparedStatement in a try-with-resources block to ensure they are closed automatically, even if an error occurs.
  • Placeholders: In a PreparedStatement, indices for ? parameters start at 1.