Filtering and Searching In API

Filtering and searching are essential features in REST APIs, especially when dealing with large datasets. They allow clients to retrieve only the relevant data, improving performance and usability.

We can do effective filtering, searching and sorting in the following ways -

1. Using Query Parameters

Query parameters in a REST API are key-value pairs appended to the URL after a ?. For filtering and searching:

• Standardized Syntax: Use clear and intuitive names for query parameters:

  • field=value: Simple equality filter (e.g., /users?role=admin).

  • field[operator]=value: For advanced filtering using operators ([gt], [lt], [gte], [lte], etc.):

  • Combine filters with logical operators like AND or OR.

• Consistent Format: Ensure naming conventions are consistent:

  • Use camelCase (userName, minPrice) or snake_case (user_name, min_price) based on your API style guide.

• Reserved Characters: Encode special characters properly to avoid parsing issues.

The simplest way to implement filtering is by using query parameters in the URL. Spring Boot makes it easy to handle query parameters with @RequestParam

2: Using Specifications (Dynamic Queries)

For more complex filtering, you can use Specifications with JPA Criteria API. This approach is flexible and allows you to build dynamic queries.

public class SearchCriteria {
    private String keyword;
    private String fieldName;
    private String operator; // e.g., EQUAL, GREATER_THAN
    private Object value;

    // Getters and Setters
}



public class GenericSpecification<T> implements Specification<T> {
    private SearchCriteria criteria;

    @Override
    public Predicate toPredicate(Root<T> root, CriteriaQuery<?> query, CriteriaBuilder builder) {
        if (criteria.getOperator().equalsIgnoreCase("EQUAL")) {
            return builder.equal(root.get(criteria.getFieldName()), criteria.getValue());
        }
        if (criteria.getOperator().equalsIgnoreCase("GREATER_THAN")) {
            return builder.greaterThan(root.get(criteria.getFieldName()), criteria.getValue().toString());
        }
        // Add other conditions here
        return null;
    }
}

For more details info see Specification

3. Combining Filtering and Searching

We can combine filtering and searching in a single endpoint to provide a powerful querying mechanism.

Ex. /products?category=electronics&price[gt]=100&brand=Samsung

Using spring data specification we can combine multiple specifications -

Specification spec = Specification.where(spec1).and(spec2).or(spec3);

4. Pagination and Sorting

We can utilize effective pagination and sorting based on our need to display properly ordered data in chunks.

• Query Parameter for Sorting: Use a sort query parameter:

  /products?sort=price,desc

  • field: Field to sort by (e.g., price).

  • direction: Sorting direction (asc or desc).

• Multiple Sort Fields: Support sorting by multiple fields:

  /products?sort=price,asc&sort=name,desc
  

• Spring Data PagingAndSortingRepository: Use the Pageable interface:

  @GetMapping("/products")
  public Page<Product> getProducts(Pageable pageable) {
      return productRepository.findAll(pageable);
  }

  Example Pageable request:

  bashCopyEdit/products?page=0&size=10&sort=price,desc

Example -

1. Simple -

   Filter record based on exact match

   1. GET /api/users?role=admin

2. Rangle Filters

   Support range queries using operators like gt, lt, gte, lte.

   1. GET /api/products?price[gt]=100&price[lte]=500

3. Multiple Filters

   1. GET /api/orders?status=delivered&date[gte]=2023-01-01&date[lte]=2023-12-31

4. Full Text Search

   1. GET /api/books?search=Spring Boot

5. Logical Operators

   1. GET /api/movies?filter=(genre=action AND rating[gte]=8) OR (director=Nolan)

6. Sorting

   1. GET /api/products?sort=price,asc

   2. GET /api/products?sort=price,desc&sort=name,asc

7. Pagination

   1. GET /api/users?page=1&size=10

8. Filtering on Nested or Related Fields

   1. GET /api/orders?customer.name=John&product.category=electronics

   Retrieves orders where the customer’s name is John and the product belongs to the electronics category.

9. Case-Insensitive Search

   1. GET /api/users?email=[e-mail]

   Matches [e-mail], [e-mail], etc.

10. Include/Exclude Fields

    1. GET /api/users?fields=id,name,email

    Includes only the id, name, and email fields in the response.

11. Reserved Characters Handling

    Ensure special characters in query parameters are encoded.

    1. /products?name=Spring%20Boot&description=advanced%2Fbeginner

    Retrieves products where the name is Spring Boot and the description contains advanced/beginner.

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Sort;
import org.springframework.web.bind.annotation.*;
import java.util.List;

@RestController
@RequestMapping("/api")
public class ProductController {

    @Autowired
    private ProductService productService;

    // 1. Simple Equality Filters
    @GetMapping("/users")
    public List<User> getUsersByRole(@RequestParam(required = false) String role) {
        return productService.getUsersByRole(role);
    }

    // 2. Range Filters
    @GetMapping("/products")
    public List<Product> getProductsByPriceRange(
        @RequestParam(required = false) Double priceGt,
        @RequestParam(required = false) Double priceLte
    ) {
        return productService.getProductsByPriceRange(priceGt, priceLte);
    }

    // 3. Full-Text Search
    @GetMapping("/books")
    public List<Book> searchBooks(@RequestParam(required = false) String search) {
        return productService.searchBooks(search);
    }

    // 4. Logical Operators
    @GetMapping("/movies")
    public List<Movie> getMoviesWithFilters(
        @RequestParam(required = false) String filter
    ) {
        return productService.getMoviesWithFilters(filter);
    }

    // 5. Sorting
    @GetMapping("/products/sorted")
    public List<Product> getSortedProducts(
        @RequestParam(required = false, defaultValue = "id,asc") String[] sort
    ) {
     List<Sort.Order> orders = new ArrayList<>();
    
    if (sort[0].contains(",")) {
        // sort=[field,direction] format
        for (String sortOrder : sort) {
            String[] _sort = sortOrder.split(",");
            orders.add(new Sort.Order(
                getSortDirection(_sort[1]), 
                _sort[0]));
        }
    } else {
        // sort=[field] format
        orders.add(new Sort.Order(
            getSortDirection("asc"), 
            sort[0]));
    }
    
    Pageable pageable = PageRequest.of(page, size, Sort.by(orders));
    
    return ResponseEntity.ok(
        productService.findProducts(name, minPrice, maxPrice, 
                                  categories, pageable));
    }

    // 6. Pagination
    @GetMapping("/users/paginated")
    public Page<User> getUsersWithPagination(Pageable pageable) {
        return productService.getUsersWithPagination(pageable);
    }

    // 7. Nested/Related Field Filtering
    @GetMapping("/orders")
    public List<Order> getOrdersByCustomerAndCategory(
        @RequestParam(required = false) String customerName,
        @RequestParam(required = false) String productCategory
    ) {
        return productService.getOrdersByCustomerAndCategory(customerName, productCategory);
    }

    // 8. Case-Insensitive Search
    @GetMapping("/users/search")
    public List<User> getUsersByEmail(@RequestParam String email) {
        return productService.getUsersByEmail(email);
    }

    // 9. Include/Exclude Fields
    @GetMapping("/users/fields")
    public List<User> getUsersWithSelectedFields(@RequestParam List<String> fields) {
        return productService.getUsersWithSelectedFields(fields);
    }
}

Summary

• Filtering: Use query parameters or Specifications to narrow down results.

• Searching: Implement simple or advanced search using query parameters or Specifications.

• Combining Filtering and Searching: Provide a unified endpoint for flexible querying.

• Pagination and Sorting: Improve performance and usability by paginating and sorting results.