Android JetPack Compose + Paging 3

About pagination

Paginated APIs, often encountered when dealing with servers and databases, break down large datasets into manageable chunks or pages. This methodology not only optimizes resource usage but also enhances the overall user experience by delivering content progressively. However, integrating and managing paginated data in a mobile app can be a challenge, from maintaining loading states to ensuring a seamless transition between pages and supporting content filtering.

In this guide I will show you how to work with paginated APIs in Android with Jetpack Compose and Paging 3 libraries.

Basics, architecture overview

The key components of the Paging 3 architecture include

PagingSource

• The foundational element responsible for loading data in chunks.
• Developers implement a custom PagingSource, defining how to retrieve data from a particular source or use implementation provided by a library supporting Paging 3. (Room can generate a paging source for your data query)

Remote Mediator

• An integral part of the Paging 3 architecture, RemoteMediator manages the coordination between remote data sources, typically backed by a network service, and the local database.
• Responsible for loading pages of data from the network and storing them in the local database, ensuring efficient and reliable pagination.

Paging Data

• Represents the paginated data stream emitted by the PagingSource.
• A Flow of PagingData is observed in the UI layer, enabling dynamic updates as new data is loaded or existing data is invalidated.

Pager

• Coordinates the interaction between the PagingSource and the UI layer.
• Configures the pagination parameters, such as page size, prefetch distance, and initial load size, providing fine-grained control over the loading behavior.

Data layer

We start with the data layer and define a Retrofit API for fetching movies from network service.

interface MoviesApi {
  @GET("/3/discover/movie?language=en-US&sort_by=popularity.desc")
  suspend fun discover(
    @Query("api_key") api_key: String,
    @Query("page") page: Int,
    @Query("with_genres") genres: String,
  ): Response<MoviesNetworkResponse>
}

Next, we will leverage the Room library, which is compatible with Paging 3, to create a local paging source.

@Dao
interface MoviesDao {
  @Query("SELECT * FROM movies ORDER BY id ASC")
  fun getMovies(): DataSource.Factory<Int, MovieDbEntity>
}

It's important to highlight that we utilize DataSource.Factory as the return type instead of PagingSource. This choice is made specifically because DataSource.Factory offers the map() method, enabling us to map MovieDbEntity instances into domain layer models.

Now we need to put together local and remote data sources in Remote mediator.

1. Determine which page to load depending on the loadType and next page value. Next page number (getRemoteKey().nextPage) is stored in the database after each successful network request.

val page = when (loadType) {
        LoadType.REFRESH -> 1
        LoadType.PREPEND -> null
        LoadType.APPEND -> repository.getRemoteKey()?.nextPage
      } ?: return Success(endOfPaginationReached = true)

For bidirectional pagination support, provide the preceding page number for the PREPEND load type. The REFRESH load type is employed when content is refreshed, initiating loading from the initial page.

2. Request data from network

Call our network data source method to fetch a page of data from the server.

val movies = moviesDataSource.discover(page)

3. Insert data into the database or refresh database if loadType is refresh

The RemoteMediator code will appear as follows:

class MoviesRemoteMediator(
  private val repository: MoviesRepository,
  private val moviesDataSource: NetworkMoviesDataSource,
) : RemoteMediator<Int, Movie>() {

  override suspend fun load(loadType: LoadType, state: PagingState<Int, Movie>): MediatorResult {

    return try {

      val page = when (loadType) {
        LoadType.REFRESH -> 1
        LoadType.PREPEND -> null
        LoadType.APPEND -> repository.getRemoteKey()?.nextPage
      } ?: return Success(endOfPaginationReached = true)

      val movies = moviesDataSource.getMovies(page)

      val nextPage = if (movies.isNotEmpty()) page + 1 else null

      repository.insertMovies(movies, nextPage, loadType == LoadType.REFRESH)

      Success(endOfPaginationReached = movies.isEmpty())

    } catch (e: Exception) {
      MediatorResult.Error(e)
    }
  }
}

RemoteMediator uses the repository to store movies list and next page number in the database. Repository also takes care of clearing the database table when data is refreshed:

class MoviesRepositoryImpl(
  private val database: MoviesDatabase
): MoviesRepository {

  override fun getMovies() = database.movies().getMovies().map { it.toDomain() }

  override suspend fun insertMovies(movies: List<Movie>, nextPage: Int?, clear: Boolean){

    database.withTransaction {
      if (clear) {
        database.movies().clear()
      }

      database.movies().putMovies(movies.map { it.toDbEntity() })

      if (nextPage != null) {
        database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, nextPage))
      }
    }
  }
}

UI layer

The Pager object serves as a bridge between remote and local data sources. It is managing the page loading, prefetching, and handling the transitions between various data sources to provide a seamless and responsive flow of paginated content within your application. You can configure the page loading behavior with the following parameters:

• pageSize parameter defines the number of items loaded in each page of data.
• initialLoadSize is the number of items loaded initially when the paging library is first initialized.
• prefetchDistance parameter determines the distance from the edge of the loaded content at which the next page should start loading.

class MoviesScreenViewModel(
  moviesSource: MoviesRemoteMediator,
  repository: MoviesRepository,
  private val movieDetails: MovieDetailsApi,
) : ViewModel() {
   val moviesFlow = Pager(
      config = PagingConfig(
        pageSize = PAGE_SIZE,
        prefetchDistance = PREFETCH_DISTANCE,
        initialLoadSize = INITIAL_LOAD,
      ),
      remoteMediator = moviesSource,
      pagingSourceFactory = repository.getMovies().asPagingSourceFactory()
    ).flow.cachedIn(viewModelScope)
}

Consuming paging data in Jetpack Compose UI

Now, let's explore how to consume the paginated data in your Compose UI.

Use the collectAsLazyPagingItems extension function to collect the PagingData and observe changes.

val movies = viewModel.moviesFlow.collectAsLazyPagingItems()

Use the LazyColumn or LazyVerticalGrid to efficiently display paginated data in your Compose UI.

LazyVerticalGrid(
      modifier = Modifier
        .fillMaxSize()
        .padding(16.dp),
      columns = GridCells.Fixed(3),
      horizontalArrangement = Arrangement.spacedBy(16.dp),
      verticalArrangement = Arrangement.spacedBy(16.dp),
    ) {

      items(
        count = movies.itemCount,
        key = movies.itemKey { it.id },
      ) { index ->

        movies[index]?.let { movie ->
          MovieCardSmall(
            movie = movie,
            onClick = { onClick(movie.id) },
          )
        }
      }
    }

Note the usage of the key property for items. The key property uniquely identifies each item, allowing Compose to efficiently update and recycle composables as data changes.

Paging state handling

LazyPagingItems offers insights into the ongoing data loading process, allowing you to effectively manage errors during pagination. By examining the loadState.append property, you can determine the loading state when appending new items and present a user-friendly indication of the current status.

For instance, the following code snippet demonstrates how to react to different loading states during item appending:

when (movies.loadState.append) {
  is LoadState.Loading -> { ProgressFooter() }
  is LoadState.Error -> { ErrorFooter(onRetryClick = { movies.retry() } }
}

Similarly, you can assess the loading status when the list data is refreshed by checking the loadState.refresh property. Extension functions isError() and isLoading() provide convenient checks for error and loading states, respectively.

fun LazyPagingItems<Movie>.isError(): Boolean = 
    loadState.refresh is LoadState.Error && itemSnapshotList.isEmpty()

fun LazyPagingItems<Movie>.isLoading(): Boolean = 
    loadState.refresh is LoadState.Loading && itemSnapshotList.isEmpty()

Incorporate these checks into your UI logic to display appropriate screens based on the loading or error states. The following code snippet demonstrates how to handle loading and error states along with providing a retry option:

when {
   movies.isLoading() -> LoadingScreen()
   movies.isError() -> ErrorScreen { movies.retry() }
   else -> MoviesGrid(movies = movies, onClick = viewModel::onMovieClick)
}

Additionally, LazyPagingItems offers a convenient retry() method, allowing you to retry the last failed data loading attempt with ease.

Adding filters

Now lets add filters to our movie list to see paginated list of movies filtered by genre.

We start with requesting genres from the server and saving them in our database. We will keep genre data and selection in the database so that we can observe it from our domain and UI layers.

interface MoviesApi {
  @GET("/3/genre/movie/list?language=en-US")
  suspend fun genres(@Query("api_key") api_key: String): Response<GenresNetworkResponse>
}

@Dao
interface GenresDao {

  @Insert(onConflict = OnConflictStrategy.REPLACE)
  suspend fun putGenres(genres: List<GenreDbEntity>)

  @Upsert(entity = GenreDbEntity::class)
  suspend fun updateGenres(genres: List<GenreDbUpdateEntity>)

  @Query("SELECT * FROM genres")
  suspend fun getGenres(): List<GenreDbEntity>

  @Query("SELECT * FROM genres")
  fun observeGenres(): Flow<List<GenreDbEntity>>
}

Now we can add with_genres parameter to our network request:

  @GET("/3/discover/movie?language=en-US&sort_by=popularity.desc")
  suspend fun discover(
    @Query("page") page: Int,
    @Query("with_genres") genres: String,
    @Query("api_key") api_key: String,
  ): Response<MoviesNetworkResponse>

Remote mediator will get selected genres from the repository

  ...
  val genres = repository.getGenreSelection()
  val movies = moviesDataSource.getMovies(page, genres)
  ...

Every time user selects a new filter we stave it in the database and restart pagination by clearing movies in the database and resetting the page to 1.

class MoviesScreenViewModel {

  fun onGenreClick(genreItem: GenreItem) = viewModelScope.launch {
    genresProvider.setSelection(genreItem.id, !genreItem.isSelected)
    repository.clearMovies()
  }
  
}

class MoviesRepositoryImpl {

  override suspend fun clearMovies() = withContext(Dispatchers.IO) {
    database.withTransaction {
      database.movies().clear()
      database.remoteKey().insertKey(RemoteKey(MoviesDatabase.MOVIES_REMOTE_KEY, 1))
    }
  }

}

Links

Source code: https://github.com/auto1-oss/android-compose-paging

Official documentation: https://developer.android.com/topic/libraries/architecture/paging/v3-overview