DataSources and Repository Patterns: Building a Robust Data Layer

In modern application development, managing data access efficiently is crucial for creating maintainable and scalable software. Two architectural patterns that significantly improve data management are the DataSource and Repository patterns. This blog post explores what these patterns are, how they work together, and how to implement them effectively with practical examples.

What Are DataSources?

DataSources are components responsible for handling data operations with a specific data origin. They abstract the details of how data is fetched, stored, or manipulated from a particular source, such as:

Remote DataSources: Handle API calls, network requests, and cloud storage
Local DataSources: Manage database operations, file system access, or in-memory caching
Third-party Service DataSources: Interface with external services like payment processors or analytics platforms

Some key characteristics of DataSources:

Single Responsibility: Each DataSource focuses on one data origin
Implementation Details: They contain the technical details of data access
Low-Level Operations: They perform primitive operations like CRUD (Create, Read, Update, Delete)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// Example of a Remote DataSource
class UserRemoteDataSource(private val apiService: ApiService) {
    suspend fun getUser(userId: String): UserDto {
        return apiService.getUser(userId)
    }

    suspend fun updateUser(userDto: UserDto): UserDto {
        return apiService.updateUser(userDto)
    }

    suspend fun deleteUser(userId: String): Boolean {
        return apiService.deleteUser(userId)
    }
}

// Example of a Local DataSource
class UserLocalDataSource(private val userDao: UserDao) {
    suspend fun getUser(userId: String): UserEntity? {
        return userDao.getUserById(userId)
    }

    suspend fun saveUser(userEntity: UserEntity) {
        userDao.insertOrUpdate(userEntity)
    }

    suspend fun deleteUser(userId: String) {
        userDao.deleteUserById(userId)
    }

    suspend fun getAllUsers(): List<UserEntity> {
        return userDao.getAllUsers()
    }
}

What Is the Repository Pattern?

The Repository pattern acts as an abstraction layer between your business logic and data sources. It provides a clean API for data access and hides the complexity of fetching, combining, and managing data from multiple sources.

Key characteristics of Repositories:

Abstraction: They hide the details of data operations
Coordination: They orchestrate data access across multiple DataSources
Domain Focus: They work with domain models rather than data models
Business Rules: They can enforce business rules related to data access

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
// Repository interface
interface UserRepository {
    suspend fun getUser(userId: String): User
    suspend fun updateUser(user: User): User
    suspend fun deleteUser(userId: String): Boolean
    suspend fun syncUserData(userId: String): User
}

// Repository implementation
class UserRepositoryImpl(
    private val remoteDataSource: UserRemoteDataSource,
    private val localDataSource: UserLocalDataSource,
    private val userMapper: UserMapper
) : UserRepository {

    override suspend fun getUser(userId: String): User {
        // Try to get from local cache first
        val localUser = localDataSource.getUser(userId)

        // If found locally, return it
        if (localUser != null) {
            return userMapper.mapEntityToDomain(localUser)
        }

        // Otherwise fetch from remote and cache it
        val remoteUser = remoteDataSource.getUser(userId)
        val userEntity = userMapper.mapDtoToEntity(remoteUser)
        localDataSource.saveUser(userEntity)

        return userMapper.mapDtoToDomain(remoteUser)
    }

    override suspend fun updateUser(user: User): User {
        // Update remote first
        val userDto = userMapper.mapDomainToDto(user)
        val updatedDto = remoteDataSource.updateUser(userDto)

        // Then update local cache
        val userEntity = userMapper.mapDtoToEntity(updatedDto)
        localDataSource.saveUser(userEntity)

        return userMapper.mapDtoToDomain(updatedDto)
    }

    override suspend fun deleteUser(userId: String): Boolean {
        // Delete from remote
        val success = remoteDataSource.deleteUser(userId)

        // If successful, also delete from local
        if (success) {
            localDataSource.deleteUser(userId)
        }

        return success
    }

    override suspend fun syncUserData(userId: String): User {
        // Force a refresh from remote
        val remoteUser = remoteDataSource.getUser(userId)
        val userEntity = userMapper.mapDtoToEntity(remoteUser)
        localDataSource.saveUser(userEntity)

        return userMapper.mapDtoToDomain(remoteUser)
    }
}

How DataSources and Repositories Work Together

The relationship between DataSources and Repositories creates a powerful data management system:

DataSources handle the “how” of data access (implementation details)
Repositories handle the “what” of data access (business requirements)

This separation provides several benefits:

1. Clean Separation of Concerns

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// Without proper separation - Everything mixed together
class UserViewModel(private val apiService: ApiService, private val userDao: UserDao) {
    fun getUser(userId: String) {
        viewModelScope.launch {
            try {
                // Try local first
                var user = userDao.getUserById(userId)

                if (user == null) {
                    // Fetch from network
                    val networkUser = apiService.getUser(userId)
                    // Convert DTO to entity
                    user = convertDtoToEntity(networkUser)
                    // Save to database
                    userDao.insertOrUpdate(user)
                }

                // Convert to UI model
                val uiModel = convertEntityToUiModel(user)
                _userState.value = UserState.Success(uiModel)
            } catch (e: Exception) {
                _userState.value = UserState.Error("Failed to load user")
            }
        }
    }
}

// With proper separation
class UserViewModel(private val userRepository: UserRepository) {
    fun getUser(userId: String) {
        viewModelScope.launch {
            try {
                val user = userRepository.getUser(userId)
                _userState.value = UserState.Success(user)
            } catch (e: Exception) {
                _userState.value = UserState.Error("Failed to load user")
            }
        }
    }
}

2. Improved Testability

With this pattern, you can easily mock repositories for testing your business logic, and mock data sources for testing your repositories:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
// Testing a ViewModel with a mocked Repository
class UserViewModelTest {
    @Test
    fun `getUser returns success state when repository returns user`() = runTest {
        // Arrange
        val mockRepository = mockk<UserRepository>()
        val user = User("1", "John Doe", "john@example.com")
        coEvery { mockRepository.getUser("1") } returns user

        val viewModel = UserViewModel(mockRepository)

        // Act
        viewModel.getUser("1")

        // Assert
        assertEquals(UserState.Success(user), viewModel.userState.value)
    }
}

// Testing a Repository with mocked DataSources
class UserRepositoryTest {
    @Test
    fun `getUser returns data from local source when available`() = runTest {
        // Arrange
        val mockLocalDataSource = mockk<UserLocalDataSource>()
        val mockRemoteDataSource = mockk<UserRemoteDataSource>()
        val mockMapper = mockk<UserMapper>()

        val userEntity = UserEntity("1", "John", "Doe", "john@example.com")
        val user = User("1", "John Doe", "john@example.com")

        coEvery { mockLocalDataSource.getUser("1") } returns userEntity
        coEvery { mockMapper.mapEntityToDomain(userEntity) } returns user

        val repository = UserRepositoryImpl(mockRemoteDataSource, mockLocalDataSource, mockMapper)

        // Act
        val result = repository.getUser("1")

        // Assert
        assertEquals(user, result)
        coVerify(exactly = 1) { mockLocalDataSource.getUser("1") }
        coVerify(exactly = 0) { mockRemoteDataSource.getUser(any()) }
    }
}

3. Flexibility and Adaptability

This pattern makes it easy to change data sources without affecting the rest of your application:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Switching from REST API to GraphQL only requires changing the DataSource implementation
class UserRemoteDataSourceRest(private val restApiService: RestApiService) : UserRemoteDataSource {
    override suspend fun getUser(userId: String): UserDto {
        return restApiService.getUser(userId)
    }
    // Other implementations
}

class UserRemoteDataSourceGraphQL(private val graphQLClient: GraphQLClient) : UserRemoteDataSource {
    override suspend fun getUser(userId: String): UserDto {
        val response = graphQLClient.execute(UserQueries.GET_USER, mapOf("id" to userId))
        return response.data.user
    }
    // Other implementations
}

// The Repository doesn't need to change
class UserRepositoryImpl(
    private val remoteDataSource: UserRemoteDataSource, // Interface, not concrete implementation
    private val localDataSource: UserLocalDataSource,
    private val userMapper: UserMapper
) : UserRepository {
    // Implementation remains the same
}

Best Practices for DataSources and Repositories

Single Responsibility: Keep DataSources focused on a single data origin
Interface-Based Design: Define repositories as interfaces for better testability
Error Handling: Implement robust error handling in repositories
Caching Strategy: Develop a clear caching strategy (time-based, event-based, etc.)
Offline Support: Use repositories to provide offline functionality
Dependency Injection: Use DI to provide DataSources to Repositories
Consistent Naming: Use consistent naming conventions across your data layer
Pagination: Implement pagination for large data sets
Reactive Patterns: Consider using Flow or LiveData for observing data changes
Transaction Support: Implement transactions for operations that modify multiple entities

Conclusion

The DataSource and Repository patterns provide a powerful approach to building a robust data layer in your applications. By separating the concerns of data access and business logic, these patterns help you create code that is more maintainable, testable, and adaptable to change.

Implementing these patterns requires some initial investment in architecture, but the benefits quickly become apparent as your application grows in complexity. With a well-designed data layer, you can more easily add features, switch data sources, implement caching strategies, and provide offline support.

Whether you’re building a small app or a large enterprise system, the principles of the DataSource and Repository patterns can help you create a solid foundation for your data management needs, leading to a more successful and sustainable project.