A microservices architecture is a development method for designing applications as modular services that seamlessly adapt to a highly scalable and dynamic environment. Microservices help solve complex issues such as speed and scalability, while also supporting continuous testing and delivery. This Zone will take you through breaking down the monolith step by step and designing a microservices architecture from scratch. Stay up to date on the industry's changes with topics such as container deployment, architectural design patterns, event-driven architecture, service meshes, and more.
Participate in DZone Research Surveys: You Can Shape Trend Reports! (+ Enter the Raffles)
Effective Communication Strategies Between Microservices: Techniques and Real-World Examples
In this article, we will implement a microservice using Clean Architecture and CQRS. The tech stack uses Kotlin, Spring WebFlux with coroutines, PostgreSQL and MongoDB, Kafka as a message broker, and the Arrow-kt functional library, which, as the documentation says, brings idiomatic functional programming to Kotlin. Clean Architecture Clean Architecture is one of the more popular software design approaches. It follows the principles of Dependency Inversion, Single Responsibility, and Separation of Concerns. It consists of concentric circles representing different layers, with the innermost layer being the most abstract and the outermost layer representing the user interface and infrastructure. By separating the concerns of the various components and enforcing the dependency rule, it becomes much easier to understand and modify the code. Depending on abstractions allows you to design your business logic flexibly without having to know the implementation details. The Domain Layer and the Application Layer are the core of the Clean Architecture. These two layers together form the application core, encapsulating the most important business rules of the system. Clean Architecture is a domain-centric architectural approach that separates business logic from technical implementation details. CQRS CQRS stands for Command and Query Responsibility Segregation, a pattern that separates reads and writes into different models, using commands to update data, and queries to read data. Using CQRS, you should have a strict separation between the write model and the read model. Those two models should be processed by separate objects and not be conceptually linked together. Those objects are not physical storage structures but are, for example, command handlers and query handlers. They’re not related to where and how the data will be stored: they’re connected to the processing behavior. Command handlers are responsible for handling commands, mutating state, or doing other side effects. Query handlers are responsible for returning the result of the requested query. They give us: Scalability, which allows for independent scaling of read and write operations Performance: By separating read and write operations, you can optimize each for performance. Reads can be optimized for fast retrieval by using denormalized data structures, caching, and specialized read models tailored to specific query needs. Flexibility allows us to model the read and write sides of the application differently, which provides flexibility in designing the data structures and processing logic to best suit the requirements of each operation. This flexibility can lead to a more efficient and maintainable system, especially in complex domains where the read and write requirements differ significantly. One of the common misconceptions about CQRS is that the commands and queries should be run on separate databases. This isn’t necessarily true, only that the behaviors and responsibilities for both should be separated. This can be within the code, within the structure of the database, or different databases. Nothing in an inner circle can know anything about something in an outer circle. In particular, the name of something declared in an outer circle must not be mentioned by the code in the inner circle. That includes functions and classes, variables, or any other named software entity. In the real world, understanding Clean Architecture can vary from person to person. Since Clean Architecture emphasizes principles such as separation of concerns, dependency inversion, and abstraction layers, different developers may interpret and implement these principles differently based on their own experiences, knowledge, and project requirements. This article shows my personal view of one of the possible ways of implementation. Ultimately, the goal of Clean Architecture is to create software systems that are maintainable, scalable, and easy to understand. Layers Presentation Layer The Presentation Layer (named api here) is the most outside layer and the entry point to our system. The most important part of the presentation layer is the controllers, which define the API endpoints in our system presented to the outside world and are responsible for: Handling interaction with the outside world Presenting, displaying, or returning responses with the data Translating the outside requests data (map requests to application layer commands) Works with framework-specific configuration setup Works on top of the application layer Let's look at the full process of command requests in the microservice. First things first: it accepts REST HTTP requests; validates input; if it's secured, checks credentials, etc.; then maps the request to the DTO the command and calls the AccountCommandService handle method. For example, let's look at creating new account and deposit balance commands methods call flow: Kotlin @Tag(name = "Accounts", description = "Account domain REST endpoints") @RestController @RequestMapping(path = ["/api/v1/accounts"]) class AccountController( private val accountCommandService: AccountCommandService, private val accountQueryService: AccountQueryService ) { @Operation( method = "createAccount", operationId = "createAccount", description = "Create new Account", responses = [ ApiResponse( description = "Create new Account", responseCode = "201", content = [Content( mediaType = MediaType.APPLICATION_JSON_VALUE, schema = Schema(implementation = AccountId::class) )] ), ApiResponse( description = "bad request response", responseCode = "400", content = [Content(schema = Schema(implementation = ErrorHttpResponse::class))] )], ) @PostMapping suspend fun createAccount( @Valid @RequestBody request: CreateAccountRequest ): ResponseEntity<out Any> = eitherScope(ctx) { accountCommandService.handle(request.toCommand()).bind() }.fold( ifLeft = { mapErrorToResponse(it) }, ifRight = { ResponseEntity.status(HttpStatus.CREATED).body(it) } ) @Operation( method = "depositBalance", operationId = "depositBalance", description = "Deposit balance", responses = [ ApiResponse( description = "Deposit balance", responseCode = "200", content = [Content( mediaType = MediaType.APPLICATION_JSON_VALUE, schema = Schema(implementation = BaseResponse::class) )] ), ApiResponse( description = "bad request response", responseCode = "400", content = [Content(schema = Schema(implementation = ErrorHttpResponse::class))] )], ) @PutMapping(path = ["/{id}/deposit"]) suspend fun depositBalance( @PathVariable id: UUID, @Valid @RequestBody request: DepositBalanceRequest ): ResponseEntity<out Any> = eitherScope(ctx) { accountCommandService.handle(request.toCommand(AccountId(id))).bind() }.fold( ifLeft = { mapErrorToResponse(it) }, ifRight = { okResponse(it) } ) } Application and Domain Layers The Application Layer contains the use cases of the application. A use case represents a specific interaction or action that the system can perform. Each use case is implemented as a command or a query. It is part of the whole application core like a Domain Layer and is responsible for: Executing the application use cases (all the actions and commands allowed to be done with the system) Fetch domain objects Manipulating domain objects The Application Layer AccountCommandService has the business logic, which runs required business rules validations, then applies changes to the domain aggregate, persists domain objects in the database, produces the domain events, and persists them in the outbox table within one single transaction. The current application used some not-required small optimization for outbox publishing. After the command service commits the transaction, we publish the event, but we don't care if this publish fails, because the polling publisher realizes that the Spring scheduler will process it anyway. Arrow greatly improves developer experience because Kotlin doesn’t ship the Either type with the standard SDK. Either is an entity whose value can be of two different types, called left and right. By convention, the right is for the success case and the left is for the error one. It allows us to express the fact that a call might return a correct value or an error, and differentiate between the two of them. The left/right naming pattern is just a convention. Either is a great way to make the error handling in your code more explicit. Making the code more explicit reduces the amount of context that you need to keep in your head, which in turn makes the code easier to understand. Kotlin interface AccountCommandService { suspend fun handle(command: CreateAccountCommand): Either<AppError, AccountId> suspend fun handle(command: ChangeAccountStatusCommand): Either<AppError, Unit> suspend fun handle(command: ChangeContactInfoCommand): Either<AppError, Unit> suspend fun handle(command: DepositBalanceCommand): Either<AppError, Unit> suspend fun handle(command: WithdrawBalanceCommand): Either<AppError, Unit> suspend fun handle(command: UpdatePersonalInfoCommand): Either<AppError, Unit> } @Service class AccountCommandServiceImpl( private val accountRepository: AccountRepository, private val outboxRepository: OutboxRepository, private val tx: TransactionalOperator, private val eventPublisher: EventPublisher, private val serializer: Serializer, private val emailVerifierClient: EmailVerifierClient, private val paymentClient: PaymentClient ) : AccountCommandService { override suspend fun handle(command: CreateAccountCommand): Either<AppError, AccountId> = eitherScope(ctx) { emailVerifierClient.verifyEmail(command.contactInfo.email).bind() val (account, event) = tx.executeAndAwait { val account = accountRepository.save(command.toAccount()).bind() val event = outboxRepository.insert(account.toAccountCreatedOutboxEvent(serializer)).bind() account to event } publisherScope.launch { publishOutboxEvent(event) } account.accountId } override suspend fun handle(command: DepositBalanceCommand): Either<AppError, Unit> = eitherScope(ctx) { paymentClient.verifyPaymentTransaction(command.accountId.string(), command.transactionId).bind() val event = tx.executeAndAwait { val foundAccount = accountRepository.getById(command.accountId).bind() foundAccount.depositBalance(command.balance).bind() val account = accountRepository.update(foundAccount).bind() val event = account.toBalanceDepositedOutboxEvent(command.balance, serializer) outboxRepository.insert(event).bind() } publisherScope.launch { publishOutboxEvent(event) } } } The Domain Layer encapsulates the most important business rules of the system. It is the place where we have to start building core business rules. In the domain-centric architecture, we start developing from the domain. The responsibilities of the Domain Layer are as follows: Defining domain models Defining rules, domain, and business errors Executing the application business logic Enforcing the business rules Domain models have data and behavior and represent the domain. We have two approaches for designing: rich and anemic domain models. Anemic models allow external manipulation of our data, and it's usually antipattern because the domain object itself doesn't control its own data. Rich domain models contain both data and behavior. The richer the behavior, the richer the domain model. It exposes only a specific set of public methods, which allows manipulation of data only in the way the domain approves, encapsulates logic, and does validations. Rich domain model properties are read-only by default. Domain models can be always valid or not; it's better to prefer always-valid domain models. At any point in time when we're working with domain state, we know it's valid and don't need to write additional validations to check it. Always-valid domain models mean they are in a valid state all the time. One more important detail is Persistence Ignorance - modeling the domain without taking into account how domain objects will be persisted. Kotlin class Account( val accountId: AccountId = AccountId(), ) { var contactInfo: ContactInfo = ContactInfo() private set var personalInfo: PersonalInfo = PersonalInfo() private set var address: Address = Address() private set var balance: Balance = Balance() private set var status: AccountStatus = AccountStatus.FREE private set var version: Long = 0 private set var updatedAt: Instant? = null private set var createdAt: Instant? = null private set fun depositBalance(newBalance: Balance): Either<AppError, Account> = either { if (balance.balanceCurrency != newBalance.balanceCurrency) raise(InvalidBalanceCurrency("invalid currency: $newBalance")) if (newBalance.amount < 0) raise(InvalidBalanceAmount("invalid balance amount: $newBalance")) balance = balance.copy(amount = (balance.amount + newBalance.amount)) updatedAt = Instant.now() this@Account } fun withdrawBalance(newBalance: Balance): Either<AppError, Account> = either { if (balance.balanceCurrency != newBalance.balanceCurrency) raise(InvalidBalanceCurrency("invalid currency: $newBalance")) if (newBalance.amount < 0) raise(InvalidBalanceAmount("invalid balance amount: $newBalance")) val newAmount = (balance.amount - newBalance.amount) if ((newAmount) < 0) raise(InvalidBalanceError("invalid balance: $newBalance")) balance = balance.copy(amount = newAmount) updatedAt = Instant.now() this@Account } fun updateStatus(newStatus: AccountStatus): Either<AppError, Account> = either { status = newStatus updatedAt = Instant.now() this@Account } fun changeContactInfo(newContactInfo: ContactInfo): Either<AppError, Account> = either { contactInfo = newContactInfo updatedAt = Instant.now() this@Account } fun changeAddress(newAddress: Address): Either<AppError, Account> = either { address = newAddress updatedAt = Instant.now() this@Account } fun changePersonalInfo(newPersonalInfo: PersonalInfo): Either<AppError, Account> = either { personalInfo = newPersonalInfo updatedAt = Instant.now() this@Account } fun incVersion(amount: Long = 1): Either<AppError, Account> = either { if (amount < 1) raise(InvalidVersion("invalid version: $amount")) version += amount updatedAt = Instant.now() this@Account } fun withVersion(amount: Long = 1): Account { version = amount updatedAt = Instant.now() return this } fun decVersion(amount: Long = 1): Either<AppError, Account> = either { if (amount < 1) raise(InvalidVersion("invalid version: $amount")) version -= amount updatedAt = Instant.now() this@Account } fun withUpdatedAt(newValue: Instant): Account { updatedAt = newValue return this } } Infrastructure Layer Next is the Infrastructure Layer, which contains implementations for external-facing services and is responsible for: Interacting with the persistence solution Interacting with other services (HTTP or gRPC clients, message brokers, etc.) Actual implementations of the interfaces from the application layer Identity concerns At the Infrastructure Layer, we have implementations of the Application Layer interfaces. The main write database used PostgreSQL with r2dbc reactive driver, and DatabaseClient with raw SQL queries. If we want to use an ORM entity, we still pass domain objects through the other layer interfaces anyway, and then inside the repository implementation, code map to the ORM entities. For this project, keep Spring annotations as is; but if we want cleaner implementation, it's possible to move them to another layer. In this example, the project SQL schema is simplified and not normalized. Kotlin interface AccountRepository { suspend fun getById(id: AccountId): Either<AppError, Account> suspend fun save(account: Account): Either<AppError, Account> suspend fun update(account: Account): Either<AppError, Account> } @Repository class AccountRepositoryImpl( private val dbClient: DatabaseClient ) : AccountRepository { override suspend fun save(account: Account): Either<AppError, Account> = eitherScope<AppError, Account>(ctx) { dbClient.sql(INSERT_ACCOUNT_QUERY.trimMargin()) .bindValues(account.withVersion(FIRST_VERSION).toPostgresEntityMap()) .fetch() .rowsUpdated() .awaitSingle() account } override suspend fun update(account: Account): Either<AppError, Account> = eitherScope(ctx) { dbClient.sql(OPTIMISTIC_UPDATE_QUERY.trimMargin()) .bindValues(account.withUpdatedAt(Instant.now()).toPostgresEntityMap(withOptimisticLock = true)) .fetch() .rowsUpdated() .awaitSingle() account.incVersion().bind() } override suspend fun getById(id: AccountId): Either<AppError, Account> = eitherScope(ctx) { dbClient.sql(GET_ACCOUNT_BY_ID_QUERY.trimMargin()) .bind(ID_FIELD, id.id) .map { row, _ -> row.toAccount() } .awaitSingleOrNull() ?: raise(AccountNotFoundError("account for id: $id not found")) } } Below is an important detail about outbox repository realization: To be able to handle the case of multiple pod instances processing in parallel outbox table, of course, we have idempotent consumers. However, as we can, we have to avoid processing the same table events more than one time. To prevent multiple instances from selecting and publishing the same events, we use FOR UPDATE SKIP LOCKED. This combination does the next thing: When one instance tries to select a batch of outbox events, if some other instance already selected these records, first, one will skip locked records and select the next available and not locked, and so on. But again, it's only my personal preferred way of implementation. The use of only polling publishers is usually the default one. As a possible alternative, use Debezium (for example), but it's up to you. Kotlin interface OutboxRepository { suspend fun insert(event: OutboxEvent): Either<AppError, OutboxEvent> suspend fun deleteWithLock( event: OutboxEvent, callback: suspend (event: OutboxEvent) -> Either<AppError, Unit> ): Either<AppError, OutboxEvent> suspend fun deleteEventsWithLock( batchSize: Int, callback: suspend (event: OutboxEvent) -> Either<AppError, Unit> ): Either<AppError, Unit> } @Component class OutboxRepositoryImpl( private val dbClient: DatabaseClient, private val tx: TransactionalOperator ) : OutboxRepository { override suspend fun insert(event: OutboxEvent): Either<AppError, OutboxEvent> = eitherScope(ctx) { dbClient.sql(INSERT_OUTBOX_EVENT_QUERY.trimMargin()) .bindValues(event.toPostgresValuesMap()) .map { row, _ -> row.get(ROW_EVENT_ID, String::class.java) } .one() .awaitSingle() .let { event } } override suspend fun deleteWithLock( event: OutboxEvent, callback: suspend (event: OutboxEvent) -> Either<AppError, Unit> ): Either<AppError, OutboxEvent> = eitherScope { tx.executeAndAwait { dbClient.sql(GET_OUTBOX_EVENT_BY_ID_FOR_UPDATE_SKIP_LOCKED_QUERY.trimMargin()) .bindValues(mutableMapOf(EVENT_ID to event.eventId)) .map { row, _ -> row.get(ROW_EVENT_ID, String::class.java) } .one() .awaitSingleOrNull() callback(event).bind() deleteOutboxEvent(event).bind() event } } override suspend fun deleteEventsWithLock( batchSize: Int, callback: suspend (event: OutboxEvent) -> Either<AppError, Unit> ): Either<AppError, Unit> = eitherScope(ctx) { tx.executeAndAwait { dbClient.sql(GET_OUTBOX_EVENTS_FOR_UPDATE_SKIP_LOCKED_QUERY.trimMargin()) .bind(LIMIT, batchSize) .map { row, _ -> row.toOutboxEvent() } .all() .asFlow() .onStart { log.info { "start publishing outbox events batch: $batchSize" } } .onEach { callback(it).bind() } .onEach { event -> deleteOutboxEvent(event).bind() } .onCompletion { log.info { "completed publishing outbox events batch: $batchSize" } } .collect() } } private suspend fun deleteOutboxEvent(event: OutboxEvent): Either<AppError, Long> = eitherScope(ctx) { dbClient.sql(DELETE_OUTBOX_EVENT_BY_ID_QUERY) .bindValues(mutableMapOf(EVENT_ID to event.eventId)) .fetch() .rowsUpdated() .awaitSingle() } } The polling publisher implementation is a scheduled process that does the same job for publishing and deleting events at the given interval, as typed earlier, and uses the same service method: Kotlin @Component @ConditionalOnProperty(prefix = "schedulers", value = ["outbox.enable"], havingValue = "true") class OutboxScheduler( private val outboxRepository: OutboxRepository, private val publisher: EventPublisher, ) { @Value("\${schedulers.outbox.batchSize}") private var batchSize: Int = 30 @Scheduled( initialDelayString = "\${schedulers.outbox.initialDelayMillis}", fixedRateString = "\${schedulers.outbox.fixedRate}" ) fun publishOutboxEvents() = runBlocking { eitherScope { outboxRepository.deleteEventsWithLock(batchSize) { publisher.publish(it) }.bind() }.fold( ifLeft = { err -> log.error { "error while publishing scheduler outbox events: $err" } }, ifRight = { log.info { "outbox scheduler published events" } } ) } } A domain event is something interesting from a business point of view that happened within the system; something that already occurred. We're capturing the fact something happened with the system. After events have been published from the outbox table to the broker, in this application, it consumes them from Kafka, and the consumers themselves call EventHandlerService methods, which builds a read model for our domain aggregates. The read model of a CQRS-based system provides materialized views of the data, typically as highly denormalized views. These views are tailored to the interfaces and display requirements of the application, which helps to maximize both display and query performance. For error handling and retry, messages prefer to use separate retry topics and listeners. Using the stream of events as the write store rather than the actual data at a point in time avoids update conflicts on a single aggregate and maximizes performance and scalability. The events can be used to asynchronously generate materialized views of the data that are used to populate the read store. As with any system where the write and read stores are separate, systems based on this pattern are only eventually consistent. There will be some delay between the event being generated and the data store being updated. Here is Kafka consumer implementation: Kotlin @Component class BalanceDepositedEventConsumer( private val eventProcessor: EventProcessor, private val kafkaTopics: KafkaTopics ) { @KafkaListener( groupId = "\${kafka.consumer-group-id:account_microservice_group_id}", topics = ["\${topics.accountBalanceDeposited.name}"], ) fun process(ack: Acknowledgment, record: ConsumerRecord<String, ByteArray>) = eventProcessor.process( ack = ack, consumerRecord = record, deserializationClazz = BalanceDepositedEvent::class.java, onError = eventProcessor.errorRetryHandler(kafkaTopics.accountBalanceDepositedRetry.name, DEFAULT_RETRY_COUNT) ) { event -> eventProcessor.on( ack = ack, consumerRecord = record, event = event, retryTopic = kafkaTopics.accountBalanceDepositedRetry.name ) } @KafkaListener( groupId = "\${kafka.consumer-group-id:account_microservice_group_id}", topics = ["\${topics.accountBalanceDepositedRetry.name}"], ) fun processRetry(ack: Acknowledgment, record: ConsumerRecord<String, ByteArray>) = eventProcessor.process( ack = ack, consumerRecord = record, deserializationClazz = BalanceDepositedEvent::class.java, onError = eventProcessor.errorRetryHandler(kafkaTopics.accountBalanceDepositedRetry.name, DEFAULT_RETRY_COUNT) ) { event -> eventProcessor.on( ack = ack, consumerRecord = record, event = event, retryTopic = kafkaTopics.accountBalanceDepositedRetry.name ) } } At the Application Layer, AccountEventsHandlerService is implemented in the following way: Kotlin interface AccountEventHandlerService { suspend fun on(event: AccountCreatedEvent): Either<AppError, Unit> suspend fun on(event: BalanceDepositedEvent): Either<AppError, Unit> suspend fun on(event: BalanceWithdrawEvent): Either<AppError, Unit> suspend fun on(event: PersonalInfoUpdatedEvent): Either<AppError, Unit> suspend fun on(event: ContactInfoChangedEvent): Either<AppError, Unit> suspend fun on(event: AccountStatusChangedEvent): Either<AppError, Unit> } @Component class AccountEventHandlerServiceImpl( private val accountProjectionRepository: AccountProjectionRepository ) : AccountEventHandlerService { override suspend fun on(event: AccountCreatedEvent): Either<AppError, Unit> = eitherScope(ctx) { accountProjectionRepository.save(event.toAccount()).bind() } override suspend fun on(event: BalanceDepositedEvent): Either<AppError, Unit> = eitherScope(ctx) { findAndUpdateAccountById(event.accountId, event.version) { account -> account.depositBalance(event.balance).bind() }.bind() } private suspend fun findAndUpdateAccountById( accountId: AccountId, eventVersion: Long, block: suspend (Account) -> Account ): Either<AppError, Account> = eitherScope(ctx) { val foundAccount = findAndValidateVersion(accountId, eventVersion).bind() val accountForUpdate = block(foundAccount) accountProjectionRepository.update(accountForUpdate).bind() } private suspend fun findAndValidateVersion( accountId: AccountId, eventVersion: Long ): Either<AppError, Account> = eitherScope(ctx) { val foundAccount = accountProjectionRepository.getById(accountId).bind() validateVersion(foundAccount, eventVersion).bind() foundAccount } } The infrastructure layer read model repository uses MongoDB's Kotlin coroutines driver: Kotlin interface AccountProjectionRepository { suspend fun save(account: Account): Either<AppError, Account> suspend fun update(account: Account): Either<AppError, Account> suspend fun getById(id: AccountId): Either<AppError, Account> suspend fun getByEmail(email: String): Either<AppError, Account> suspend fun getAll(page: Int, size: Int): Either<AppError, AccountsList> suspend fun upsert(account: Account): Either<AppError, Account> } Kotlin @Component class AccountProjectionRepositoryImpl( mongoClient: MongoClient, ) : AccountProjectionRepository { private val accountsDB = mongoClient.getDatabase(ACCOUNTS_DB) private val accountsCollection = accountsDB.getCollection<AccountDocument>(ACCOUNTS_COLLECTION) override suspend fun save(account: Account): Either<AppError, Account> = eitherScope<AppError, Account>(ctx) { val insertResult = accountsCollection.insertOne(account.toDocument()) log.info { "account insertOneResult: $insertResult, account: $account" } account } override suspend fun update(account: Account): Either<AppError, Account> = eitherScope(ctx) { val filter = and(eq(ACCOUNT_ID, account.accountId.string()), eq(VERSION, account.version)) val options = FindOneAndUpdateOptions().upsert(false).returnDocument(ReturnDocument.AFTER) accountsCollection.findOneAndUpdate( filter, account.incVersion().bind().toBsonUpdate(), options ) ?.toAccount() ?: raise(AccountNotFoundError("account with id: ${account.accountId} not found")) } override suspend fun upsert(account: Account): Either<AppError, Account> = eitherScope(ctx) { val filter = and(eq(ACCOUNT_ID, account.accountId.string())) val options = FindOneAndUpdateOptions().upsert(true).returnDocument(ReturnDocument.AFTER) accountsCollection.findOneAndUpdate( filter, account.toBsonUpdate(), options ) ?.toAccount() ?: raise(AccountNotFoundError("account with id: ${account.accountId} not found")) } override suspend fun getById(id: AccountId): Either<AppError, Account> = eitherScope(ctx) { accountsCollection.find<AccountDocument>(eq(ACCOUNT_ID, id.string())) .firstOrNull() ?.toAccount() ?: raise(AccountNotFoundError("account with id: $id not found")) } override suspend fun getByEmail(email: String): Either<AppError, Account> = eitherScope(ctx) { val filter = and(eq(CONTACT_INFO_EMAIL, email)) accountsCollection.find(filter).firstOrNull()?.toAccount() ?: raise(AccountNotFoundError("account with email: $email not found")) } override suspend fun getAll( page: Int, size: Int ): Either<AppError, AccountsList> = eitherScope<AppError, AccountsList>(ctx) { parZip(coroutineContext, { accountsCollection.find() .skip(page * size) .limit(size) .map { it.toAccount() } .toList() }, { accountsCollection.find().count() }) { list, totalCount -> AccountsList( page = page, size = size, totalCount = totalCount, accountsList = list ) } } } Read queries' way through the layers is very similar: we accept HTTP requests at the API layer: Kotlin @Tag(name = "Accounts", description = "Account domain REST endpoints") @RestController @RequestMapping(path = ["/api/v1/accounts"]) class AccountController( private val accountCommandService: AccountCommandService, private val accountQueryService: AccountQueryService ) { @Operation( method = "getAccountByEmail", operationId = "getAccountByEmail", description = "Get account by email", responses = [ ApiResponse( description = "Get account by email", responseCode = "200", content = [Content( mediaType = MediaType.APPLICATION_JSON_VALUE, schema = Schema(implementation = AccountResponse::class) )] ), ApiResponse( description = "bad request response", responseCode = "400", content = [Content(schema = Schema(implementation = ErrorHttpResponse::class))] )], ) @GetMapping(path = ["/email/{email}"]) suspend fun getAccountByEmail( @PathVariable @Email @Size( min = 6, max = 255 ) email: String ): ResponseEntity<out Any> = eitherScope(ctx) { accountQueryService.handle(GetAccountByEmailQuery(email)).bind() }.fold( ifLeft = { mapErrorToResponse(it) }, ifRight = { ResponseEntity.ok(it.toResponse()) } ) } Application Layer AccountQueryService methods: Kotlin interface AccountQueryService { suspend fun handle(query: GetAccountByIdQuery): Either<AppError, Account> suspend fun handle(query: GetAccountByEmailQuery): Either<AppError, Account> suspend fun handle(query: GetAllAccountsQuery): Either<AppError, AccountsList> } Kotlin @Service class AccountQueryServiceImpl( private val accountRepository: AccountRepository, private val accountProjectionRepository: AccountProjectionRepository ) : AccountQueryService { override suspend fun handle(query: GetAccountByIdQuery): Either<AppError, Account> = eitherScope(ctx) { accountRepository.getById(query.id).bind() } override suspend fun handle(query: GetAccountByEmailQuery): Either<AppError, Account> = eitherScope(ctx) { accountProjectionRepository.getByEmail(query.email).bind() } override suspend fun handle(query: GetAllAccountsQuery): Either<AppError, AccountsList> = eitherScope(ctx) { accountProjectionRepository.getAll(page = query.page, size = query.size).bind() } } And it uses PostgreSQL or MongoDB repositories to get the data depending on the query use case: Kotlin @Component class AccountProjectionRepositoryImpl( mongoClient: MongoClient, ) : AccountProjectionRepository { private val accountsDB = mongoClient.getDatabase(ACCOUNTS_DB) private val accountsCollection = accountsDB.getCollection<AccountDocument>(ACCOUNTS_COLLECTION) override suspend fun getByEmail(email: String): Either<AppError, Account> = eitherScope(ctx) { val filter = and(eq(CONTACT_INFO_EMAIL, email)) accountsCollection.find(filter) .firstOrNull() ?.toAccount() ?: raise(AccountNotFoundError("account with email: $email not found")) } } Final Thoughts In real-world applications, we have to implement many more necessary features, like K8s health checks, circuit breakers, rate limiters, etc., so this project is simplified for demonstration purposes. The source code is in my GitHub, please star it if is helpful and useful for you. For feedback or questions, feel free to contact me!
Docker has become an essential tool for developers, offering consistent and isolated environments without installing full-fledged products locally. The ideal setup for microservice development using Spring Boot with MySQL as the backend often involves a remotely hosted database. However, for rapid prototyping or local development, running a MySQL container through Docker offers a more streamlined approach. I encountered a couple of issues while attempting to set up this configuration with the help of Docker Desktop for a proof of concept. An online search revealed a lack of straightforward guides on integrating Spring Boot microservices with MySQL in Docker Desktop; most resources primarily focus on containerizing the Spring Boot application. Recognizing this gap, I decided to write this short article. Prerequisites Before diving in, we must have the following: A foundational understanding of Spring Boot and microservices architecture Familiarity with Docker containers Docker Desktop installed on our machine Docker Desktop Setup We can install Docker Desktop using this link. Installation is straightforward and includes steps that can be navigated efficiently, as illustrated in the accompanying screenshots. Configuring MySQL Container Once we have installed the Docker desktop when we launch, we will get through some standard questions, and we can skip the registration part. Once the desktop app is ready, then we need to search for the MySQL container, as shown below: We need to click Pull and then Run the container. Once you run the container, the settings dialog will pop up, as shown below. Please enter the settings as below: MYSQL_ROOT_PASSWORD: This environment variable specifies the password that will be set for the MySQL root superuser account. MYSQL_DATABASE: This environment variable allows us to specify the name of a database that will be created on image startup. If a user/password was supplied (see below), that user will be granted superuser access (corresponding to GRANT ALL) to this database. MYSQL_USER, MYSQL_PASSWORD: These variables are used to create a new user and set that user's password. This user will be granted superuser permissions for the database specified by the MYSQL_DATABASE variable. Upon running the container, Docker Desktop displays logs indicating the container's status. We can now connect to the MySQL instance using tools like MySQL Workbench to manage database objects. Spring Application Configuration In the Spring application, we can configure the configurations below in the application.properties. YAML spring.esign.datasource.jdbc-url=jdbc:mysql://localhost:3306/e-sign?allowPublicKeyRetrieval=true&useSSL=false spring.esign.datasource.username=e-sign spring.esign.datasource.password=Password1 We opted for a custom prefix spring.esign over the default spring.datasource for our database configuration within the Spring Boot application. This approach shines in scenarios where the application requires connections to multiple databases. To enable this custom configuration, we need to define the Spring Boot configuration class ESignDbConfig: Java @Configuration @EnableTransactionManagement @EnableJpaRepositories( entityManagerFactoryRef = "eSignEntityManagerFactory", transactionManagerRef = "eSignTransactionManager", basePackages ="com.icw.esign.repository") public class ESignDbConfig { @Bean("eSignDataSource") @ConfigurationProperties(prefix="spring.esign.datasource") public DataSource geteSignDataSource(){ return DataSourceBuilder.create().type(HikariDataSource.class).build(); } @Bean(name = "eSignEntityManagerFactory") public LocalContainerEntityManagerFactoryBean eSignEntityManagerFactory( EntityManagerFactoryBuilder builder, @Qualifier("eSignDataSource") DataSource dataSource) { return builder.dataSource(dataSource).packages("com.icw.esign.dao") .build(); } @Bean(name = "eSignTransactionManager") public PlatformTransactionManager eSignTransactionManager(@Qualifier("eSignEntityManagerFactory") EntityManagerFactory entityManagerFactory) { return new JpaTransactionManager(entityManagerFactory); } } @Bean("eSignDataSource"): This method defines a Spring bean for the eSign module's data source. The @ConfigurationProperties(prefix="spring.esign.datasource") annotation is used to automatically map and bind all configuration properties starting with spring.esign.datasource from the application's configuration files (like application.properties or application.yml) to this DataSource object. The method uses DataSourceBuilder to create and configure a HikariDataSource, a highly performant JDBC connection pool. This implies that the eSign module will use a dedicated database whose connection parameters are isolated from other modules or the main application database. @Bean(name = "eSignEntityManagerFactory"): This method creates a LocalContainerEntityManagerFactoryBean, which is responsible for creating the EntityManagerFactory. This factory is crucial for managing JPA entities specific to the eSign module. The EntityManagerFactory is configured to use the eSignDataSource for its database operations and to scan the package com.icw.esign.dao for entity classes. This means that only entities in this package or its subpackages will be managed by this EntityManagerFactory and thus, can access the eSign database. @Bean(name = "eSignTransactionManager"): This defines a PlatformTransactionManager specific way of managing transactions of the eSignmodule's EntityManagerFactory. This transaction manager ensures that all database operations performed by entities managed by the eSignEntityManagerFactory are wrapped in transactions. It enables the application to manage transaction boundaries, roll back operations on failures, and commit changes when operations succeed. Repository Now that we have defined configurations, we can create repository classes and build other objects required for the API endpoint. Java @Repository public class ESignDbRepository { private static final Logger logger = LoggerFactory.getLogger(ESignDbRepository.class); @Qualifier("eSignEntityManagerFactory") @Autowired private EntityManager entityManager; @Autowired ObjectMapper objectMapper; String P_GET_DOC_ESIGN_INFO = "p_get_doc_esign_info"; public List<DocESignMaster> getDocumentESignInfo(String docUUID) { StoredProcedureQuery proc = entityManager.createStoredProcedureQuery(P_GET_DOC_ESIGN_INFO, DocESignMaster.class); proc.registerStoredProcedureParameter("v_doc_uuid", String.class, ParameterMode.IN); proc.setParameter("v_doc_uuid", docUUID); try { return (List<DocESignMaster>) proc.getResultList(); } catch (PersistenceException ex) { logger.error("Error while fetching document eSign info for docUUID: {}", docUUID, ex); } return Collections.emptyList(); } } @Qualifier("eSignEntityManagerFactory"): Specifies which EntityManagerFactory should be used to create EntityManager, ensuring that the correct database configuration is used for eSign operations. Conclusion Integrating Spring Boot microservices with Docker Desktop streamlines microservice development and testing. This guide walks through the essential steps of setting up a Spring Boot application and ensuring seamless service communication with a MySQL container hosted on the Docker Desktop application. This quick setup guide is useful for proof of concept or setting up an isolated local development environment.
What Is OAuth2? OAuth 2.0 is an authorization protocol. It provides the framework to obtain limited access to a protected resource by a third-party application on behalf of the resource owner. For example, we log in to our LinkedIn account using Google account username and password. The Google authorization (OAuth2.0) server grants a temporary access token to LinkedIn which authorizes the user to access LinkedIn resources. Note that here, LinkedIn trusts Google to validate the user and acts as an authorization proxy. What Are Microservices? A microservice is a service-oriented architecture pattern wherein applications are built as a collection of various smallest independent service units. It is a software engineering approach that focuses on decomposing an application into single-function modules with well-defined interfaces. These modules can be independently deployed and operated by small teams who own the entire life-cycle of the service. Why Is OAuth2 a Good Solution for Secure Communications With Microservices? This idea of separation of concern leverages microservices security by decoupling the part that does authorization from business logic. The responsibility is delegated to a centralized and trusted authorization server and the actual application is free from security concerns in this regard. It promotes the granularity of service which microservices typically are all about. Apart from reducing complexity, OAuth 2.0 in microservices provides a platform to implement consistent and standard security policies across the system. The authorization is flexible, meaning it can be revoked at any time. This helps security management to restrict unnecessary or limited access to resources. Since access tokens provided by the OAuth 2.0 server are stateless (JSON Web Token - JWT), it eliminates the need for storage and transmission of sensitive credentials or session data between microservices. Overall, both (OAuth 2.0 and microservices) in combination enhance the performance and scalability of the system. Purpose I wanted a solution where we could easily capture OAuth2 and OAuth2 clients for secure communication with all of the microservices, focusing on how to achieve OAuth2 full flavor in a microservices architecture. The user can’t access API without a token. The token will be available when the user is given basic authentication details to generate a token for access API. All requests will consider one entry point (API Gateway), but service-to-service can communicate. The API Gateway will use dynamic routing with the Zuul Netflix OSS component. Every request will check authorization when the request arrives in the service, and the service will request the authorization server to verify if it is either authenticated or not. The entire Meta configuration settled into the central configuration on GitHub (you can manage it on any repository). Goal Achieve authentication/authorization, based on Spring Security, OAuth2, and OAuth2 client Understanding microservices architecture using Spring Cloud and Netflix OSS Demonstration of microservice architecture based on Java, Spring, and OAuth2 Spring Cloud and Microservices Firstly, we do not write a microservice. We write a service that eventually will be called microservice when deployed with other services to form an application. Having said that, Spring Cloud just gives you abstractions over some set of tools (Eureka, Zuul, Feign, Ribbon, etc.), making it easy for you to integrate with spring applications. However, you can also achieve microservice architecture without using Spring Cloud. You can take advantage of tools like Kubernetes, Docker Swarm, HAProxy, Kong, NGINX, etc. to achieve the same. Using Spring Cloud has its own pros and cons and vice versa. High-Level Microservice Architecture With Authorizations Users log in to the system using basic authorization and login credentials. The user will get a token if the user's basic auth and login credentials are matched. Next, the user sends a request to access data from the service. The API gateway receives the request and checks with the authorization server. Every request has one entry point API Gateway. Security checking and dynamic routing to the service Every service has a single database to manipulate data. Spring Cloud Key Concept and Features Spring Cloud works for microservices to manage configuration. Intelligent routing and services discovery Service-to-service call Load balancing (it properly distributes network traffic to the backend server) Leadership election (the application works with another application as a third-party system) Global lock (two threads are not accessed simultaneously for the same resource at the same time) Distributed configuration and messaging If you want to avail many services in one application, then the cloud-based application is an easy way. Spring Cloud works in the same way. Spring Boot Key Concept and Features Spring Boot works to create microservices. Spring application creates a stand-alone Spring application. Web application HTTP embedded (Tomcat, Jetty, or Undertow); no need to deploy WAR file Externalized configuration Security (it is secure inbuilt with basic authentication on all HTTP endpoints) Application event and listener Spring Boot works on product-based web applications. It is used for unit test development and integration test time reduction. Spring Cloud Advantages Provides cloud service development Microservice-based architecture and configuration Provides inter-service communication Based on the Spring Boot model Spring Cloud 5 Main Annotations 1. @EnableConfigServer This annotation converts the application into a server which more applications use to get their configuration. 2. @EnableEurekaServer This annotation is used for Eureka Discovery Services for other applications that can be used to locate services using it. 3. @EnableDiscoveryClient This annotation helps an application register in the service discovery and discover other services using it. 4. @EnableCircuitBreaker Use the circuit breaker pattern to continue operating when related services fail and prevent cascading failure. This annotation is used for Hystrix Circuit Breaker. 5. @HyStrixCommand(fallbackmethod="MethodName") Hystrix is a latency and fault tolerance library for distributed systems. 4 Common Netflix Components Spring Cloud Netflix provides Netflix OSS integrations for Spring Boot apps through autoconfiguration and binding to the Spring Environment and other Spring programming model idioms. With a few simple annotations, you can quickly enable and configure the common patterns inside your application and build large distributed systems with battle-tested Netflix components. The patterns provided include Service Discovery (Eureka), Circuit Breaker (Hystrix), Intelligent Routing (Zuul), and Client-Side Load Balancing (Ribbon). 1. Eureka (Service Registration and Discovery) REST service which registers itself at the registry (Eureka Client) Web application, which is consuming the REST service as a registry-aware client (Spring Cloud Netflix Feign Client) 2. Ribbon (Dynamic Routing and Load Balancer) Ribbon primarily provides client-side load-balancing algorithms. APIs that integrate load balancing, fault tolerance, caching/batching on top of other Ribbon modules and Hystrix REST client built on top of Apache HttpClient integrated with load balancers (deprecated and being replaced by ribbon module Configurable load-balancing rules 3. Hystrix (Circuit Breaker) Hystrix is a fault-tolerance Java library. This tool is designed to separate points of access to remote services, systems, and 3rd-party libraries in a distributed environment like microservices. It improves the overall system by isolating the failing services and preventing the cascading effect of failures. 4. Zuul (Edge Server) Zuul is the front door for all requests from devices and websites to the backend of the Netflix streaming application. Zuul will serve as our API gateway. Handle dynamic routing Built to enable dynamic routing, monitoring, resiliency, and security What Is a Feign Client? Netflix provides Feign as an abstraction over REST-based calls, by which microservices can communicate with each other, but developers do not have to bother about REST internal details. Feign Client, which works on the declarative principle. We must create an interface/contract, then Spring creates the original implementation on the fly, so a REST-based service call is abstracted from developers. Not only that — if you want to customize the call, like encoding your request or decoding the response in a custom object, you can do it with Feign in a declarative way. Feign, as a client, is an important tool for microservice developers to communicate with other microservices via Rest API. The Feign Client uses a declarative approach for accessing the API. To use it, we must first enable the Spring Cloud support for it on our Spring Boot Application with the @EnableFeignClients annotation at the class level on a @Configuration class. Server Side Load Balancing In JavaEE architecture, we deploy our WAR/EAR files into multiple application servers, then we create a pool of servers and put a load balancer (Netscaler) in front of it, which has a public IP. The client makes a request using that public IP, and Netscaler decides in which internal application server it forwards the request by round robin or sticky session algorithm. We call it server-side load balancing. Technology Stack Java 8+ Spring latest Spring Security OAuth2, OAuth2 Client Spring Cloud Netflix OSS PostgreSQL IntelliJ How To Implement OAuth2 Security in Microservices Step 1: Create Project "central configuration" for All Services With microservices, we create a central config server where all configurable parameters of microservices are written and version-controlled. The benefit of a central config server is that if we change a property for a microservice, it can reflect that on the fly without redeploying the microservice. You can create a project using spring initializr. application.properties: Properties files xxxxxxxxxx 1 13 1 spring.application.name=ehealth-central-configuration 2 server.port=8888 3 eureka.client.service-url.defaultZone=http://localhost:8761/eureka/ 4 # available profiles of the application 5 spring.profiles.active=local,development,production 6 spring.cloud.config.server.git.uri=https://github.com/amran-bd/cloud-config 7 spring.cloud.config.server.git.clone-on-start=true 8 spring.cloud.config.server.git.search-paths=patient-management-service,ehealth-api-gateway,eureka-service-discovery,clinic-management-service 9 management.security.enabled=false 10 #To remove WAR - Could not locate PropertySource: None of labels [] found 11 health.config.enabled=false 12 # To remove I/O Issue Could not locate PropertySource: I/O error on GET request for 13 spring.cloud.config.enabled=false Hint: You can use your Git server or local machine. A new service name will be added if a new service is introduced. Here is the link to my GitHub repository if you would like to use it. EhealthCentralConfigurationApplication.Java Class Example: Java xxxxxxxxxx 1 15 1 package com.amran.central.config; 2 3 import org.springframework.boot.SpringApplication; 4 import org.springframework.boot.autoconfigure.SpringBootApplication; 5 import org.springframework.cloud.config.server.EnableConfigServer; 6 7 @EnableConfigServer 8 @SpringBootApplication 9 public class EhealthCentralConfigurationApplication { 10 11 public static void main(String[] args) { 12 SpringApplication.run(EhealthCentralConfigurationApplication.class, args); 13 } 14 15 } You must include an annotation @EnableConfigServer. Example of Creating a Central Configuration for Services: Hint: Create folder <projectName>/<projectName-development.properties>. Step 2: Create a Project "Discovery Server" for All Discoverable Services We have already discussed the discovery server in this article. bootstrap.properties: Properties files xxxxxxxxxx 1 1 spring.application.name=eureka-service-discovery 2 spring.profiles.active=development 3 # ip and port of the config server 4 spring.cloud.config.uri=http://localhost:8888 5 # expose actuator endpoints 6 management.endpoints.web.exposure.include=refresh 7 8 management.security.enabled=false 9 spring.cloud.config.fail-fast=true Here, we can enable and disable other actuator endpoints through property files.If you want to enable all actuator endpoints, then add the following property:management.endpoints.web.exposure.include=*. To enable only specific actuator endpoints, provide the list of endpoint IDs: management.endpoints.web.exposure.include=health,info, beans,env In some cases, it may be desirable to fail the startup of a service if it cannot connect to the Config Server. If this is the desired behavior, set the bootstrap configuration property spring.cloud.config.fail.Fast=true and the client will halt with an exception. EurkeaServiceDiscoveryApplication.java Class Example: Java xxxxxxxxxx 1 15 1 package com.amran.service.discovery; 2 3 import org.springframework.boot.SpringApplication; 4 import org.springframework.boot.autoconfigure.SpringBootApplication; 5 import org.springframework.cloud.netflix.eureka.server.EnableEurekaServer; 6 7 @SpringBootApplication 8 @EnableEurekaServer 9 public class EurekaServiceDiscoveryApplication { 10 11 public static void main(String[] args) { 12 SpringApplication.run(EurekaServiceDiscoveryApplication.class, args); 13 } 14 15 } You must include an annotation @EnableEurekaServer. Step 3: Create Project "API Gateway" for All Services Entry Points This is the most valuable portion. Here, we write the authorization server in the same project. API Gateway Project Structure application.yml Properties files xxxxxxxxxx 1 19 1 #hystrix: 2 # command: 3 # default: 4 # execution: 5 # isolation: 6 # thread: 7 # timeoutInMilliseconds: 5000 8 hystrix: 9 command: 10 clinic-management-service: 11 execution: 12 isolation: 13 thread: 14 timeoutInMilliseconds: 5000 15 patient-management-service: 16 execution: 17 isolation: 18 thread: 19 timeoutInMilliseconds: 5000 Here, define timeout for every separate service; you can use the default. bootstrap.properties Properties files xxxxxxxxxx 1 1 spring.application.name=ehealth-api-gateway 2 spring.profiles.active=development 3 # ip and port of the config server 4 spring.cloud.config.uri=http://localhost:8888 5 # expose actuator endpoints 6 management.endpoints.web.exposure.include=refresh 7 8 management.security.enabled=false 9 spring.cloud.config.fail-fast=true Central Configuration Example ehealth-api-gateway-development.properties: Properties files x 1 spring.application.name=ehealth-api-gateway 2 server.port=8080 3 eureka.client.service-url.defaultZone=http://localhost:8761/eureka/ 4 5 ## PostgreSQL 6 spring.datasource.url=jdbc:postgresql://localhost:3307/ehealth-security 7 spring.datasource.username=postgres 8 spring.datasource.password=test1373 9 spring.datasource.type=com.zaxxer.hikari.HikariDataSource 10 # Hikari will use the above plus the following to setup connection pooling 11 spring.datasource.hikari.minimumIdle=3 12 spring.datasource.hikari.maximumPoolSize=500 13 spring.datasource.hikari.idleTimeout=30000 14 spring.datasource.hikari.poolName=SpringBootJPAHikariCP 15 spring.datasource.hikari.maxLifetime=2000000 16 spring.datasource.hikari.connectionTimeout=30000 17 spring.datasource.pool-prepared-statements=true 18 spring.datasource.max-open-prepared-statements=250 19 spring.jpa.hibernate.connection.provider_class=org.hibernate.hikaricp.internal.HikariCPConnectionProvider 20 spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQL82Dialect 21 22 #Hibernate Configuration 23 spring.jpa.generate-ddl = true 24 spring.jpa.hibernate.ddl-auto=update 25 spring.jpa.show-sql=true 26 27 server.error.include-stacktrace=never 28 29 30 #feign.hystrix.enabled=true 31 #hystrix.shareSecurityContext=true 32 33 #All url come with prefix/api will interpret 34 zuul.prefix=/api 35 36 #Dynamic Service Registration in Eureka Server (API Gateway) 37 zuul.routes.patient-management-service.path=/patient-management-service/** 38 #zuul.routes.patient-management-service.url=http://localhost:8081 39 zuul.routes.patient-management-service.sensitive-headers 40 zuul.routes.patient-management-service.service-id=patient-management-service 41 42 zuul.routes.clinic-management-service.path=/clinic-management-service/** 43 #zuul.routes.patient-management-service.url=http://localhost:8082 44 zuul.routes.clinic-management-service.sensitive-headers 45 zuul.routes.clinic-management-service.service-id=clinic-management-service Zuul filtered 4-types while doing dynamic routing. Zuul filters store request and state information in (and share it using) the RequestContext. You can use that to get to the HttpServletRequest and then log the HTTP method and URL of the request before it is sent on its way. ErrorFilter, PreFilter, PostFilter, and RouteFilter Java xxxxxxxxxx 1 30 1 package com.amran.api.gateway.filter; 2 3 import com.netflix.zuul.ZuulFilter; 4 5 /** 6 * @Author : Amran Hosssain on 6/27/2020 7 */ 8 public class RouteFilter extends ZuulFilter { 9 10 @Override 11 public String filterType() { 12 return "route"; 13 } 14 15 @Override 16 public int filterOrder() { 17 return 1; 18 } 19 20 @Override 21 public boolean shouldFilter() { 22 return true; 23 } 24 25 @Override 26 public Object run() { 27 System.out.println("Inside Route Filter"); 28 return null; 29 } 30 } EhealthApiGatewayApplication.java Class Example Java xxxxxxxxxx 1 42 1 package com.amran.api.gateway; 2 3 import com.amran.api.gateway.filter.PostFilter; 4 import com.amran.api.gateway.filter.PreFilter; 5 import com.amran.api.gateway.filter.ErrorFilter; 6 import com.amran.api.gateway.filter.RouteFilter; 7 import org.springframework.boot.SpringApplication; 8 import org.springframework.boot.autoconfigure.SpringBootApplication; 9 import org.springframework.cloud.client.circuitbreaker.EnableCircuitBreaker; 10 import org.springframework.cloud.client.discovery.EnableDiscoveryClient; 11 import org.springframework.cloud.netflix.zuul.EnableZuulProxy; 12 import org.springframework.cloud.openfeign.EnableFeignClients; 13 import org.springframework.context.annotation.Bean; 14 15 @EnableFeignClients 16 @EnableCircuitBreaker 17 @EnableDiscoveryClient 18 @EnableZuulProxy 19 @SpringBootApplication 20 public class EhealthApiGatewayApplication { 21 22 public static void main(String[] args) { 23 SpringApplication.run(EhealthApiGatewayApplication.class, args); 24 } 25 26 @Bean 27 public PreFilter preFilter() { 28 return new PreFilter(); 29 } 30 @Bean 31 public PostFilter postFilter() { 32 return new PostFilter(); 33 } 34 @Bean 35 public ErrorFilter errorFilter() { 36 return new ErrorFilter(); 37 } 38 @Bean 39 public RouteFilter routeFilter() { 40 return new RouteFilter(); 41 } 42 } AsEhealthApiGatewayApplication.java class annotation has been already discussed, see above if it is not clear. Spring Security and Oauth2 Implementation in Microservices Architecture I have done OAuth2 implementation based on Spring Security. Step 4: Create a Project "patient-management-service" Patient-related data will be manipulated. POM.xml XML xxxxxxxxxx 1 114 1 <?xml version="1.0" encoding="UTF-8"?> 2 <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 3 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"> 4 <modelVersion>4.0.0</modelVersion> 5 <parent> 6 <groupId>org.springframework.boot</groupId> 7 <artifactId>spring-boot-starter-parent</artifactId> 8 <version>2.3.1.RELEASE</version> 9 <relativePath/> <!-- lookup parent from repository --> 10 </parent> 11 <groupId>com.amran.patient.management</groupId> 12 <artifactId>patient-management-service</artifactId> 13 <version>0.0.1-SNAPSHOT</version> 14 <name>patient-management-service</name> 15 <description>patient-management-service project for Spring Boot</description> 16 17 <properties> 18 <java.version>1.8</java.version> 19 <spring-cloud.version>Hoxton.SR5</spring-cloud.version> 20 </properties> 21 22 <dependencies> 23 <dependency> 24 <groupId>org.springframework.boot</groupId> 25 <artifactId>spring-boot-starter-actuator</artifactId> 26 </dependency> 27 <dependency> 28 <groupId>org.springframework.boot</groupId> 29 <artifactId>spring-boot-starter-data-jpa</artifactId> 30 </dependency> 31 <dependency> 32 <groupId>org.springframework.cloud</groupId> 33 <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> 34 </dependency> 35 <dependency> 36 <groupId>org.springframework.boot</groupId> 37 <artifactId>spring-boot-starter-web</artifactId> 38 </dependency> 39 40 <dependency> 41 <groupId>org.springframework.cloud</groupId> 42 <artifactId>spring-cloud-config-client</artifactId> 43 </dependency> 44 45 <dependency> 46 <groupId>org.springframework.boot</groupId> 47 <artifactId>spring-boot-starter-security</artifactId> 48 </dependency> 49 <dependency> 50 <groupId>org.springframework.security.oauth</groupId> 51 <artifactId>spring-security-oauth2</artifactId> 52 <version>2.5.0.RELEASE</version> 53 <scope>compile</scope> 54 </dependency> 55 <dependency> 56 <groupId>org.postgresql</groupId> 57 <artifactId>postgresql</artifactId> 58 <scope>runtime</scope> 59 </dependency> 60 <dependency> 61 <groupId>org.springframework.boot</groupId> 62 <artifactId>spring-boot-configuration-processor</artifactId> 63 <optional>true</optional> 64 </dependency> 65 <dependency> 66 <groupId>org.projectlombok</groupId> 67 <artifactId>lombok</artifactId> 68 <optional>true</optional> 69 </dependency> 70 <dependency> 71 <groupId>org.springframework.boot</groupId> 72 <artifactId>spring-boot-starter-test</artifactId> 73 <scope>test</scope> 74 <exclusions> 75 <exclusion> 76 <groupId>org.junit.vintage</groupId> 77 <artifactId>junit-vintage-engine</artifactId> 78 </exclusion> 79 </exclusions> 80 </dependency> 81 </dependencies> 82 83 <dependencyManagement> 84 <dependencies> 85 <dependency> 86 <groupId>org.springframework.cloud</groupId> 87 <artifactId>spring-cloud-dependencies</artifactId> 88 <version>${spring-cloud.version}</version> 89 <type>pom</type> 90 <scope>import</scope> 91 </dependency> 92 </dependencies> 93 </dependencyManagement> 94 95 <build> 96 <finalName>${project.artifactId}</finalName> 97 <plugins> 98 <plugin> 99 <groupId>org.springframework.boot</groupId> 100 <artifactId>spring-boot-maven-plugin</artifactId> 101 </plugin> 102 </plugins> 103 <resources> 104 <resource> 105 <filtering>true</filtering> 106 <directory>src/main/resources</directory> 107 <includes> 108 <include>*.properties</include> 109 </includes> 110 </resource> 111 </resources> 112 </build> 113 114 </project> bootstrap.properties Properties files x 1 server.url = Patient Management Service Working... 2 spring.application.name=patient-management-service 3 spring.profiles.active=development 4 # ip and port of the config server where we can get our central configuration. 5 spring.cloud.config.uri=http://localhost:8888 6 # expose actuator endpoints 7 management.endpoints.web.exposure.include=refresh 8 9 management.security.enabled=false 10 spring.cloud.config.fail-fast=true 11 12 13 ##Security parameter for request verification ## 14 #we consider basic authorization and Token. In auth server verified this token generated by authorization server (Self) based below criteria. 15 client_id=kidclient 16 client_credential = kidsecret 17 check_authorization_url = http://localhost:8080/oauth/check_token 18 resources_id = ehealth PatientManagementServiceApplication.java Class Example Java xxxxxxxxxx 1 14 1 package com.amran.patient.management; 2 3 import org.springframework.boot.SpringApplication; 4 import org.springframework.boot.autoconfigure.SpringBootApplication; 5 import org.springframework.cloud.client.discovery.EnableDiscoveryClient; 6 7 @EnableDiscoveryClient 8 @SpringBootApplication 9 public class PatientManagementServiceApplication { 10 11 public static void main(String[] args) { 12 SpringApplication.run(PatientManagementServiceApplication.class, args); 13 } 14 } You must annotate @EnableDiscoveryClient in the class so the Eureka server will be discovered as a service or client. Security OAuth2-Client: ResourceServerConfig.java, WebSecurityConfig.Java Class Example You need a WebSecurityConfigurerAdapter to secure the /authorize endpoint and to provide a way for users to authenticate. A Spring Boot application would do that for you (by adding its own WebSecurityConfigurerAdapter with HTTP basic auth). It creates a filter chain with order=0 by default and protects all resources unless you provide a request marcher. The @EnableResourceServer does something similar, but the filter chain it adds is at order=3 by default. WebSecurityConfigurerAdapter has an @Order(100) annotation. So first the ResourceServer will be checked (authentication), and then your checks in your extension of WebSecurityConfigureAdapter will be checked. Java xxxxxxxxxx 1 38 1 package com.amran.patient.management.security; 2 3 import org.springframework.beans.factory.annotation.Value; 4 import org.springframework.context.annotation.Configuration; 5 import org.springframework.http.HttpMethod; 6 import org.springframework.security.config.annotation.web.builders.HttpSecurity; 7 import org.springframework.security.oauth2.config.annotation.web.configuration.EnableResourceServer; 8 import org.springframework.security.oauth2.config.annotation.web.configuration.ResourceServerConfigurerAdapter; 9 import org.springframework.security.oauth2.config.annotation.web.configurers.ResourceServerSecurityConfigurer; 10 11 /** 12 * @Author : Amran Hosssain on 6/27/2020 13 */ 14 @Configuration 15 @EnableResourceServer 16 public class ResourceServerConfig extends ResourceServerConfigurerAdapter { 17 18 @Value("${resources_id}") 19 private String resourceId; 20 21 @Override 22 public void configure(HttpSecurity http) throws Exception { 23 http 24 .headers().frameOptions().disable() 25 .and() 26 .csrf().disable() 27 .authorizeRequests() 28 .antMatchers("/eureka/**").permitAll() 29 .anyRequest() 30 .authenticated(); 31 } 32 33 @Override 34 public void configure(ResourceServerSecurityConfigurer resources) throws Exception { 35 resources.resourceId(resourceId); 36 } 37 38 } Java xxxxxxxxxx 1 44 1 package com.amran.patient.management.security; 2 3 import org.springframework.beans.factory.annotation.Value; 4 import org.springframework.context.annotation.Bean; 5 import org.springframework.context.annotation.Configuration; 6 import org.springframework.security.authentication.AuthenticationManager; 7 import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity; 8 import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter; 9 import org.springframework.security.oauth2.provider.authentication.OAuth2AuthenticationManager; 10 import org.springframework.security.oauth2.provider.token.RemoteTokenServices; 11 import org.springframework.security.oauth2.provider.token.ResourceServerTokenServices; 12 13 /** 14 * @Author : Amran Hosssain on 6/27/2020 15 */ 16 @Configuration 17 @EnableWebSecurity 18 public class WebSecurityConfig extends WebSecurityConfigurerAdapter { 19 20 @Value("${client_id}") 21 private String clientId; 22 23 @Value("${client_credential}") 24 private String clientSecret; 25 26 @Value("${check_authorization_url}") 27 private String checkAuthUrl; 28 29 @Bean 30 public ResourceServerTokenServices tokenServices() { 31 RemoteTokenServices tokenServices = new RemoteTokenServices(); 32 tokenServices.setClientId(clientId); 33 tokenServices.setClientSecret(clientSecret); 34 tokenServices.setCheckTokenEndpointUrl(checkAuthUrl); 35 return tokenServices; 36 } 37 38 @Override 39 public AuthenticationManager authenticationManagerBean() throws Exception { 40 OAuth2AuthenticationManager authenticationManager = new OAuth2AuthenticationManager(); 41 authenticationManager.setTokenServices(tokenServices()); 42 return authenticationManager; 43 } 44 } Check How It Works 1. Generate Token Project run sequence: CentralConfigServer->DiscoveryServer->API Gateway Server-> Others Service 2. Client Details In Database 3. User Record 4. Generate Token 5. Call Patient Management Service (Zuul Dynamic Routing) 6. Direct Call Patient Service (Token Verify From Auth Server) Note: Without a token, you can't call. 7. Call Clinic Management Service (Zuul Dynamic Routing) Conclusion I am trying to show OAuth2 implementation in microservice architecture with secure communication, single entry point, dynamic routing, fail-back solutions, centralized configurations, and OAuth2-client implementation in service to secure every API and every request to ensure authorization. Source Code The full source code can be found here.
Microservices have emerged as a transformative architectural approach in the realm of software development, offering a paradigm shift from monolithic structures to a more modular and scalable system. At its core, microservices involve breaking down complex applications into smaller, independently deployable services that communicate seamlessly, fostering agility, flexibility, and ease of maintenance. This decentralized approach allows developers to focus on specific functionalities, enabling rapid development, continuous integration, and efficient scaling to meet the demands of modern, dynamic business environments. As organizations increasingly embrace the benefits of microservices, this article explores the key principles, advantages, and challenges associated with this architectural style, shedding light on its pivotal role in shaping the future of software design and deployment. A fundamental characteristic of microservices applications is the ability to design, develop, and deploy each microservice independently, utilizing diverse technology stacks. Each microservice functions as a self-contained, autonomous application with its own dedicated persistent storage, whether it be a relational database, a NoSQL DB, or even a legacy file storage system. This autonomy enables individual microservices to scale independently, facilitating seamless real-time infrastructure adjustments and enhancing overall manageability. NCache Caching Layer in Microservice Architecture In scenarios where application transactions surge, bottlenecks may persist, especially in architectures where microservices store data in non-scalable relational databases. Simply deploying additional instances of the microservice doesn't alleviate the problem. To address these challenges, consider integrating NCache as a distributed cache at the caching layer between microservices and datastores. NCache serves not only as a cache but also functions as a scalable in-memory publisher/subscriber messaging broker, facilitating asynchronous communication between microservices. Microservice Java application performance optimization can be achieved by the cache techniques like Cache item locking, grouping Cache data, Hibernate Caching, SQL Query, data structure, spring data cache technique pub-sub messaging, and many more with NCache. Please check the out-of-the-box features provided by NCache. Using NCache as Hibernate Second Level Java Cache Hibernate First-Level Cache The Hibernate first-level cache serves as a fundamental standalone (in-proc) cache linked to the Session object, limited to the current session. Nonetheless, a drawback of the first-level cache is its inability to share objects between different sessions. If the same object is required by multiple sessions, each triggers a database trip to load it, intensifying database traffic and exacerbating scalability issues. Furthermore, when the session concludes, all cached data is lost, necessitating a fresh fetch from the database upon the next retrieval. Hibernate Second-Level Cache For high-traffic Hibernate applications relying solely on the first-level cache, deployment in a web farm introduces challenges related to cache synchronization across servers. In a web farm setup, each node operates a web server—such as Apache, Oracle WebLogic, etc.—with multiple instances of httpd processes to serve requests. Each Hibernate first-level cache in these HTTP worker processes maintains a distinct version of the same data directly cached from the database, posing synchronization issues. This is why Hibernate offers a second-level cache with a provider model. The Hibernate second-level cache enables you to integrate third-party distributed (out-proc) caching providers to cache objects across sessions and servers. Unlike the first-level cache, the second-level cache is associated with the SessionFactory object and is accessible to the entire application, extending beyond a single session. Enabling the Hibernate second-level cache results in the coexistence of two caches: the first-level cache and the second-level cache. Hibernate endeavors to retrieve objects from the first-level cache first; if unsuccessful, it attempts to fetch them from the second-level cache. If both attempts fail, the objects are directly loaded from the database and cached. This configuration substantially reduces database traffic, as a significant portion of the data is served by the second-level distributed cache. NCache Java has implemented a Hibernate second-level caching provider by extending org.hibernate.cache.CacheProvider. Integrating NCache Java Hibernate distributed caching provider with the Hibernate application requires no code changes. This integration enables you to scale your Hibernate application to multi-server configurations without the database becoming a bottleneck. NCache also delivers enterprise-level distributed caching features, including data size management, data synchronization across servers, and more. To incorporate the NCache Java Hibernate caching provider, a simple modification of your hibernate.cfg.xml and ncache.xml is all that is required. Thus, with the NCache Java Hibernate distributed cache provider, you can achieve linear scalability for your Hibernate applications seamlessly, requiring no alterations to your existing code. Code Snippet Java // Configure Hibernate properties programmatically Properties hibernateProperties = new Properties(); hibernateProperties.put("hibernate.connection.driver_class", "org.h2.Driver"); hibernateProperties.put("hibernate.connection.url", "jdbc:h2:mem:testdb"); hibernateProperties.put("hibernate.show_sql", "false"); hibernateProperties.put("hibernate.hbm2ddl.auto", "create-drop"); hibernateProperties.put("hibernate.cache.use_query_cache", "true"); hibernateProperties.put("hibernate.cache.use_second_level_cache", "true"); hibernateProperties.put("hibernate.cache.region.factory_class", "org.hibernate.cache.jcache.internal.JCacheRegionFactory"); hibernateProperties.put("hibernate.javax.cache.provider", "com.alachisoft.ncache.hibernate.jcache.HibernateNCacheCachingProvider"); // Set other Hibernate properties as needed Configuration configuration = new Configuration() .setProperties(hibernateProperties).addAnnotatedClass(Product.class); Logger.getLogger("org.hibernate").setLevel(Level.OFF); // Build the ServiceRegistry ServiceRegistry serviceRegistry = new StandardServiceRegistryBuilder() .applySettings(configuration.getProperties()).build(); // Build the SessionFactory SessionFactory factory = configuration.buildSessionFactory(serviceRegistry); // Create a List of Product objects ArrayList<Product> products = (ArrayList<Product>) getProducts(); // Open a new Hibernate session to save products to the database. This also caches it try (Session session = factory.openSession()) { Transaction transaction = session.beginTransaction(); // save() method saves products to the database and caches it too System.out.println("ProductID, Name, Price, Category"); for (Product product : products) { System.out.println("- " + product.getProductID() + ", " + product.getName() + ", " + product.getPrice() + ", " + product.getCategory()); session.save(product); } transaction.commit(); System.out.println(); // Now open a new session to fetch products from the DB. // But, these products are actually fetched from the cache try (Session session = factory.openSession()) { List<Product> productList = (List<Product>) session.createQuery("from Product").list(); if (productList != null) { printProductDetails(productList); } } Integrate NCache with Hibernate to effortlessly cache the results of queries. When these objects are subsequently fetched by Hibernate, they are retrieved from the cache, thereby avoiding a costly trip to the database. From the above code sample, the products are saved in the database, and it also caches; now, when the new session opens to fetch the product details, it will fetch from the Cache and avoid an unnecessary database trip. Learn more about Hibernate Caching Scaling With NCache Pub/Sub Messaging NCache is a distributed in-memory caching solution designed for .NET. Its compatibility extends to Java through a native client and third-party integrations, ensuring seamless support for both platforms. NCache serves as an in-memory distributed data store tailored for .NET and Java, offering a feature-rich, in-memory pub/sub mechanism for event-driven communication. This makes it straightforward to set up NCache as a messaging broker, employing the Pub/Sub model for seamless asynchronous communication between microservices. Using NCache In-Memory Pub/Sub for Microservices NCache enables Pub/Sub functionality by establishing a topic where microservices can publish and subscribe to events. These events are published to the NCache message broker outside the microservice. Within each subscribing microservice, there exists an event handler to manage the corresponding event once it has been published by the originating microservice. In the realm of Java microservices, NCache functions as an event bus or message broker, facilitating the relay of messages to one or multiple subscribers. In the context of Pub/Sub models that necessitate a communication channel, NCache serves as a medium for topics. This entails the publisher dispatching messages to the designated topic and subscribers receiving notifications through the same topic. Employing NCache as the medium for topics promotes loose coupling within the model, offering enhanced abstraction and additional advantages for distributed topics. Publish The code snippet below initializes the messageService object using NCache MessagingService package. Initializing the Topic Java // Create a Topic in NCache. MessagingService messagingService = cache.getMessagingService(); Topic topic = messagingService.createTopic(topicName); // Create a thread pool for publishers ExecutorService publisherThreadPool = Executors.newFixedThreadPool(2); The below code snippet used to define register the subscribers to this topic Register subscribers to this Topic MessageReceivedListener subscriptionListener1 = new MessageReceivedListener() { @Override public void onMessageReceived(Object o, MessageEventArgs messageEventArgs) { messageReceivedSubscription1(messageEventArgs.getMessage()); } }; MessageReceivedListener subscriptionListener2 = new MessageReceivedListener() { @Override public void onMessageReceived(Object o, MessageEventArgs messageEventArgs) { messageReceivedSubscription2(messageEventArgs.getMessage()); } }; TopicSubscription subscription1 = topic.createSubscription(subscriptionListener1); TopicSubscription subscription2 = topic.createSubscription(subscriptionListener2); NCache provides two variants of durable subscriptions to cater to the message durability needs within your Java microservices: Shared Durable Subscriptions: This allows multiple subscribers to connect to a single subscription. The Round Robin approach is employed to distribute messages among the various subscribers. Even if a subscriber exits the network, messages persistently flow between the active subscribers. Exclusive Durable Subscriptions: In this type, only one active subscriber is allowed on a subscription at any given time. No new subscriber requests are accepted for the same subscription until the existing connection is active. Learn more Pub/Sub Messaging with NCache implementation here Pub/Sub Messaging in Cache: An Overview SQL Query on Cache NCache provides your microservices with the capability to perform SQL-like queries on indexed cache data. This functionality becomes particularly beneficial when the values of the keys storing the desired information are not known. It abstracts much of the lower-level cache API calls, contributing to clearer and more maintainable application code. This feature is especially advantageous for individuals who find SQL-like commands more intuitive and comfortable to work with. NCache provides functionality for searching and removing cache data through queries similar to SQL's SELECT and DELETE statements. However, operations like INSERT and UPDATE are not available. For executing SELECT queries within the cache, NCache utilizes ExecuteReader; the ExecuteScalar function is used to carry out a query and retrieve the first row's first column from the resulting data set, disregarding any extra columns or rows. For NCache SQL queries to function, indexes must be established on all objects undergoing search. This can be achieved through two methods: configuring the cache or utilizing code with "Custom Attributes" to annotate object fields. When objects are added to the cache, this approach automatically creates indexes on the specified fields. Code Snippet Java String cacheName = "demoCache"; // Connect to the cache and return a cache handle Cache cache = CacheManager.getCache(cacheName); // Adds all the products to the cache. This automatically creates indexes on various // attributes of Product object by using "Custom Attributes". addSampleData(cache); // $VALUE$ keyword means the entire object instead of individual attributes that are also possible String sql = "SELECT $VALUE$ FROM com.alachisoft.ncache.samples.Product WHERE category IN (?, ?) AND price < ?"; QueryCommand sqlCommand = new QueryCommand(sql); List<String> catParamList = new ArrayList<>(Arrays.asList(("Electronics"), ("Stationery"))); sqlCommand.getParameters().put("category", catParamList); sqlCommand.getParameters().put("price", 2000); // ExecuteReader returns ICacheReader with the query resultset CacheReader resultSet = cache.getSearchService().executeReader(sqlCommand); List<Product> fetchedProducts = new ArrayList<>(); if (resultSet.getFieldCount() > 0) { while (resultSet.read()) { // getValue() with $VALUE$ keyword returns the entire object instead of just one column fetchedProducts.add(resultSet.getValue("$VALUE$", Product.class)); } } printProducts(fetchedProducts); Utilize SQL in NCache to perform queries on cached data by focusing on object attributes and Tags, rather than solely relying on keys. In this example, we utilize "Custom Attributes" to generate an index on the Product object. Learn more about SQL Query with NCache in Java Query Data in Cache Using SQL Read-Thru and Write-Thru Utilize the Data Source Providers feature of NCache to position it as the primary interface for data access within your microservices architecture. When a microservice needs data, it should first query the cache. If the data is present, the cache supplies it directly. Otherwise, the cache employs a read-thru handler to fetch the data from the datastore on behalf of the client, caches it, and then provides it to the microservice. In a similar fashion, for write operations (such as Add, Update, Delete), a microservice can perform these actions on the cache. The cache then automatically carries out the corresponding write operation on the datastore using a write-thru handler. Furthermore, you have the option to compel the cache to fetch data directly from the datastore, regardless of the presence of a possibly outdated version in the cache. This feature is essential when microservices require the most current information and complements the previously mentioned cache consistency strategies. The integration of the Data Source Provider feature not only simplifies your application code but also, when combined with NCache's database synchronization capabilities, ensures that the cache is consistently updated with fresh data for processing. ReadThruProvider For implementing Read-Through caching, it's necessary to create an implementation of the ReadThruProvider interface in Java Here's a code snippet to get started with implementing Read-Thru in your microservices: Java ReadThruOptions readThruOptions = new ReadThruOptions(ReadMode.ReadThru, _readThruProviderName); product = _cache.get(_productId, readThruOptions, Product.class); Read more about Read-Thru implementation here: Read-Through Provider Configuration and Implementation WriteThruProvider: For implementing Write-Through caching, it's necessary to create an implementation of the WriteThruProvider interface in Java The code snippet to get started with implementing Write-Thru in your microservices: Java _product = new Product(); WriteThruOptions writeThruOptions = new WriteThruOptions(WriteMode.WriteThru, _writeThruProviderName) CacheItem cacheItem= new CacheItem(_customer) _cache.insert(_product.getProductID(), cacheItem, writeThruOptions); Read more about Write-Thru implementation here: Write-Through Provider Configuration and Implementation Summary Microservices are designed to be autonomous, enabling independent development, testing, and deployment from other microservices. While microservices provide benefits in scalability and rapid development cycles, some components of the application stack can present challenges. One such challenge is the use of relational databases, which may not support the necessary scale-out to handle growing loads. This is where a distributed caching solution like NCache becomes valuable. In this article, we have seen the variety of ready-to-use features like pub/sub messaging, data caching, SQL Query, Read-Thru and Write-Thru, and Hibernate second-level Java Cache techniques offered by NCache that simplify and streamline the integration of data caching into your microservices application, making it an effortless and natural extension.
Technical Architecture First, let's turn to the architecture, which will be explained in detail. Let's look at each of these tiers in detail. Let me explain the architecture in detail. These components are commonly associated with the architecture of applications that follow the principles of Domain-Driven Design (DDD) and Model-View-Controller (MVC) or similar architectural patterns. Let me cover this one by one: Entities Entities represent the core business objects or concepts in your application. They encapsulate data related to the business domain. For example, in an Employee Management System, an employee entity might have attributes like name, email, and salary related to an employee. Repositories Repositories are responsible for handling the data access logic. They provide an abstraction over the data storage, allowing the application to interact with the data without worrying about the underlying storage details. For example, an EmployeeRepository would handle operations like storing, retrieving, updating, and deleting employee records in the database. Services Services contain business logic that doesn't naturally fit within the methods of an entity. They orchestrate interactions between entities and repositories to fulfill higher-level use cases. For example, an EmployeeService might have methods to calculate bonuses, process employee transfers, or handle complex business rules involving multiple entities. Mappers Mappers are responsible for transforming data between different layers of the application. They convert data from database entities to domain objects and vice versa. For example, an EmployeeMapper might convert an Employee entity to a data transfer object (EmployeeRequest) that can be sent over the network or used by the presentation layer. Controllers Controllers handle incoming requests from the user interface or external systems. They interpret user input, invoke the necessary services or business logic, and prepare the response to be sent back. In a web application, a controller receives an HTTP request, extracts the necessary data, and delegates the request to the appropriate service. It then formats the service response and sends it back to the client. Frontend: You have the option of building Native Apps like Android and iOS. Desktop browser apps or mobile browser apps can be built using React or Angular frameworks. Best Practices for Implementation of the Architecture Entity: Useful Tips Name the package as “entities” under the feature name Set id as Long and generation type as identity Name the class and table in plurals like users Use Lombok for constructor and getter/setter code Have a length for every String field Set nullable to either true/false Use references to other tables like @ManyToOne. Remember, the table created is automatic, and what you write in the entity matters. Use @OneToMany bidirectional if you wish to save the values in multiple tables in one call. Use @ManyToMany to join tables. Create a separate Join class if there are fields in the join table apart from join id columns. Identify the right inheritance type for is-a relationship. Pick between single table, class table, and concrete table inheritance based on the number of fields in every class. Example Java package org.project.feature.entities; import jakarta.persistence.*; import lombok.AllArgsConstructor; import lombok.Getter; import lombok.NoArgsConstructor; import lombok.Setter; @Getter @Setter @NoArgsConstructor @AllArgsConstructor @Entity @Table(name = "users") public class Users { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; @Column(length = 100, nullable = false) private String firstName; @Column(length = 100, nullable = false) private String lastName; @Column(length = 100, nullable = false, unique = true) private String email; } Repository: Useful Tips Name the package as “repositories” under the feature name Extend the JpaRepository with the entity name and id as Long As much as possible, use the style as a method for querying the entity like findByEmail Use batch operations for multiple entries to the database, like saveAll Use Optional for the return type as much as possible Example Java package org.project.feature.repositories; import org.project.feature.entities.Users; import org.springframework.data.jpa.repository.JpaRepository; public interface UsersRepository extends JpaRepository<Users, Long> { } Service: Useful Tips Name the package as “services” under the feature name Create an interface for all the operations within the service and create an implementation class Use @AllArgsConstructor for @Autowired annotation Accept the Request object and return the Response object from the model’s package. If multiple repositories need to be called, it should be called in a transaction unless you wish to start a new transaction. If you wish to call multiple services, the particular service has to be named as an aggregate service and within a transaction. Do not return the ResponseEntity from the service; it is the job of the controller tier. Example Java package org.project.feature.services; import org.project.feature.models.UsersRequest; import org.project.feature.models.UsersResponse; import java.util.List; public interface UsersService { UsersResponse createUser(UsersRequest usersRequest); UsersResponse getUserById(Long userId); List<UsersResponse> getAllUsers(); UsersResponse updateUser(Long id, UsersRequest users); void deleteUser(Long userId); } Java package org.project.feature.services.impl; import lombok.AllArgsConstructor; import org.project.feature.entities.Users; import org.project.feature.mappers.UsersMapper; import org.project.feature.models.UsersRequest; import org.project.feature.models.UsersResponse; import org.project.feature.repositories.UsersRepository; import org.project.feature.services.UsersService; import org.springframework.stereotype.Service; import java.util.ArrayList; import java.util.List; import java.util.Optional; @Service @AllArgsConstructor public class UsersServiceImpl implements UsersService { private final UsersRepository usersRepository; private final UsersMapper usersMapper; @Override public UsersResponse createUser(UsersRequest usersRequest) { Users users = usersMapper.convertRequestToEntity(usersRequest); Users saved = usersRepository.save(users); return usersMapper.convertEntityToResponse(saved); } @Override public UsersResponse getUserById(Long userId) { Optional<Users> optionalUser = usersRepository.findById(userId); return usersMapper.convertEntityToResponse(optionalUser.get()); } @Override public List<UsersResponse> getAllUsers() { List<Users> users = usersRepository.findAll(); List<UsersResponse> usersResponses = new ArrayList<>(); for (Users user : users) usersResponses.add(usersMapper.convertEntityToResponse(user)); return usersResponses; } @Override public UsersResponse updateUser(Long id, UsersRequest usersRequest) { Optional<Users> user = usersRepository.findById(id); Users existingUsers = user.orElse(null); existingUsers.setFirstName(usersRequest.getFirstName()); existingUsers.setLastName(usersRequest.getLastName()); existingUsers.setEmail(usersRequest.getEmail()); Users updatedUsers = usersRepository.save(existingUsers); return usersMapper.convertEntityToResponse(updatedUsers); } @Override public void deleteUser(Long userId) { usersRepository.deleteById(userId); } } Mappers: Useful Tips Name the package as “mappers” under the feature name Create an interface called Mapper using Generics and convert the entity to a model and vice versa using this mapper Do not use the entity as a return object in the controller tier Example Java package org.project.feature.mappers; import org.project.feature.entities.Users; import org.project.feature.models.UsersRequest; import org.project.feature.models.UsersResponse; import org.springframework.stereotype.Component; @Component public class UsersMapper { public UsersResponse convertEntityToResponse(Users users) { UsersResponse usersResponse = new UsersResponse(); usersResponse.setId(users.getId()); usersResponse.setFirstName(users.getFirstName()); usersResponse.setLastName(users.getLastName()); usersResponse.setEmail(users.getEmail()); return usersResponse; } public Users convertRequestToEntity(UsersRequest usersRequest) { Users users = new Users(); users.setFirstName(usersRequest.getFirstName()); users.setLastName(usersRequest.getLastName()); users.setEmail(usersRequest.getEmail()); return users; } } Model: Useful Tips Name the package as “models” under the feature name All requests and response objects will be stored here Use @Data annotation for the model classes The model should act as a frontend for the API, and the service should convert the model to an entity for talking to the repository. Example Java package org.project.feature.models; import jakarta.persistence.Column; import lombok.Data; import java.io.Serializable; @Data public class UsersRequest implements Serializable { private String firstName; private String lastName; private String email; } Java package org.project.feature.models; import jakarta.persistence.Column; import lombok.Data; import java.io.Serializable; @Data public class UsersResponse implements Serializable { private Long id; private String firstName; private String lastName; private String email; } Controller: Useful Tips Name the package as “controllers” under the feature name Try to create an API for every resource under a bounded context Make the resource names in plural, like /API/users For CRUD Operations: Use HTTP POST for create an operation with Request as the body Use HTTP PUT for update operation Use HTTP GET for retrieve all records Use HTTP GET with /{id} for retrieve with an identifier Use HTTP DELETE with /{id} to delete the record For operation other, the CRUD try avoiding a verb as much as possible Implement error handling at the controller tier Implement validation at the controller tier with @Valid Realize the difference between API thinking and RPC thinking. It is key to understanding APIs. Example Java package org.project.feature.controllers; import lombok.AllArgsConstructor; import org.project.feature.models.UsersRequest; import org.project.feature.models.UsersResponse; import org.project.feature.services.UsersService; import org.springframework.http.HttpStatus; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.*; import java.util.List; @RestController @AllArgsConstructor @RequestMapping("/api/users") public class UsersController { private final UsersService usersService; @PostMapping public ResponseEntity<UsersResponse> createUser(@RequestBody UsersRequest usersRequest){ UsersResponse savedUsers = usersService.createUser(usersRequest); return new ResponseEntity<>(savedUsers, HttpStatus.CREATED); } @GetMapping("{id}") public ResponseEntity<UsersResponse> getUserById(@PathVariable("id") Long userId){ UsersResponse users = usersService.getUserById(userId); return new ResponseEntity<>(users, HttpStatus.OK); } @GetMapping public ResponseEntity<List<UsersResponse>> getAllUsers(){ List<UsersResponse> users = usersService.getAllUsers(); return new ResponseEntity<>(users, HttpStatus.OK); } @PutMapping("{id}") public ResponseEntity<UsersResponse> updateUser(@PathVariable("id") Long userId, @RequestBody UsersRequest usersRequest){ UsersResponse updatedUsers = usersService.updateUser(userId, usersRequest); return new ResponseEntity<>(updatedUsers, HttpStatus.OK); } @DeleteMapping("{id}") public ResponseEntity<String> deleteUser(@PathVariable("id") Long userId){ usersService.deleteUser(userId); return new ResponseEntity<>("User successfully deleted!", HttpStatus.OK); } } Conclusion In a long-term vision, as and when the code base becomes bulky, you might need to use the Strangler pattern to take out some of the services and deploy them as a separate microservice. This kind of coding structure will help then. If you get your basics right from the very beginning, then later on, the ride will be smooth.
In fintech application mobile apps or the web, deploying new features in areas like loan applications requires careful validation. Traditional testing with real user data, especially personally identifiable information (PII), presents significant challenges. Synthetic transactions offer a solution, enabling the thorough testing of new functionalities in a secure and controlled environment without compromising sensitive data. By simulating realistic user interactions within the application, synthetic transactions enable developers and QA teams to identify potential issues in a controlled environment. Synthetic transactions help in ensuring that every aspect of a financial application functions correctly after any major updates or new features are rolled out. In this article, we delve into one of the approaches for using synthetic transactions. Synthetic Transactions for Financial Applications Key Business Entity At the heart of every financial application lies a key entity, be it a customer, user, or loan application itself. This entity is often defined by a unique identifier, serving as the cornerstone for transactions and operations within the system. The inception point of this entity, when it is first created, presents a strategic opportunity to categorize it as either synthetic or real. This categorization is critical, as it determines the nature of interactions the entity will undergo. Marking an entity as synthetic or for test purposes from the outset allows for a clear delineation between test and real data within the application's ecosystem. Subsequently, all transactions and operations conducted with this entity can be safely recognized as part of synthetic transactions. This approach ensures that the application's functionality can be thoroughly tested in a realistic environment. Intercepting and Managing Synthetic Transactions A critical component of implementing synthetic transactions lies in the interception and management of these transactions at the HTTP request level. Utilizing Spring's HTTP Interceptor mechanism, we can discern and process synthetic transactions by examining specific HTTP headers. The below visual outlines the coordination between a synthetic HTTP interceptor and a state manager in managing the execution of an HTTP request: Figure 1: Synthetic HTTP interceptor and state manager The SyntheticTransactionInterceptor acts as the primary gatekeeper, ensuring that only transactions identified as synthetic are allowed through the testing pathways. Below is the implementation: Java @Component public class SyntheticTransactionInterceptor implements HandlerInterceptor { protected final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired SyntheticTransactionService syntheticTransactionService; @Autowired SyntheticTransactionStateManager syntheticTransactionStateManager; @Override public boolean preHandle(HttpServletRequest request,HttpServletResponse response, Object object) throws Exception { String syntheticTransactionId = request.getHeader("x-synthetic-transaction-uuid"); if (syntheticTransactionId != null && !syntheticTransactionId.isEmpty()){ if (this.syntheticTransactionService.validateTransactionId(syntheticTransactionId)){ logger.info(String.format("Request initiated for synthetic transaction with transaction id:%s", syntheticTransactionId)); this.syntheticTransactionStateManager.setSyntheticTransaction(true); this.syntheticTransactionStateManager.setTransactionId(syntheticTransactionId); } } return true; } } In this implementation, the interceptor looks for a specific HTTP header (x-synthetic-transaction-uuid) carrying a UUID. This UUID is not just any identifier but a validated, time-limited key designated for synthetic transactions. The validation process includes checks on the UUID's validity, its lifespan, and whether it has been previously used, ensuring a level of security and integrity for the synthetic testing process. After a synthetic ID is validated by the SyntheticTransactionInterceptor, the SyntheticTransactionStateManager plays a pivotal role in maintaining the synthetic context for the current request. The SyntheticTransactionStateManager is designed with request scope in mind, meaning its lifecycle is tied to the individual HTTP request. This scoping is essential for preserving the integrity and isolation of synthetic transactions within the application's broader operational context. By tying the state manager to the request scope, the application ensures that synthetic transaction states do not bleed over into unrelated operations or requests. Below is the implementation of the synthetic state manager: Java @Component @RequestScope public class SyntheticTransactionStateManager { private String transactionId; private boolean syntheticTransaction; public String getTransactionId() { return transactionId; } public void setTransactionId(String transactionId) { this.transactionId = transactionId; } public boolean isSyntheticTransaction() { return syntheticTransaction; } public void setSyntheticTransaction(boolean syntheticTransaction) { this.syntheticTransaction = syntheticTransaction; } } When we persist the key entity, be it a customer, user, or loan application—the application's service layer or repository layer consults the SyntheticTransactionStateManager to confirm the transaction's synthetic nature. If the transaction is indeed synthetic, the application proceeds to persist not only the synthetic identifier but also an indicator that the entity itself is synthetic. This sets the foundations for the synthetic transaction flow. This approach ensures that from the moment an entity is marked as synthetic, all related operations and future APIs, whether they involve data processing or business logic execution, are conducted in a controlled manner. For further API calls initiated from the financial application, upon reaching the microservice, we load the application context for that specific request based on the token or entity identifier provided. During the context loading, we ascertain whether the key business entity (e.g., loan application, user/customer) is synthetic. If affirmative, we then set the state manager's syntheticTransaction flag to true and also assign the synthetic transactionId from the application context. This approach negates the need to pass a synthetic transaction ID header for subsequent calls within the application flow. We only need to send a synthetic transaction ID during the initial API call that creates the key business entity. Since this step involves using explicit headers that may not be supported by the financial application, whether it's a mobile or web platform, we can manually make this first API call with Postman or a similar tool. Afterwards, the application can continue with the rest of the flow in the financial application itself. Beyond managing synthetic transactions within the application, it's also crucial to consider how external third-party API calls behave within the context of the synthetic transaction. External Third-Party API Interactions In financial applications handling key entities with personally identifiable information (PII), we conduct validations and fraud checks on user-provided data, often leveraging external third-party services. These services are crucial for tasks such as PII validation and credit bureau report retrieval. However, when dealing with synthetic transactions, we cannot make calls to these third-party services. The solution involves creating mock responses or utilizing stubs for these external services during synthetic transactions. This approach ensures that while synthetic transactions undergo the same processing logic as real transactions, they do so without the need for actual data submission to third-party services. Instead, we simulate the responses that these services would provide if they were called with real data. This allows us to thoroughly test the integration points and data-handling logic of our application. Below is the code snippet for pulling the bureau report. This call happens as part of the API call where the key entity is been created, and then subsequently we pull the applicant's bureau report: Java @Override @Retry(name = "BUREAU_PULL", fallbackMethod = "getBureauReport_Fallback") public CreditBureauReport getBureauReport(SoftPullParams softPullParams, ErrorsI error) { CreditBureauReport result = null; try { Date dt = new Date(); logger.info("UWServiceImpl::getBureauReport method call at :" + dt.toString()); CreditBureauReportRequest request = this.getCreditBureauReportRequest(softPullParams); RestTemplate restTemplate = this.externalApiRestTemplateFactory.getRestTemplate(softPullParams.getUserLoanAccountId(), "BUREAU_PULL", softPullParams.getAccessToken(), "BUREAU_PULL", error); HttpHeaders headers = this.getHttpHeaders(softPullParams); HttpEntity<CreditBureauReportRequest> entity = new HttpEntity<>(request, headers); long startTime = System.currentTimeMillis(); String uwServiceEndPoint = "/transaction"; String bureauCallUrl = String.format("%s%s", appConfig.getUnderwritingTransactionApiPrefix(), uwServiceEndPoint); if (syntheticTransactionStateManager.isSyntheticTransaction()) { result = this.syntheticTransactionService.getPayLoad(syntheticTransactionStateManager.getTransactionId(), "BUREAU_PULL", CreditBureauReportResponse.class); result.setCustomerId(softPullParams.getUserAccountId()); result.setLoanAccountId(softPullParams.getUserLoanAccountId()); } else { ResponseEntity<CreditBureauReportResponse> responseEntity = restTemplate.exchange(bureauCallUrl, HttpMethod.POST, entity, CreditBureauReportResponse.class); result = responseEntity.getBody(); } long endTime = System.currentTimeMillis(); long timeDifference = endTime - startTime; logger.info("Time taken for API call BUREAU_PULL/getBureauReport call 1: " + timeDifference); } catch (HttpClientErrorException exception) { logger.error("HttpClientErrorException occurred while calling BUREAU_PULL API, response string: " + exception.getResponseBodyAsString()); throw exception; } catch (HttpStatusCodeException exception) { logger.error("HttpStatusCodeException occurred while calling BUREAU_PULL API, response string: " + exception.getResponseBodyAsString()); throw exception; } catch (Exception ex) { logger.error("Error occurred in getBureauReport. Detail error:", ex); throw ex; } return result; } The code snippet above is quite elaborate, but we don't need to get into the details of that. What we need to focus on is the code snippet below: Java if (syntheticTransactionStateManager.isSyntheticTransaction()) { result = this.syntheticTransactionService.getPayLoad(syntheticTransactionStateManager.getTransactionId(), "BUREAU_PULL", CreditBureauReportResponse.class); result.setCustomerId(softPullParams.getUserAccountId()); result.setLoanAccountId(softPullParams.getUserLoanAccountId()); } else { ResponseEntity<CreditBureauReportResponse> responseEntity = restTemplate.exchange(bureauCallUrl, HttpMethod.POST, entity, CreditBureauReportResponse.class); result = responseEntity.getBody(); } It checks for the synthetic transaction with the SyntheticTransactionStateManager. If true, then instead of going to a third party, it calls the internal service SyntheticTransactionService to get the Synthetic Bureau report data. Synthetic Data Service Synthetic data service SyntheticTransactionServiceImpl is a general utility service whose responsibility is to pull the synthetic data from the data store, parse it, and convert it to the object type that is been passed as part of the parameter. Below is the implementation for the service: Java @Service @Qualifier("syntheticTransactionServiceImpl") public class SyntheticTransactionServiceImpl implements SyntheticTransactionService { private final Logger logger = LoggerFactory.getLogger(this.getClass()); @Autowired SyntheticTransactionRepository syntheticTransactionRepository; @Override public <T> T getPayLoad(String transactionUuid, String extPartnerServiceType, Class<T> responseType) { T payload = null; try { SyntheticTransactionPayload syntheticTransactionPayload = this.syntheticTransactionRepository.getSyntheticTransactionPayload(transactionUuid, extPartnerServiceType); if (syntheticTransactionPayload != null && syntheticTransactionPayload.getPayload() != null){ ObjectMapper objectMapper = new ObjectMapper() .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false); payload = objectMapper.readValue(syntheticTransactionPayload.getPayload(), responseType); } } catch (Exception ex){ logger.error("An error occurred while getting the synthetic transaction payload, detail error:", ex); } return payload; } @Override public boolean validateTransactionId(String transactionId) { boolean result = false; try{ if (transactionId != null && !transactionId.isEmpty()) { if (UUID.fromString(transactionId).toString().equalsIgnoreCase(transactionId)) { //Removed other validation checks, this could be financial application specific check. } } } catch (Exception ex){ logger.error("SyntheticTransactionServiceImpl::validateTransactionId - An error occurred while validating the synthetic transaction id, detail error:", ex); } return result; } With the generic method getPayLoad(), we provide a high degree of reusability, capable of returning various types of synthetic responses. This reduces the need for multiple, specific mock services for different external interactions. For storing the different payloads for different types of external third-party services, we use a generic table structure as below: MySQL CREATE TABLE synthetic_transaction ( id int NOT NULL AUTO_INCREMENT, transaction_uuid varchar(36) ext_partner_service varchar(30) payload mediumtext create_date datetime DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (id) ); ext_partner_service: This is an external service identifier for which we pull the payload from the table. In this above example for bureau report, it would be BUREAU_PULL. Conclusion In our exploration of synthetic transactions within fintech applications, we've highlighted their role in enhancing the reliability, and integrity of fintech solutions. By leveraging synthetic transactions, we simulate realistic user interactions while circumventing the risks tied to handling real personally identifiable information (PII). This approach enables our developers and QA teams to rigorously test new functionalities and updates in a secure, controlled environment. Moreover, our strategy in integrating synthetic transactions through mechanisms such as HTTP interceptors and state managers showcases a versatile approach applicable across a wide array of applications. This method not only simplifies the incorporation of synthetic transactions but also significantly boosts reusability, alleviating the need to devise unique workflows for each third-party service interaction. This approach significantly enhances the reliability and security of financial application solutions, ensuring that new features can be deployed with confidence.
As a quick recap, in Part 1: We built a simple gRPC service for managing topics and messages in a chat service (like a very simple version of Zulip, Slack, or Teams). gRPC provided a very easy way to represent the services and operations of this app. We were able to serve (a very rudimentary implementation) from localhost on an arbitrary port (9000 by default) on a custom TCP protocol. We were able to call the methods on these services both via a CLI utility (grpc_cli) as well as through generated clients (via tests). The advantage of this approach is that any app/site/service can access this running server via a client (we could also generate JS or Swift or Java clients to make these calls in the respective environments). At a high level, the downsides to this approach to this are: Network access: Usually, a network request (from an app or a browser client to this service) has to traverse several networks over the internet. Most networks are secured by firewalls that only permit access to specific ports and protocols (80:http, 443:https), and having this custom port (and protocol) whitelisted on every firewall along the way may not be tractable. Discomfort with non-standard tools: Familiarity and comfort with gRPC are still nascent outside the service-building community. For most service consumers, few things are easier and more accessible than HTTP-based tools (cURL, HTTPie, Postman, etc). Similarly, other enterprises/organizations are used to APIs exposed as RESTful endpoints, so having to build/integrate non-HTTP clients imposes a learning curve. Use a Familiar Cover: gRPC-Gateway We can have the best of both worlds by enacting a proxy in front of our service that translates gRPC to/from the familiar REST/HTTP to/from the outside world. Given the amazing ecosystem of plugins in gRPC, just such a plugin exists — the gRPC-Gateway. The repo itself contains a very in-depth set of examples and tutorials on how to integrate it into a service. In this guide, we shall apply it to our canonical chat service in small increments. A very high-level image (courtesy of gRPC-Gateway) shows the final wrapper architecture around our service: This approach has several benefits: Interoperability: Clients that need and only support HTTP(s) can now access our service with a familiar facade. Network support: Most corporate firewalls and networks rarely allow non-HTTP ports. With the gRPC-Gateway, this limitation can be eased as the services are now exposed via an HTTP proxy without any loss in translation. Client-side support: Today, several client-side libraries already support and enable REST, HTTP, and WebSocket communication with servers. Using the gRPC-Gateway, these existing tools (e.g., cURL, HTTPie, postman) can be used as is. Since no custom protocol is exposed beyond the gRPC-Gateway, complexity (for implementing clients for custom protocols) is eliminated (e.g., no need to implement a gRPC generator for Kotlin or Swift to support Android or Swift). Scalability: Standard HTTP load balancing techniques can be applied by placing a load-balancer in front of the gRPC-Gateway to distribute requests across multiple gRPC service hosts. Building a protocol/service-specific load balancer is not an easy or rewarding task. Overview You might have already guessed: protoc plugins again come to the rescue. In our service's Makefile (see Part 1), we generated messages and service stubs for Go using the protoc-gen-go plugin: protoc --go_out=$OUT_DIR --go_opt=paths=source_relative \ --go-grpc_out=$OUT_DIR --go-grpc_opt=paths=source_relative \ --proto_path=$PROTO_DIR \ $PROTO_DIR/onehub/v1/*.proto A Brief Introduction to Plugins The magic of the protoc plugin is that it does not perform any generation on its own but orchestrates plugins by passing the parsed Abstract Syntax Tree (AST) across plugins. This is illustrated below: Step 0: Input files (in the above case, onehub/v1/*.proto) are passed to the protoc plugin. Step 1: The protoc tool first parses and validates all proto files. Step 2:protoc then invokes each plugin in its list command line arguments in turn by passing a serialized version of all the proto files it has parsed into an AST. Step 3: Each proto plugin (in this case, go and go-grpcreads this serialized AST via its stdin. The plugin processes/analyzes these AST representations and generates file artifacts. Note that there does not need to be a 1:1 correspondence between input files (e.g., A.proto, B.proto, C.proto) and the output file artifacts it generates. For example, the plugin may create a "single" unified file artifact encompassing all the information in all the input protos. The plugin writes out the generated file artifacts onto its stdout. Step 4: protoc tool captures the plugin's stdout and for each generated file artifact, serializes it onto disk. Questions How does protoc know which plugins to invoke? Any command line argument to protoc in the format --<pluginname>_out is a plugin indicator with the name "pluginname". In the above example, protoc would have encountered two plugins: go and go-grpc. Where does protoc find the plugin? protoc uses a convention of finding an executable with the name protoc-gen-<pluginname>. This executable must be found in the folders in the $PATH variable. Since plugins are just plain executables these can be written in any language. How can I serialize/deserialize the AST? The wire format for the AST is not needed. protoc has libraries (in several languages) that can be included by the executables that can deserialize ASTs from stdin and serialize generated file artifacts onto stdout. Setup As you may have guessed (again), our plugins will also need to be installed before they can be invoked by protoc. We shall install the gRPC-Gateway plugins. For a detailed set of instructions, follow the gRPC-Gateway installation setup. Briefly: go get \ github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \ github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \ google.golang.org/protobuf/cmd/protoc-gen-go \ google.golang.org/grpc/cmd/protoc-gen-go-grpc # Install after the get is required go install \ github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \ github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \ google.golang.org/protobuf/cmd/protoc-gen-go \ google.golang.org/grpc/cmd/protoc-gen-go-grpc This will install the following four plugins in your $GOBIN folder: protoc-gen-grpc-gateway - The GRPC Gateway generator protoc-gen-openapiv2 - Swagger/OpenAPI spec generator protoc-gen-go - The Go protobuf protoc-gen-go-grpc - Go gRPC server stub and client generator Make sure that your "GOBIN" folder is in your PATH. Add Makefile Targets Assuming you are using the example from Part 1, add an extra target to the Makefile: gwprotos: echo "Generating gRPC Gateway bindings and OpenAPI spec" protoc -I . --grpc-gateway_out $(OUT_DIR) \ --grpc-gateway_opt logtostderr=true \ --grpc-gateway_opt paths=source_relative \ --grpc-gateway_opt generate_unbound_methods=true \ --proto_path=$(PROTO_DIR)/onehub/v1/ \ $(PROTO_DIR)/onehub/v1/*.proto Notice how the parameter types are similar to one in Part 1 (when we were generating go bindings). For each file X.proto, just like the go and go-grpc plugin, an X.pb.gw.go file is created that contains the HTTP bindings for our service. Customizing the Generated HTTP Bindings In the previous sections .pb.gw.go files were created containing default HTTP bindings of our respective services and methods. This is because we had not provided any URL bindings, HTTP verbs (GET, POST, etc.), or parameter mappings. We shall address that shortcoming now by adding custom HTTP annotations to the service's definition. While all our services have a similar structure, we will look at the Topic service for its HTTP annotations. Topic service with HTTP annotations: syntax = "proto3"; import "google/protobuf/field_mask.proto"; option go_package = "github.com/onehub/protos"; package onehub.v1; import "onehub/v1/models.proto"; import "google/api/annotations.proto"; /** * Service for operating on topics */ service TopicService { /** * Create a new sesssion */ rpc CreateTopic(CreateTopicRequest) returns (CreateTopicResponse) { option (google.api.http) = { post: "/v1/topics", body: "*", }; } /** * List all topics from a user. */ rpc ListTopics(ListTopicsRequest) returns (ListTopicsResponse) { option (google.api.http) = { get: "/v1/topics" }; } /** * Get a particular topic */ rpc GetTopic(GetTopicRequest) returns (GetTopicResponse) { option (google.api.http) = { get: "/v1/topics/{id=*}" }; } /** * Batch get multiple topics by ID */ rpc GetTopics(GetTopicsRequest) returns (GetTopicsResponse) { option (google.api.http) = { get: "/v1/topics:batchGet" }; } /** * Delete a particular topic */ rpc DeleteTopic(DeleteTopicRequest) returns (DeleteTopicResponse) { option (google.api.http) = { delete: "/v1/topics/{id=*}" }; } /** * Updates specific fields of a topic */ rpc UpdateTopic(UpdateTopicRequest) returns (UpdateTopicResponse) { option (google.api.http) = { patch: "/v1/topics/{topic.id=*}" body: "*" }; } } /** * Topic creation request object */ message CreateTopicRequest { /** * Topic being updated */ Topic topic = 1; } /** * Response of an topic creation. */ message CreateTopicResponse { /** * Topic being created */ Topic topic = 1; } /** * An topic search request. For now only paginations params are provided. */ message ListTopicsRequest { /** * Instead of an offset an abstract "page" key is provided that offers * an opaque "pointer" into some offset in a result set. */ string page_key = 1; /** * Number of results to return. */ int32 page_size = 2; } /** * Response of a topic search/listing. */ message ListTopicsResponse { /** * The list of topics found as part of this response. */ repeated Topic topics = 1; /** * The key/pointer string that subsequent List requests should pass to * continue the pagination. */ string next_page_key = 2; } /** * Request to get an topic. */ message GetTopicRequest { /** * ID of the topic to be fetched */ string id = 1; } /** * Topic get response */ message GetTopicResponse { Topic topic = 1; } /** * Request to batch get topics */ message GetTopicsRequest { /** * IDs of the topic to be fetched */ repeated string ids = 1; } /** * Topic batch-get response */ message GetTopicsResponse { map<string, Topic> topics = 1; } /** * Request to delete an topic. */ message DeleteTopicRequest { /** * ID of the topic to be deleted. */ string id = 1; } /** * Topic deletion response */ message DeleteTopicResponse { } /** * The request for (partially) updating an Topic. */ message UpdateTopicRequest { /** * Topic being updated */ Topic topic = 1; /** * Mask of fields being updated in this Topic to make partial changes. */ google.protobuf.FieldMask update_mask = 2; /** * IDs of users to be added to this topic. */ repeated string add_users = 3; /** * IDs of users to be removed from this topic. */ repeated string remove_users = 4; } /** * The request for (partially) updating an Topic. */ message UpdateTopicResponse { /** * Topic being updated */ Topic topic = 1; } Instead of having "empty" method definitions (e.g., rpc MethodName(ReqType) returns (RespType) {}), we are now seeing "annotations" being added inside methods. Any number of annotations can be added and each annotation is parsed by the protoc and passed to all the plugins invoked by it. There are tons of annotations that can be passed and this has a "bit of everything" in it. Back to the HTTP bindings: Typically an HTTP annotation has a method, a URL path (with bindings within { and }), and a marking to indicate what the body parameter maps to (for PUT and POST methods). For example, in the CreateTopic method, the method is a POST request to "v1/topic " with the body (*) corresponding to the JSON representation of the CreateTopicRequest message type; i.e., our request is expected to look like this: { "Topic": {... topic object...} } Naturally, the response object of this would be the JSON representation of the CreateTopicResponse message. The other examples in the topic service, as well as in the other services, are reasonably intuitive. Feel free to read through it to get any finer details. Before we are off to the next section implementing the proxy, we need to regenerate the pb.gw.go files to incorporate these new bindings: make all We will now see the following error: google/api/annotations.proto: File not found. topics.proto:8:1: Import "google/api/annotations.proto" was not found or had errors. Unfortunately, there is no "package manager" for protos at present. This void is being filled by an amazing tool: Buf.build (which will be the main topic in Part 3 of this series). In the meantime, we will resolve this by manually copying (shudder) http.proto and annotations.proto manually. So, our protos folder will have the following structure: protos ├── google │ └── api │ ├── annotations.proto │ └── http.proto └── onehub └── v1 └── topics.proto └── messages.proto └── ... However, we will follow a slightly different structure. Instead of copying files to the protos folder, we will create a vendors folder at the root and symlink to it from the protos folder (this symlinking will be taken care of by our Makefile). Our new folder structure is: onehub ├── Makefile ├── ... ├── vendors │ ├── google │ │ └── api │ │ ├── annotations.proto │ │ └── http.proto ├── proto └── google -> onehub/vendors/google └── onehub └── v1 └── topics.proto └── messages.proto └── ... Our updated Makefile is shown below. Makefile for HTTP bindings: # Some vars to detemrine go locations etc GOROOT=$(which go) GOPATH=$(HOME)/go GOBIN=$(GOPATH)/bin # Evaluates the abs path of the directory where this Makefile resides SRC_DIR:=$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST)))) # Where the protos exist PROTO_DIR:=$(SRC_DIR)/protos # where we want to generate server stubs, clients etc OUT_DIR:=$(SRC_DIR)/gen/go all: createdirs printenv goprotos gwprotos openapiv2 cleanvendors goprotos: echo "Generating GO bindings" protoc --go_out=$(OUT_DIR) --go_opt=paths=source_relative \ --go-grpc_out=$(OUT_DIR) --go-grpc_opt=paths=source_relative \ --proto_path=$(PROTO_DIR) \ $(PROTO_DIR)/onehub/v1/*.proto gwprotos: echo "Generating gRPC Gateway bindings and OpenAPI spec" protoc -I . --grpc-gateway_out $(OUT_DIR) \ --grpc-gateway_opt logtostderr=true \ --grpc-gateway_opt paths=source_relative \ --grpc-gateway_opt generate_unbound_methods=true \ --proto_path=$(PROTO_DIR) \ $(PROTO_DIR)/onehub/v1/*.proto openapiv2: echo "Generating OpenAPI specs" protoc -I . --openapiv2_out $(SRC_DIR)/gen/openapiv2 \ --openapiv2_opt logtostderr=true \ --openapiv2_opt generate_unbound_methods=true \ --openapiv2_opt allow_merge=true \ --openapiv2_opt merge_file_name=allservices \ --proto_path=$(PROTO_DIR) \ $(PROTO_DIR)/onehub/v1/*.proto printenv: @echo MAKEFILE_LIST=$(MAKEFILE_LIST) @echo SRC_DIR=$(SRC_DIR) @echo PROTO_DIR=$(PROTO_DIR) @echo OUT_DIR=$(OUT_DIR) @echo GOROOT=$(GOROOT) @echo GOPATH=$(GOPATH) @echo GOBIN=$(GOBIN) createdirs: rm -Rf $(OUT_DIR) mkdir -p $(OUT_DIR) mkdir -p $(SRC_DIR)/gen/openapiv2 cd $(PROTO_DIR) && ( \ if [ ! -d google ]; then ln -s $(SRC_DIR)/vendors/google . ; fi \ ) cleanvendors: rm -f $(PROTO_DIR)/google Now running Make should be error-free and result in the updated bindings in the .pb.gw.go files. Implementing the HTTP Gateway Proxy Lo and behold, we now have a "proxy" (in the .pw.gw.go files) that translates HTTP requests and converts them into gRPC requests. On the return path, gRPC responses are also translated to HTTP responses. What is now needed is a service that runs an HTTP server that continuously facilitates this translation. We have now added a startGatewayService method in cmd/server.go that now also starts an HTTP server to do all this back-and-forth translation: import ( ... // previous imports // new imports "context" "net/http" "github.com/grpc-ecosystem/grpc-gateway/v2/runtime" ) func startGatewayServer(grpc_addr string, gw_addr string) { ctx := context.Background() mux := runtime.NewServeMux() opts := []grpc.DialOption{grpc.WithInsecure()} // Register each server with the mux here if err := v1.RegisterTopicServiceHandlerFromEndpoint(ctx, mux, grpc_addr, opts); err != nil { log.Fatal(err) } if err := v1.RegisterMessageServiceHandlerFromEndpoint(ctx, mux, grpc_addr, opts); err != nil { log.Fatal(err) } http.ListenAndServe(gw_addr, mux) } func main() { flag.Parse() go startGRPCServer(*addr) startGatewayServer(*gw_addr, *addr) } In this implementation, we created a new runtime.ServeMux and registered each of our gRPC services' handlers using the v1.Register<ServiceName>HandlerFromEndpoint method. This method associates all of the URLs found in the <ServiceName> service's protos to this particular mux. Note how all these handlers are associated with the port on which the gRPC service is already running (port 9000 by default). Finally, the HTTP server is started on its own port (8080 by default). You might be wondering why we are using the NewServeMux in the github.com/grpc-ecosystem/grpc-gateway/v2/runtime module and not the version in the standard library's net/http module. This is because the grpc-gateway/v2/runtime module's ServeMux is customized to act specifically as a router for the underlying gRPC services it is fronting. It also accepts a list of ServeMuxOption (ServeMux handler) methods that act as a middleware for intercepting an HTTP call that is in the process of being converted to a gRPC message sent to the underlying gRPC service. These middleware can be used to set extra metadata needed by the gRPC service in a common way transparently. We will see more about this in a future post about gRPC interceptors in this demo service. Generating OpenAPI Specs Several API consumers seek OpenAPI specs that describe RESTful endpoints (methods, verbs, body payloads, etc). We can generate an OpenAPI spec file (previously Swagger files) that contains information about our service methods along with their HTTP bindings. Add another Makefile target: openapiv2: echo "Generating OpenAPI specs" protoc -I . --openapiv2_out $(SRC_DIR)/gen/openapiv2 \ --openapiv2_opt logtostderr=true \ --openapiv2_opt generate_unbound_methods=true \ --openapiv2_opt allow_merge=true \ --openapiv2_opt merge_file_name=allservices \ --proto_path=$(PROTO_DIR) \ $(PROTO_DIR)/onehub/v1/*.proto Like all other plugins, the openapiv2 plugin also generates one .swagger.json per .proto file. However, this changes the semantics of Swagger as each Swagger is treated as its own "endpoint." Whereas, in our case, what we really want is a single endpoint that fronts all the services. In order to contain a single "merged" Swagger file, we pass the allow_merge=true parameter to the above command. In addition, we also pass the name of the file to be generated (merge_file_name=allservices). This results in gen/openapiv2/allservices.swagger.json file that can be read, visualized, and tested with SwaggerUI. Start this new server, and you should see something like this: % onehub % go run cmd/server.go Starting grpc endpoint on :9000: Starting grpc gateway server on: :8080 The additional HTTP gateway is now running on port 8080, which we will query next. Testing It All Out Now, instead of making grpc_cli calls, we can issue HTTP calls via the ubiquitous curl command (also make sure you install jq for pretty printing your JSON output): Create a Topic % curl -s -d '{"topic": {"name": "First Topic", "creator_id": "user1"}' localhost:8080/v1/topics | jq { "topic": { "createdAt": "2023-07-07T20:53:31.629771Z", "updatedAt": "2023-07-07T20:53:31.629771Z", "id": "1", "creatorId": "user1", "name": "First Topic", "users": [] } } And another: % curl -s localhost:8080/v1/topics -d '{"topic": {"name": "Urgent topic", "creator_id": "user2", "users": ["user1", "user2", "user3"]}' | jq { "topic": { "createdAt": "2023-07-07T20:56:52.567691Z", "updatedAt": "2023-07-07T20:56:52.567691Z", "id": "2", "creatorId": "user2", "name": "Urgent topic", "users": [ "user1", "user2", "user3" ] } } List All Topics % curl -s localhost:8080/v1/topics | jq { "topics": [ { "createdAt": "2023-07-07T20:53:31.629771Z", "updatedAt": "2023-07-07T20:53:31.629771Z", "id": "1", "creatorId": "user1", "name": "First Topic", "users": [] }, { "createdAt": "2023-07-07T20:56:52.567691Z", "updatedAt": "2023-07-07T20:56:52.567691Z", "id": "2", "creatorId": "user2", "name": "Urgent topic", "users": [ "user1", "user2", "user3" ] } ], "nextPageKey": "" } Get Topics by IDs Here, "list" values (e.g., ids) are possibly by repeating them as query parameters: % curl -s "localhost:8080/v1/topics?ids=1&ids=2" | jq { "topics": [ { "createdAt": "2023-07-07T20:53:31.629771Z", "updatedAt": "2023-07-07T20:53:31.629771Z", "id": "1", "creatorId": "user1", "name": "First Topic", "users": [] }, { "createdAt": "2023-07-07T20:56:52.567691Z", "updatedAt": "2023-07-07T20:56:52.567691Z", "id": "2", "creatorId": "user2", "name": "Urgent topic", "users": [ "user1", "user2", "user3" ] } ], "nextPageKey": "" } Delete a Topic Followed by a Listing % curl -sX DELETE "localhost:8080/v1/topics/1" | jq {} % curl -s "localhost:8080/v1/topics" | jq { "topics": [ { "createdAt": "2023-07-07T20:56:52.567691Z", "updatedAt": "2023-07-07T20:56:52.567691Z", "id": "2", "creatorId": "user2", "name": "Urgent topic", "users": [ "user1", "user2", "user3" ] } ], "nextPageKey": "" } Best Practices Separation of Gateway and gRPC Endpoints In our example, we served the Gateway and gRPC services on their own addresses. Instead, we could have directly invoked the gRPC service methods, i.e., by directly creating NewTopicService(nil) and invoking methods on those. However, running these two services separately meant we could have other (internal) services directly access the gRPC service instead of going through the Gateway. This separation of concerns also meant these two services could be deployed separately (when on different hosts) instead of needing a full upgrade of the entire stack. HTTPS Instead of HTTP However in this example, the startGatewayServer method started an HTTP server, it is highly recommended to have the gateway over an HTTP server for security, preventing man-in-the-middle attacks, and protecting clients' data. Use of Authentication This example did not have any authentication built in. However, authentication (authn) and authorization (authz) are very important pillars of any service. The Gateway (and the gRPC service) are no exceptions to this. The use of middleware to handle authn and authz is critical to the gateway. Authentication can be applied with several mechanisms like OAuth2 and JWT to verify users before passing a request to the gRPC service. Alternatively, the tokens could be passed as metadata to the gRPC service, which can perform the validation before processing the request. The use of middleware in the Gateway (and interceptors in the gRPC service) will be shown in Part 4 of this series. Caching for Improved Performance Caching improves performance by avoiding database (or heavy) lookups of data that may be frequently accessed (and/or not often modified). The Gateway server can also employ cache responses from the gRPC service (with possible expiration timeouts) to reduce the load on the gRPC server and improve response times for clients. Note: Just like authentication, caching can also be performed at the gRPC server. However, this would not prevent excess calls that may otherwise have been prevented by the gateway service. Using Load Balancers While also applicable to gRPC servers, HTTP load balancers (in front of the Gateway) enable sharding to improve the scalability and reliability of our services, especially during high-traffic periods. Conclusion By adding a gRPC Gateway to your gRPC services and applying best practices, your services can now be exposed to clients using different platforms and protocols. Adhering to best practices also ensures reliability, security, and high performance. In this article, we have: Seen the benefits of wrapping our services with a Gateway service Added HTTP bindings to an existing set of services Learned the best practices for enacting Gateway services over your gRPC services In the next post, we will take a small detour and introduce a modern tool for managing gRPC plugins and making it easy to work with them.
In the vibrant world of microservices and event-driven architectures, messaging queues have emerged as a critical component, enabling asynchronous communication, decoupling systems, and unlocking scalability. However, with a vibrant landscape of options, choosing the optimal queue can feel like navigating a complex maze. This guide empowers you to confidently make informed decisions by delving into key factors, popular choices, and tailored considerations for diverse use cases. What Are Message Queues? Imagine a message queue as a central communication hub within your application. Producers (applications or services) deposit messages containing data or tasks, while consumers (other applications or services) retrieve them for processing. This asynchronous communication paradigm fosters the following: Loose coupling: Components only need to interact with the queue, not directly with each other. Scalability: Queues can easily scale horizontally to handle increasing workloads. Fault tolerance: Messages persist in the queue even if producers or consumers experience temporary outages. Fundamental Factors To Ponder Understanding common messaging patterns is crucial for selecting the right queue: Messaging Patterns Identify your communication pattern. Point-to-point: Messages flow from a producer to a designated consumer (task processing, log integration). Apache RocketMQ, RabbitMQ, and ActiveMQ are suitable contenders. Publish-subscribe: Producers disseminate messages to interested consumers (notifications, real-time feeds). Explore Apache Kafka, NATS, or Pulsar. Fan-out: Producers broadcast messages to all subscribers (system alerts, data backups). RabbitMQ, ActiveMQ, and Artemis are strong options. Performance Requirements Evaluate your throughput, latency, and scalability needs. Throughput: Messages processed per second (high-frequency trading, IoT data ingestion). Apache Kafka, Pulsar, and Redpanda showcase high-performance capabilities. Latency: Time taken for messages to reach consumers (real-time systems, chat applications). Consider NATS, RabbitMQ, or Artemis for low-latency performance. Scalability: Ability to handle growing message volumes (microservices architectures, big data pipelines). Apache Kafka, Pulsar, and Azure Service Bus excel in scaling with demand. Reliability Guarantees Assess the importance of guaranteed message delivery and order in your application. Delivery: Must messages always reach consumers? (mission-critical systems, financial transactions). Apache Kafka, RabbitMQ, and ActiveMQ prioritize reliable delivery. Order: Does message processing require strict order adherence? (stream processing, logs). Apache Kafka, Artemis, and ActiveMQ provide strong ordering guarantees. Deployment Environment Choose between on-premises solutions for internal control or cloud-managed options for easier integration and scalability. On-premises: Managed within your infrastructure (internal applications, sensitive data). RabbitMQ, ActiveMQ, and Artemis are popular on-premises choices. Cloud: Leveraged as a managed service (microservices deployments, data lake processing). Amazon SQS, Azure Service Bus, and Google Cloud Pub/Sub offer comprehensive cloud management. Development Ease Prioritize queues with familiar programming languages and libraries (Python, Java) for seamless integration into your development workflow. Programming languages: Widely supported languages and libraries (Java, Python, Go). RabbitMQ, Apache Kafka, and NATS provide extensive language support. Monitoring and tooling: Evaluate available tools for debugging, performance analysis, and operational visibility. Kafka, RabbitMQ, and ActiveMQ offer robust monitoring capabilities but compare dashboards and metrics offered by different queues. Popular Messaging Queues: Where They Shine Apache Kafka Boasts high throughput, scalability, distributed architecture, multi-tenancy, durability, and support for various message patterns. ideal for high-throughput use cases (order processing, event-driven microservices). Consider for: Scalability: Handles massive data volumes efficiently. Durability: Guarantees message persistence and delivery. RabbitMQ Renowned for its lightweight design, ease of use, flexibility, and suitability for small to medium-sized projects (task processing, log aggregation). Consider for: Ease of use: Simple installation and configuration. Flexibility: Supports various messaging patterns and integrations. Apache RocketMQ An emerging contender that excels in high-throughput, low-latency scenarios, it is designed for Alibaba's massive scale, making it ideal for demanding situations. Consider for: High performance: Handles millions of messages per second, making it suitable for demanding scenarios. Low latency: Achieves sub-millisecond message delivery for real-time applications. Scalability: Scales horizontally to meet growing workloads seamlessly. Reliability: Offers guaranteed message delivery and message order. Flexibility: Supports diverse messaging patterns and message types. Cloud-Native: Integrates well with cloud environments like Kubernetes. Apache Pulsar This cloud-native, distributed offering supports multiple message patterns and features persistent storage. Suitable for scalable architectures (microservices communication, real-time chat). Consider for: Cloud integration: Seamless integration with cloud platforms. Multi-tenancy: Supports multiple applications on a single cluster. Amazon SQS Provides a fully managed, reliable, and cost-effective solution, seamlessly integrating with other AWS services (web applications, data pipelines). Consider for: Managed service: Reduced operational overhead. Cost-effectiveness: Pay-per-use pricing model. Azure Service Bus A managed option supporting multiple patterns and offering tight integration with the Azure ecosystem (cloud-based microservices, event-driven systems). Consider for: Azure integration: Tight coupling with Azure services. Managed service: Easy deployment and management. Google Cloud Pub/Sub This fully managed offering delivers high throughput, low latency, and on-demand scaling and integrates with Google Cloud (real-time applications, data ingestion). Consider for: Scalability: Scales on-demand to meet changing workloads. Real-time applications: Low latency for time-sensitive messaging. Real-World Examples E-commerce platform: Kafka for order processing, RabbitMQ for product recommendations. IoT data ingestion: Pulsar for scalable data collection, SQS for cost-effective storage. Real-time chat application: NATS for low-latency messaging, Pub/Sub for global reach. Log aggregation: Kafka for high-throughput log ingestion, ActiveMQ for flexible routing. Making the Decision With a clear understanding of your needs and the available options, the selection process becomes more targeted. Here are some final tips: Prioritize requirements: Identify the must-have features and prioritize them based on your unique use case. Evaluate trade-offs: No single queue excels in all aspects. Weigh the strengths and weaknesses of each option against your requirements. Proof of concept (POC): Consider conducting a POC with shortlisted candidates to experience them firsthand and make an informed decision. Community and support: A healthy community and readily available support resources can be invaluable for troubleshooting and learning. Cost: Consider open source versus commercial offerings, licensing fees, and cloud provider costs. Security: Prioritize authentication, authorization, encryption, and access control mechanisms. Additional Resources Apache Kafka RabbitMQ Apache RocketMQ Apache Pulsar Amazon SQS Azure Service Bus Google Cloud Pub/Sub Conclusion Selecting the right messaging queue is not a one-size-fits-all approach. By thoughtfully evaluating these factors and aligning them with your specific project requirements, you can confidently navigate the messaging queue landscape and select the solution that empowers your applications to flourish through efficient, reliable, and scalable communication. Remember to factor in your organizational experience, ensuring compatibility with existing systems and leveraging past expertise. Align your choice with your future roadmap to support long-term architecture and scalability needs.
As a software developer, tightly coupled, monolithic applications can make you feel bogged down. Enter Event-Driven Architecture (EDA), a promising addition to the world of software development. This paradigm is all about events: changes in your system that trigger actions in other parts, leading to reactive, loosely coupled, and highly responsive systems. Sound intriguing? Let's dive in and see how EDA can empower your development journey. How Does Software Based on Event-Driven Architecture Work? Imagine a user placing an order on your e-commerce website. In EDA terms, this is an event, a significant change that triggers a chain reaction. The order creation event gets published, and interested parties subscribe and react accordingly. The inventory system updates stock, the payment processor charges the customer, and the shipping module prepares for delivery. Each service reacts independently, based on the event it's interested in, creating a loosely coupled ecosystem. What Are the Benefits of Using Event-Driven Architecture for Software Developers? This event-centric approach comes with a bunch of perks for developers: Scalability on Demand Need to handle peak traffic? No problem! EDA scales horizontally by adding more event consumers. This system gets rid of monolithic bottlenecks. Built-In Resilience Events are like mini transactions, allowing for fault tolerance and easy recovery. A failed service won't derail the entire system. Improved Flexibility EDA adapts easily. Thanks to the loose coupling, developers can add new services without affecting existing ones. Real-Time Reactivity Want instant responses? EDA enables event-driven microservices that react to changes in real-time, perfect for building responsive systems. Where Do You See EDA in Action? The possibilities are endless when it comes to what can be defined as an event. Some of the common examples of events that people normally create every day include: When a new user signs up on a website to create an account Subscribing to a YouTube channel is also an event. E-commerce order processing Real-time analytics in IoT Systems Chat applications constantly update messages What Are the Components of EDA? Broadly speaking, there are four components to Event-Driven Architecture, which are listed below with a brief description: Event: The user action causes a state change. Service or event handler: The event causes the services or event handler to react appropriately. It can include a process or further event generation. Event loop: The event loop is responsible for facilitating a smooth flow of interactions between the events and services. Event flow layers: There are three event flow layers, namely, event producer, event consumer, and event channel/router. What Are Some of the Challenges and Considerations When Using Event-Driven Architecture? No silver bullet exists, and EDA comes with its own set of challenges. Debugging distributed systems can be trickier, and designing complex event flows requires careful planning. But fear not; with the right tools and knowledge, these challenges are manageable. Start Your EDA Journey Ready to explore the world of events? Dive into resources like the Apache Kafka documentation or try out frameworks like Spring Cloud Stream. Start with small projects to get comfortable, and soon, you'll be building powerful, reactive systems like a pro! Remember: EDA is a paradigm shift, not a replacement. Consider your project's specific needs and carefully evaluate the trade-offs before diving in. The Future Is Event-Driven EDA is more than just a trend; it is a powerful approach shaping the future of software development. With its flexibility, scalability, and real-time capabilities, EDA empowers developers to build robust and responsive systems that can adapt to the ever-changing demands of the digital world. So, what are you waiting for? Embrace the event-driven revolution and unleash the power of reactive systems! The emerging trends like serverless computing and event sourcing will further enhance the power of EDA. Therefore, developers who want to stay up-to-date and offer better services must consider adding this to their arsenal of skill sets.
Take a transformative journey into the realm of system design with our tutorial, tailored for software engineers aspiring to architect solutions that seamlessly scale to serve millions of users. In this exploration, we use the fictitious MarsExpress, a local delivery startup in Albuquerque, as a model to illustrate the evolution from a community service to a global force. MarsExpress, our focal point, currently operates on an aging monolithic system — a legacy structure that once served its purpose locally but now yearns for a comprehensive overhaul. Join us as we unravel the intricacies of system design, not merely for theoretical understanding but as a hands-on approach to rejuvenating a legacy system into a globally scalable software solution. This tutorial transcends conventional coding tutorials, shifting the focus towards the strategic decisions and methodologies that propel a software solution from a local operation to a worldwide phenomenon. Forget geographical limitations and delve into the world of strategic architectural choices that dictate scalability. MarsExpress, with its legacy monolith, serves as our canvas, unraveling the intricacies of system design without delving into code specifics. Instead, we provide software engineers with the tools to engineer solutions that scale effortlessly and globally. Join us on this odyssey, where MarsExpress becomes a model for the transformation from localized operations to a globally impactful service. This tutorial is an open invitation for software engineers keen on mastering system design — an essential skill set for crafting software solutions that cater to millions. Legacy Monolithic MarsExpress currently operates on a traditional monolithic architecture, often referred to as the “legacy system.” In this setup, the server is openly accessible via the Internet, offering a RESTful API to manage essential business logic. This API serves as the bridge for communication between mobile and web client applications and the server, enabling the exchange of data and commands. One notable aspect of the current system is its responsibility for delivering static content, such as images and application bundles. These elements are stored directly on the local disk of the server, contributing to the overall functionality and user experience. Additionally, the application server is closely linked to the database, which is housed on the same server. This connection facilitates the seamless interaction between the application’s logic and the data it relies on, creating a centralized environment for information storage and retrieval. Scaling Vertically In our quest to transform MarsExpress into a global powerhouse, the first step involves scaling vertically to bolster its capacity and handle a substantial increase in user demand. Scaling vertically, often referred to as “scaling up,” focuses on enhancing the capabilities of the existing server and infrastructure to manage a higher load of users. As the server grapples with an increasing user load, a short-term remedy involves upgrading to a larger server equipped with enhanced CPU, memory, and disk space. However, it’s important to note that this serves as a temporary solution, and over time, even the most robust server will encounter its capacity constraints. Also, the lack of redundancy in a single-server architecture makes the system vulnerable to failures. In the event of hardware issues or routine maintenance, downtime becomes an unavoidable consequence, rendering the entire system temporarily inaccessible. Performance bottlenecks also emerge as a noteworthy issue. With a burgeoning user base and expanding data volume, a single server can become a significant performance bottleneck. This bottleneck manifests as slower response times, adversely affecting the overall user experience. Geographic limitations pose another challenge. A single server, typically located in a specific geographic region, can result in latency for users situated in distant locations. This constraint becomes increasingly pronounced when aspiring to cater to a global user base. The concentration of data on a single server also raises concerns about data loss. In the unfortunate event of a catastrophic failure or unexpected circumstances, the risk of losing significant data becomes a stark reality. Additionally, the maintenance and upgrade processes on a single server can be cumbersome. Implementing updates or performing maintenance tasks often requires system downtime, impacting users’ access to services and making the overall system management less flexible. In light of these drawbacks, it becomes imperative to explore more robust and scalable system design approaches, especially when aiming to create a production environment capable of handling millions of users while ensuring reliability and optimal performance. Scaling Horizontally In a single-server architecture, horizontal scaling emerges as a strategic solution to accommodate increasing demands and ensure the system’s ability to handle a burgeoning user base. Horizontal scaling involves adding more servers to the system and distributing the workload across multiple machines. Unlike vertical scaling, which involves enhancing the capabilities of a single server, horizontal scaling focuses on expanding the server infrastructure horizontally. One of the key advantages of horizontal scaling is its potential to improve system performance and responsiveness. By distributing the workload across multiple servers, the overall processing capacity increases, alleviating performance bottlenecks and enhancing the user experience. Moreover, horizontal scaling offers improved fault tolerance and reliability. The redundancy introduced by multiple servers reduces the risk of a single point of failure. In the event of hardware issues or maintenance requirements, traffic can be seamlessly redirected to other available servers, minimizing downtime and ensuring continuous service availability. Scalability becomes more flexible with horizontal scaling. As user traffic fluctuates, additional servers can be provisioned or scaled down dynamically to match the demand. This elasticity ensures efficient resource utilization and cost-effectiveness, as resources are allocated based on real-time requirements. Load Balancer In the realm of horizontal scaling, a load balancer becomes our strategic ally. It acts as a guardian at the gateway, diligently directing incoming requests to the array of servers in our cluster. Here’s how it seamlessly integrates into our horizontally scaled architecture. A load balancer ensures that incoming requests are evenly distributed across all available servers. This prevents any single server from bearing the brunt of heavy traffic, promoting optimal resource utilization. the effectiveness of a load balancer often depends on the strategy and algorithm it uses to distribute incoming requests among servers. In load-balancing strategies, algorithms are categorized into two primary types: static and dynamic. These classifications represent distinct approaches to distributing incoming network traffic across multiple servers. Each type serves specific purposes and is tailored to meet particular requirements in system design. Static load balancing algorithms, in contrast, follow predetermined patterns for distributing incoming requests among available servers. While they offer simplicity and ease of implementation, like: Round Robin: Round Robin is a static load balancing algorithm that distributes incoming requests in a circular order among available servers. This method is straightforward, ensuring an even distribution of traffic without considering the current load or capacity of each server. It is well-suited for environments with relatively uniform servers and stable workloads. Weighted Round Robin: Weighted Round Robin is a static load balancing approach similar to Round Robin but introduces the concept of assigning weights to each server based on its capacity or performance. This static method allows administrators to predetermine the load distribution, considering variations in server capacities. IP Hash: IP Hash is a static load-balancing algorithm that utilizes a hash function based on the client’s IP address to determine the server for each request. This ensures session persistence, directing requests from the same client to the same server. While effective for maintaining stateful connections, it may lead to uneven distribution if the IP hashing isn’t well-distributed. Randomized: The Randomized load balancing algorithm introduces an element of unpredictability by randomly selecting a server for each request. This static method can be advantageous in scenarios where a uniform distribution of requests is not critical, adding an element of randomness to the load distribution. Dynamic load balancing algorithms adapt in real time to the changing conditions of a system. These algorithms continuously assess server health, current loads, and response times, adjusting the distribution of incoming requests accordingly. Like: Least Connections: Dynamic in nature, the Least Connections algorithm routes incoming traffic to the server with the fewest active connections. This dynamic approach adapts to real-time connection loads on servers, efficiently distributing requests and optimizing resource usage based on the current server states. Least Response Time: The Least Response Time algorithm dynamically directs traffic to the server with the fastest response time. It optimizes server performance and responsiveness, ensuring that users are consistently directed to the server with the lowest latency. Least Resource Utilization: Dynamic in its behavior, the Least Resource Utilization algorithm routes traffic to the server with the lowest resource utilization, considering factors such as CPU and memory usage. This dynamic approach responds to changes in server resource usage, optimizing for efficiency. Adaptive Load Balancing: Adaptive Load Balancing dynamically adjusts the distribution algorithm based on real-time server health and load conditions. This dynamic approach continuously adapts to changing circumstances, offering optimal performance by responding to fluctuations in server states. The dynamic load balancing algorithm I would recommend for “MarsExpress,” as it scales globally, is the Least Connections algorithm. This algorithm is particularly advantageous because it considers the current state of the network when making routing decisions and assigning new requests to the server with the fewest active connections. One of the primary benefits of the Least Connections method is its ability to adapt to traffic variations. As the system expands, it is likely to experience unpredictable spikes in usage. The algorithm can manage these fluctuations by distributing incoming requests to servers with lighter loads, thus preventing any single server from being overwhelmed. This algorithm also supports session persistence, which is critical for a delivery system that requires transaction consistency. Modified to consider session affinity, the Least Connections method ensures that requests from a specific user during a session are consistently directed to the same server, maintaining a continuous user experience. Data Replication Data replication is a fundamental aspect of large-scale distributed systems, playing a critical role in enhancing their efficiency, reliability, and availability. In such systems, data replication involves creating multiple copies of data and distributing them across different servers or locations. This strategy is vital for ensuring high availability; if one node fails or becomes inaccessible, users can still access data from another node, minimizing downtime and service disruptions. Additionally, replication aids in load balancing by allowing requests to be distributed across multiple nodes, thereby reducing the load on any single server and improving overall system performance. It also enhances data access speed, as users can access data from the nearest or least busy replica, significantly reducing latency. While data replication is prominently recognized for its role in distributed databases, it is important to note that its utility extends beyond traditional data storage systems. Replication can be equally vital in caching layers, such as those implemented using cache servers like Redis. Selecting the appropriate replication strategy for a system can be a complex decision, as various strategies offer distinct advantages and challenges. The suitability of a replication strategy largely depends on the specific needs and contexts of the use case at hand. Some strategies may excel in certain scenarios, while others might be more effective under different circumstances. three main replication strategies are: Leader-Follower (also known as Master-Slave) Replication: In the Leader-Follower replication strategy, one node (the leader or master) handles all write operations, and several other nodes (followers or slaves) replicate these changes. The leader node receives all update requests, processes them, and then propagates the changes to its followers. This method ensures consistency and simplifies conflict resolution, as there is a single authoritative source for data updates. Multi-Leader Replication: Multi-Leader replication allows multiple nodes to act as leaders, each capable of handling write operations. These leaders then synchronize their data with each other. This approach is beneficial in distributed systems where nodes are geographically dispersed, as it allows writes to occur closer to where the data is being used, reducing latency. Leaderless Replication: In the Leaderless replication model, all nodes are treated equally and can handle both read and write operations. When a write occurs, it is usually written to multiple nodes to ensure redundancy. Reads may require responses from multiple nodes to ensure the data is up-to-date, based on a quorum-like system. This model offers high availability and fault tolerance, as there is no single point of failure, and operations can continue even if some nodes are down. Also, conflict resolution is a critical aspect of database replication to ensure data consistency and integrity. Various conflict resolution strategies can be employed, including: Last Write Wins (LWW): This strategy resolves conflicts by accepting the last write operation as the correct version, potentially discarding earlier conflicting changes. Timestamp-Based Resolution: Conflicts are resolved based on timestamps associated with write operations, with the latest timestamp taking precedence. Manual Resolution: In some cases, conflicts require manual intervention by administrators or users to determine the correct version of the data. Conflict Avoidance: Designing data models and application logic to minimize the likelihood of conflicts through techniques like logical clocks and unique identifiers. Consistency Matters Consistency in distributed systems is crucial as it ensures that all users and processes have a uniform view of data at any given time. This is vital for maintaining data integrity and preventing conflicts or errors that can arise from disparate data states. In scenarios like financial transactions, inventory management, or any system where data accuracy is paramount, consistency ensures reliable and predictable interactions. Without consistency, systems can produce incorrect results, leading to potential data loss, erroneous decisions, or system failures. In summary, consistency is key to the reliable and accurate operation of distributed systems. Eventual consistency is a model used in distributed systems, where it’s accepted that all copies of data across nodes may not be immediately consistent following a change but will become consistent after a period. This approach allows for high system availability and performance, especially in environments with network latency and partitioning issues. Eventual consistency is well-suited for applications where immediate data consistency is not critical, and slight delays in data synchronization can be tolerated. This model is often used in large-scale, distributed databases and applications like social networks, where scalability and availability are prioritized over strict consistency. The linearizability consistency model in distributed systems ensures that operations appear to occur instantaneously and sequentially, even if they are executed concurrently. This model provides a high level of data integrity and consistency, making it ideal for systems where precise coordination of operations is critical. Linearizability is akin to having a single, global clock dictating the order of all operations, thus simplifying the understanding and predictability of system behavior. It’s most beneficial in scenarios requiring strict consistency, like financial transactions or critical data updates, where the exact ordering of operations is vital. The sequential consistency model in distributed systems is a consistency level where the result of execution of operations (like read and write) is as if the operations were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program. It ensures system-wide ordering of operations, making it simpler to understand than more relaxed consistency models. This model is useful for applications where the order of operations needs to reflect the program order, but it does not require operations to be instantaneous like Linearizability. For example, consider an online marketplace with a shared inventory system. When multiple sellers update their inventory concurrently, the sequential consistency model ensures that these updates are reflected across the system in the order they were made. The causal consistency model in distributed systems ensures that operations that are causally related maintain a specific order across all nodes, while operations that are unrelated can be seen in any order. This model is crucial in scenarios like social media platforms, where user actions such as posting a message and commenting on that message are causally linked. The causal consistency ensures that if a user sees comments only after the original message is visible. In such a system, if one user comments on another user’s post, the comment will not appear to others until the original post is also visible, maintaining a logical and understandable sequence of events. This consistency model is ideal for applications where the context and sequence of interactions are important for user experience and data integrity. Stronger consistency models impose order constraints that limit the utility of asynchronous replication, while weaker models offer more flexibility at the risk of stale data. This understanding is crucial for selecting the right replication approach. The trade-off with asynchronous replication’s flexibility is the potential lag between leader and follower nodes, which can introduce several consistency challenges that need careful consideration. Consistency in distributed systems can be fraught with challenges. One of them is read-your-own-write inconsistency. It is a phenomenon encountered in distributed systems where data written or updated by a user is not instantly visible to that same user upon subsequent reads. In other words, when a user makes a change to their data, such as updating their profile information or posting a message, they expect to immediately see the updated data when they access it again. However, in distributed systems, there can be delays in propagating these changes across all nodes or replicas, leading to a situation where the user’s own updates are not immediately reflected in their own subsequent read requests. This inconsistency can result from factors like replication lag, network delays, or the inherent complexities of maintaining real-time data synchronization across distributed nodes. It’s a challenge in systems where maintaining immediate consistency can be difficult due to the need to balance performance, availability, and data accuracy. To address this issue, distributed systems often employ various synchronization mechanisms and strategies to minimize the delay and provide users with a more coherent and real-time experience. it becomes evident that discussing inconsistency and consistency requires dedicated articles that explore their nuances, challenges, and solutions. Conclusion In this initial segment of our series, we’ve embarked on MarsExpress’s journey, transforming its legacy monolithic architecture into a scalable structure ready for global challenges. We’ve explored the fundamentals of vertical and horizontal scaling, along with load balancing and data replication, setting the stage for more complex scalability solutions. As we look ahead, the next part of our series will delve into the realms of caching and sharding, which are crucial for enhancing performance and managing data efficiently on a global scale. These advanced techniques will be pivotal in propelling MarsExpress to new heights, ensuring it can handle the demands of millions seamlessly. Join us as we continue to unravel the intricacies of system design, which is essential for any software engineer aiming to build robust, scalable systems in today’s dynamic technological landscape.
Amol Gote
Solution Architect,
Innova Solutions (Client - iCreditWorks Start Up)
Ray Elenteny
Solution Architect,
SOLTECH
Nicolas Duminil
Silver Software Architect,
Simplex Software
Satrajit Basu
Chief Architect,
TCG Digital