Design: Real-Time Chat Explained
ADVANCED › System Design
Start a chat system design by splitting the problem in two: is the app in the foreground or not. While the chat screen is open and visible, you want the lowest possible latency in both directions, the user sends, and the server or other participants can push back into that same connection without the client asking again. A persistent WebSocket gives you exactly that: one long-lived, full-duplex TCP connection, opened once, that both sides can write to at any time. Compare the alternatives. HTTP polling means opening a new connection every second whether or not anything happened, which burns battery and radio time for mostly empty responses. Server-Sent Events are server to client only, so the client still needs a second channel to send messages, which defeats the point. FCM is not built for this either, it is a best-effort, throttled delivery system meant for waking an app, not a low-latency duplex pipe for a live conversation. So: while foregrounded, WebSocket is the transport, everything else either costs more or cannot carry traffic in both directions.
val client = OkHttpClient()
val request = Request.Builder().url("wss://chat.example.com/ws").build()
val ws = client.newWebSocket(request, object : WebSocketListener() {
override fun onMessage(webSocket: WebSocket, text: String) {
handleMessage(text)
}
override fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {
scheduleReconnect()
}
})
A persistent connection only works while the OS is willing to keep it open, and Android's Doze mode and App Standby are explicitly designed to suspend background network activity to save battery once the app isn't in the foreground for a while. That kills the WebSocket. The tempting fix, a foreground service that holds the socket open indefinitely, is the wrong trade: it's a persistent notification, a battery drain, and it's not what foreground services are meant for. The sanctioned path for reaching a backgrounded or fully killed app is a high-priority FCM message. It wakes the process just long enough to fetch the new message from the server and either update local state or post a notification, then the process can go back to sleep. So the full picture is two systems with two different jobs, not one competing with the other: WebSocket while the screen is open, FCM as the wake-up call once it isn't.
class ChatMessagingService : FirebaseMessagingService() {
override fun onMessageReceived(message: RemoteMessage) {
val payload = message.data["payload"] ?: return
showNotification(payload)
}
}
The whole point of optimistic send is that the UI never waits on the network. The instant the user taps send, insert the message locally with a pending status and render it right away, the round trip to the server happens after the fact, not before. From there a message moves through a small state machine. Pending means it is queued on the device and hasn't been acknowledged. Sent means the server has acked receipt. Delivered means the recipient's device has actually received it. Read means the recipient opened it. The sender's own client is the one that sets pending, the server's acknowledgment is what flips it to sent, and it's the recipient's device that reports delivered and read back through the same channel. The harder question interviewers like to poke at is retries. If the socket drops mid-send and the client resends the same message, what stops the server from creating a second, duplicate message? The fix is a client-generated id, created once and reused on every resend of that same logical message. The server treats it as an idempotency key: the first arrival creates the message, every later arrival with the same id is a no-op. Only once the server actually persists the message does it hand back a canonical server id and sequence number.
data class SendMessageRequest(
val clientMessageId: String = UUID.randomUUID().toString(),
val roomId: String,
val text: String
)
// Server upserts on clientMessageId: a duplicate retry is a silent no-op
Ordering across every participant's device can't depend on each phone's own clock. Wall clocks drift, users travel across time zones, and a clock can simply be set wrong, so two devices can disagree about which message came first. The fix is to make the server the single authority: when it persists a message it assigns a sequence number, or an equivalent server timestamp, and every client sorts by that value rather than by when a message happened to arrive on the wire. That authoritative sequence buys you something else almost for free: gap detection. If a client is watching sequence numbers and sees the stream jump from 100 straight to 103, it knows immediately that 101 and 102 are missing, most likely a message that got dropped during a reconnect, and it can ask the server for exactly that range. That's far cheaper than reloading the whole conversation from scratch, and it keeps the local copy complete without the user noticing anything went wrong.
fun detectGaps(messages: List<MessageEntity>): List<LongRange> {
val sorted = messages.sortedBy { it.serverSeq }
val gaps = mutableListOf<LongRange>()
for (i in 1 until sorted.size) {
val prev = sorted[i - 1].serverSeq
val curr = sorted[i].serverSeq
if (curr - prev > 1) gaps.add((prev + 1)..(curr - 1))
}
return gaps
}
As with any offline-capable Android app, the message list should follow a single, unambiguous rule: the UI never binds directly to the socket, and it never binds directly to the network response either. Everything, socket events, FCM-triggered fetches, and the results of paginated history requests, gets written into a local Room database first. The UI observes that database through a Flow and renders whatever is in it. This is what makes the screen behave identically whether the device is online or offline: there's exactly one path data takes to reach the screen, so there's exactly one place to reason about correctness. It also means the repository layer, not the UI, owns the job of reconciling data that arrives from multiple sources.
@Dao
interface MessageDao {
@Query("SELECT * FROM messages WHERE roomId = :roomId ORDER BY serverSeq ASC")
fun observeMessages(roomId: String): Flow<List<MessageEntity>>
}
Having Room as the single source of truth creates a new problem worth naming explicitly: the same message can legitimately reach the device twice. It might arrive live over the WebSocket, and then again when a high-priority FCM message triggers a fetch after a reconnect. This is a different kind of duplicate than the retry problem from earlier: that one was about the client sending the same message twice, this one is about the client receiving the same message twice through two different paths. The fix follows the same principle though: make the write idempotent. Key the local row on the canonical server message id, the one the server handed back once it persisted the message, and use an upsert. The second delivery, whichever path it comes from, just overwrites the row with identical data, so the UI never renders two copies.
@Entity(tableName = "messages")
data class MessageEntity(
@PrimaryKey val serverId: String,
val roomId: String,
val text: String,
val serverSeq: Long
)
@Dao
interface MessageDao {
@Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun upsert(message: MessageEntity)
}
Typing indicators and read receipts are high-frequency but low-value signals: nobody needs to know about every single keystroke, they just need a rough, timely sense that the other person is typing or has read a message. Firing a network call on every keystroke or every scroll position would spend far more battery and bandwidth than the feature is worth. The standard fix is to debounce and batch: wait for a short pause before sending a typing signal, coalesce multiple low-priority updates together where possible, and have the server auto-expire the typing state after a few seconds of silence rather than waiting for an explicit stopped-typing event that might never arrive if the app is killed.
private val typingEvents = MutableSharedFlow<Unit>(extraBufferCapacity = 1)
val debouncedTyping = typingEvents
.debounce(500)
.onEach { sendTypingSignal() }
.launchIn(viewModelScope)
A WebSocket can end up half-open: the connection object still looks alive on the device, but the other end, often after a radio handoff between cell towers or a Wi-Fi to mobile switch, is long gone. This matters because a half-open socket silently swallows outgoing messages while giving no error, which is worse than an obvious disconnect. The OS's own TCP keepalive exists but is far too slow and unreliable on mobile to catch this promptly. So chat clients add their own signal on top: send a small application-level ping periodically, and expect a pong back within a short window. If the pong doesn't arrive in time, treat the connection as dead, tear it down, and reconnect. This is the classic interview gotcha: candidates who haven't hit it in production usually assume a broken socket will just throw on the next read, and it often won't.
webSocket.send("""{"type":"ping"}""")
var pongWatchdog: Job = scope.launch {
delay(10_000)
webSocket.cancel()
}
// In onMessage: if (msg.type == "pong") pongWatchdog.cancel()
Once you've detected a dead connection, the next question is how fast to retry. Reconnecting immediately in a tight loop feels responsive for one client, but imagine a server blip that drops every connected client at the same instant: if they all retry on the same fixed schedule, the flood of simultaneous reconnect attempts can overwhelm the server just as it's trying to recover, a thundering herd. The standard fix is exponential backoff, doubling the delay after each failed attempt up to a cap, combined with a small random jitter added to each client's delay so that even clients with identical attempt counts don't all fire at the same millisecond.
fun reconnectDelay(attempt: Int): Long {
val base = minOf(30_000L, 1_000L * (1L shl attempt))
val jitter = (Math.random() * 1_000).toLong()
return base + jitter
}
Messages composed while the device is offline need to survive more than a dropped socket, they need to survive the app process being killed entirely. That means the pending send queue can't live in memory, it has to be a persisted table in Room with a pending status, written the moment the user hits send. When connectivity returns, that queue needs flushing, ideally immediately if the socket reconnects, but as a durable fallback you also want a WorkManager job that only fires when the network is actually available. A one-time, expedited work request constrained to NetworkType.CONNECTED does that, and enqueuing it as unique work with an existing-work policy of KEEP stops duplicate flush jobs piling up if connectivity flaps on and off for a while. Because every queued message already carries the client-generated id from the optimistic-send step, a flush that runs twice by accident is harmless, the server just no-ops the ones it's already seen.
val flushWork = OneTimeWorkRequestBuilder<SendQueueWorker>()
.setConstraints(Constraints(requiredNetworkType = NetworkType.CONNECTED))
.setExpedited(OutOfQuotaPolicy.RUN_AS_NON_EXPEDITED_WORK_REQUEST)
.build()
WorkManager.getInstance(context).enqueueUniqueWork(
"flush_send_queue",
ExistingWorkPolicy.KEEP,
flushWork
)
Scrolling back through chat history has the same pagination problem as any feed, except it's worse here because new messages can land in the middle of the exact range you're paging through. Page-number style pagination, using OFFSET and LIMIT, assumes the underlying rows hold still between requests. They don't: if five new messages arrive while the user is on page two, OFFSET-based paging will either skip rows or show the same one twice, because row number 40 means something different after an insert than it did before. Cursor pagination, also called keyset pagination, sidesteps this by anchoring each request to a stable boundary instead of a row count, typically the id or timestamp of the last message the client already has. Because that boundary doesn't move when new rows are inserted elsewhere, the client can keep paging backward through history correctly no matter how much new traffic arrives at the front of the conversation.
@GET("rooms/{id}/messages")
suspend fun getMessages(
@Path("id") roomId: String,
@Query("before") beforeId: String?,
@Query("limit") limit: Int = 50
): List<Message>
Cursor pagination tells you how to ask the server for the next page, but it doesn't by itself say where that data should live once it arrives, and the earlier rule was that Room, not the network response, is the source of truth the UI renders from. Paging 3 has a component built for exactly this seam: a RemoteMediator. Its job is to notice when the UI has scrolled to a boundary that needs more data, fetch that page from the network, and write it into Room. The UI itself never touches the RemoteMediator directly, it's backed by a PagingSource that reads from Room, so the screen is always rendering local data, and the RemoteMediator's only role is keeping that local data topped up. That gives you an offline-first paged history list without the UI needing to know or care whether a given screenful came from cache or from a fresh network call.
class MessageRemoteMediator(
private val db: AppDatabase,
private val api: ChatApi,
private val roomId: String
) : RemoteMediator<Int, MessageEntity>() {
override suspend fun load(
loadType: LoadType,
state: PagingState<Int, MessageEntity>
): MediatorResult {
val cursor = when (loadType) {
LoadType.PREPEND -> return MediatorResult.Success(endOfPaginationReached = true)
LoadType.REFRESH -> null
LoadType.APPEND -> state.lastItemOrNull()?.serverId
}
val page = api.getMessages(roomId, before = cursor, limit = state.config.pageSize)
db.messageDao().upsert(page.map { it.toEntity() })
return MediatorResult.Success(endOfPaginationReached = page.isEmpty())
}
}
Photos and videos don't belong on the real-time channel at all. Base64-encoding a file into WebSocket frames bloats it by roughly a third and blocks the connection other messages need to flow through, and FCM is worse for this, its payload is capped at around four kilobytes, nowhere near enough for an image. The pattern that works is to decouple the binary from the message: upload the file separately over ordinary resumable HTTP to object storage, which gives you upload progress and the ability to retry just the failed chunk rather than the whole file, and once that finishes, send a small control message over the normal channel that just carries the resulting URL. The real-time channel stays lean and fast for everyone in the room, and the heavy transfer gets the retry and progress handling it actually needs.
val body = MultipartBody.Builder()
.setType(MultipartBody.FORM)
.addFormDataPart("file", fileName, file.asRequestBody("image/jpeg".toMediaType()))
.build()
val blobUrl = httpClient.newCall(Request.Builder().url(uploadUrl).post(body).build())
.execute().body?.string()
socket.send("""{"type":"image","url":"$blobUrl","roomId":"$roomId"}""")