Skip to content
Building in Public

Three Servers, One Door, Zero Downtime

Neil Camm 8 min read

I was recently interviewing with a small startup, and one of the questions was, "How comfortable are you with DevOps and managing servers?"

At a small company that's a fair thing to ask. There's no infrastructure team to hand things off to, so whoever writes the code usually keeps it running too. Instead of listing tools, I explained how totalHOA actually runs. By the end of the answer I realized it was worth writing down.

Here's the short version. totalHOA runs on three servers, and only one of them faces the internet.

The three servers

Each server has one job.

  • The edge server is the front door. Every visitor to every community site arrives here first. It handles the secure connection (the padlock in your browser) and sends each request to the right app server.

  • The app server runs totalHOA itself, including the web pages, the background jobs, live notifications and search.

  • The database server holds the data, and it only talks to the app server.

The three are connected by a private network that the outside world can't see. Uploaded files are stored separately in the cloud, so the servers themselves don't hold anything that can't be rebuilt.

Why keep it this simple?

You can run software on some very sophisticated platforms these days. I chose three plain servers on purpose.

  • I understand every piece. When something breaks late at night, I can log in and fix it myself instead of digging through layers of tooling.

  • The costs are predictable. It's a fixed monthly bill, and the edge server is the smallest one the hosting company sells.

  • It gives me what actually matters, which is separation. The database is never on the internet, and the server that faces the internet holds no code and no passwords. You don't need a complicated platform for that. You need three servers and a firewall.

The door to the compound

I think of the edge server as the door to a walled compound. There's one way in, and the buildings inside have no entrances facing the street.

Having a single entrance turns out to be very useful.

There's less to attack. Only the edge server is reachable from the internet, and it runs almost nothing. The app and database servers are effectively invisible. You can't break into a door you can't find.

Every rule is set up once. Because all traffic comes through one place, one set of rules protects every community on the platform, including ones that sign up tomorrow.

  • A firewall blocks everything except normal web traffic.

  • A tool called fail2ban watches for suspicious behavior, like someone repeatedly guessing a password, and automatically bans that address. Once banned, their traffic is dropped before it reaches anything that matters.

  • Every connection is encrypted, and every response includes standard browser security settings.

  • The server itself is hardened with automatic security updates, tightly controlled logins, and alerts if important files change.

New communities go live without any server work. Normally every website needs its own security certificate, set up and renewed by hand. The edge server does this automatically. When someone visits a new community's site for the first time, the edge server checks with the app that it's a real customer, then issues the certificate on the spot. Adding a community is just adding it in the app.

It decides where each community goes. This was one of the main reasons I added the edge server in the first place. It runs what's called a reverse proxy, which is software that receives every request and forwards it to a server behind the scenes. Because every community has its own web address, the proxy can send each community to whichever app server I choose.

Right now, while we're just starting out, there's only one app server, so every community goes there. But nothing about the design depends on that. Tomorrow I could add a second app server and move some communities onto it, give a large community a server of its own, or bring a new server online, test it with one community, and move the rest over gradually. Residents would never notice, because they still come through the same front door.

It opens up options. Because everything passes through the door, I can add protections there later without touching the app, like slowing down anyone hammering the login page, blocking known bad bots, showing a friendly maintenance page during an outage, or spreading traffic across more servers as the platform grows.

But a door isn't enough on its own

The analogy has a limit. A door stops traffic that looks wrong, but someone with a stolen password walks through like any resident would. So every building inside still has its own locks.

  • The app checks every request and makes sure people only see their own community's data.

  • The database only accepts connections from the app server, even from inside the private network.

  • Passwords and keys live only on the app server, never on the edge.

Getting through the front door isn't the same as getting in. You want both.

One app server, five jobs

"App server" sounds like one thing, but it's really doing five jobs at once.

  1. Serving web pages to residents and board members

  2. Running background jobs like sending emails, processing payments and reading uploaded documents

  3. Running scheduled tasks like event reminders and invoices

  4. Pushing live updates so notifications show up without refreshing the page

  5. Powering search

Sharing one machine is the right call for now. It's simpler and cheaper, and at our current size there's plenty of room.

But I know where the limit is. Only the first of those jobs is about serving visitors. The other four happen in the background, and they all draw on the same processing power and memory as the web pages. As traffic and the number of communities grow, that background work will start competing with the visitors. A big batch of document uploads or a large email send could slow down page loads for everyone, simply because they're sharing a machine.

So as we scale, the plan is to move that work off the main app server and onto infrastructure of its own, with dedicated servers for background jobs, search and live updates. The app server can then focus on one thing, serving web pages quickly, and the heavy lifting happens elsewhere without getting in the way.

Updates without downtime

I also like how updates reach the live site. It's a technique called atomic deployment.

The simple way to update a website is to change the files while it's running. The problem is that for a minute or two the site is half old and half new, and anyone visiting at that moment can hit an error. If the update fails partway, the site stays broken.

Atomic deployment avoids that.

  1. When I push an update, the complete new version is built in its own separate folder, while the current version keeps serving visitors.

  2. Everything gets prepared and checked in that folder.

  3. Only when it's fully ready does the site switch over in a single instant. One visitor gets the old version, and the next gets the new one.

Think of it like building a new stage behind the curtain and swapping it in between scenes, instead of rebuilding the stage in front of the audience.

This makes a few things possible.

  • Visitors never see a half-updated site.

  • A failed update does no harm. If anything goes wrong while building, the switch never happens, and the live site doesn't change.

  • Undoing a bad update takes seconds. The last few versions are kept, so going back is just switching to the previous one.

The whole process, from pushing an update to it being live, takes under two minutes.

Room to grow

The nice thing about this setup is that it grows in simple steps.

  1. Bigger servers. The easiest upgrade, and it goes a long way.

  2. Split the app server's jobs onto their own servers.

  3. Add more app servers. The edge server already chooses which app server each community goes to, so adding capacity means adding a server and pointing some communities at it.

  4. Scale the database with a bigger machine, then extra copies to share the load.

Through all of that, the edge server stays the single front door. Customers never have to change anything, even as the servers behind the door change.

Every server is also set up by a script I can run again at any time, so building a new one takes minutes, not days.

What I'd tell anyone building something similar

  • Start simple, and separate the important parts. Keep your database off the internet and your public-facing server free of anything valuable.

  • Build one front door, and put your protections there.

  • Lock the doors inside too. The front door is the first layer, not the only one.

  • Update atomically, so a bad update is a non-event.

  • Choose tools you understand. The best infrastructure is the kind you can fix yourself.

So, back to that interview question. How comfortable am I with DevOps? Comfortable enough to have built this, run it in production, and know what I'd change as it grows. For a small team, I think that's what the question is really asking.