Zero downtime deployments with Nginx

Relevant blog post explaining mostly the same: https://kaiwern.com/posts/2021/05/15/blue-green-deployment-with-nginx-for-elixir-phoenix-release/

Preparation 🔗

Ensure your environment is set up with Nginx and that your application can be started in such a way that it listens on a specific port you can control. This might involve configuring your application to read a port number from an environment variable or a configuration file.

Deployment Script 🔗

Create a deployment script that performs the following steps:

• Step A: Prepare the Green Deployment
  • Deploy the new version of the application to a new directory.
  • Start the new version on a different port. Ensure this port is not used by the current Blue deployment or any other services.
• Step B: Health Check
  • Perform a health check on the Green deployment to ensure it’s running correctly. This could be as simple as a curl request to the health endpoint.
• Step C: Update Nginx Configuration
  • Modify the Nginx configuration to route traffic to the new port where the Green deployment is running.
  • This involves updating the upstream configuration in Nginx to point to the new port.

upstream myapp {
    server 127.0.0.1:<green_port>;
}

• Step D: Reload Nginx
  • Reload Nginx to apply the configuration changes without downtime. Use the nginx -s reload command.
• Step E: Monitor and Cleanup
  • Monitor the logs and performance of the Green deployment to ensure everything is running smoothly.
  • Once you’re confident in the Green deployment’s stability, stop the Blue deployment and reclaim any resources it was using.

Automation with CI/CD Tools 🔗

To fully automate the process, you can integrate the deployment script into a CI/CD pipeline using tools like Jenkins, GitLab CI, GitHub Actions, or CircleCI. Here’s a simplified workflow:

• Trigger: The pipeline is triggered by a push to the master branch or a specific deployment tag.
• Build: Compile the code, run tests, and build your application.
• Deploy: Run the deployment script as outlined above. This script performs the actual deployment and switch over to the new version.

Script Example 🔗

Here’s a very simplified version of what the script part could look like. Note that actual implementation details will depend on your specific application setup:

#!/bin/bash

# Configuration
GREEN_PORT=8081
BLUE_PORT=8080
NGINX_CONFIG="/etc/nginx/sites-available/myapp"
APP_DIRECTORY="/var/www/myapp"


# Deploy new version (Green)
# This would include pulling the new code, compiling it if necessary, and any other setup required
cp -R "${APP_DIRECTORY}_blue" "${APP_DIRECTORY}_green"
# Assume start_app.sh takes a port number as an argument
bash start_app.sh $GREEN_PORT "${APP_DIRECTORY}_green"


# Health check
if ! curl -f <http://localhost>:$GREEN_PORT/health; then
  echo "Health check failed"
  exit 1
fi


# Update Nginx to point to Green
sed -i "s/server 127.0.0.1:$BLUE_PORT;/server 127.0.0.1:$GREEN_PORT;/" $NGINX_CONFIG


# Reload Nginx
nginx -s reload


# Additional steps might include monitoring and eventually cleaning up the old deployment

Script considerations 🔗

• Error Handling: Your script should include error handling to revert changes if anything goes wrong during the deployment.
• Security: Ensure your deployment process is secure, particularly if you’re pulling code or artifacts from external sources.
• Environment Variables: Use environment variables for configuration to make your application more flexible and your scripts more generic.