Setting Up a Custom VPN Server with Nginx Proxy and OpenVPN

Setting up a custom VPN server can greatly enhance your security and privacy online. With a combination of Nginx as a reverse proxy and OpenVPN for secure tunneling, you can build a robust, efficient VPN solution tailored to your needs. In this guide, we’ll walk through the technical steps of configuring these tools together on a Linux server.

Prerequisites

• A Linux server (Ubuntu or Debian recommended).
• Root or sudo access to the server.
• Basic knowledge of Linux command-line operations.
• Installed Nginx and OpenVPN packages.

Installing OpenVPN

First, ensure that OpenVPN is installed on your server. You can do this by running the following command:

sudo apt update && sudo apt install openvpn easy-rsa

After installation, we will configure the OpenVPN server to handle VPN connections securely. We will also use Easy-RSA to manage SSL certificates for secure communications.

Setting Up OpenVPN

We will now set up OpenVPN to create the necessary certificates and keys for secure communication. Start by setting up the Easy-RSA directory:

make-cadir /openvpn-ca
cd /openvpn-ca

Once the directory is set, configure the vars file to match your organization’s details:

nano vars

Update the following lines in the vars file:

export KEY_COUNTRY="US"
export KEY_PROVINCE="CA"
export KEY_CITY="San Francisco"
export KEY_ORG="YourOrg"
export KEY_EMAIL="youremail@example.com"

Now, generate the server keys and certificates:

source ./vars
./clean-all
./build-ca
./build-key-server server
./build-dh

Next, create a certificate for each client who will connect to the VPN:

./build-key clientname

Configuring OpenVPN Server

After generating the certificates, configure the OpenVPN server by editing the server configuration file:

sudo nano /etc/openvpn/server.conf

Here’s an example configuration file:

port 1194
proto udp
dev tun
ca ca.crt
cert server.crt
key server.key
dh dh2048.pem
server 10.8.0.0 255.255.255.0
ifconfig-pool-persist ipp.txt
push "redirect-gateway def1 bypass-dhcp"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
keepalive 10 120
cipher AES-256-CBC
comp-lzo
user nobody
group nobody
persist-key
persist-tun
status openvpn-status.log
verb 3

Once the configuration file is ready, start the OpenVPN service:

sudo systemctl start openvpn@server

Installing and Configuring Nginx as Reverse Proxy

Next, install Nginx if it’s not already installed:

sudo apt install nginx

Now configure Nginx to act as a reverse proxy to the OpenVPN service. First, we need to create a new Nginx configuration file:

sudo nano /etc/nginx/sites-available/openvpn

Here’s an example configuration for Nginx to forward VPN traffic:

server {
    listen 443 ssl;
    server_name vpn.yourdomain.com;

    ssl_certificate /etc/nginx/ssl/vpn.crt;
    ssl_certificate_key /etc/nginx/ssl/vpn.key;

    location / {
        proxy_pass https://localhost:1194;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Enable this site configuration by creating a symlink:

sudo ln -s /etc/nginx/sites-available/openvpn /etc/nginx/sites-enabled/

Finally, restart Nginx to apply the changes:

sudo systemctl restart nginx

Testing the VPN Setup

Now that both OpenVPN and Nginx are configured, it’s time to test the setup. Ensure that your firewall allows UDP traffic on port 1194:

sudo ufw allow 1194/udp
sudo ufw reload

Then, attempt to connect from a client device using the OpenVPN client configuration file. The connection should route through Nginx, which acts as the reverse proxy.

Final Configuration and Troubleshooting

If the connection fails, check the OpenVPN logs for errors:

sudo tail -f /var/log/openvpn.log

Also, verify Nginx logs to ensure that traffic is correctly being forwarded:

sudo tail -f /var/log/nginx/error.log

Finally, if you encounter any issues with SSL certificates, ensure they are correctly configured in both Nginx and OpenVPN.

Learn how to stay safe online in this free 34-page eBook.

Leave this field empty if you're human: