Skip to main content

Reference Architecture with Terraform: VM-Series in GCP, Centralized Architecture, Common NGFW Option

Palo Alto Networks produces several validated reference architecture design and deployment documentation guides, which describe well-architected and tested deployments. When deploying VM-Series in a public cloud, the reference architectures guide users toward the best security outcomes, whilst reducing rollout time and avoiding common integration efforts. The Terraform code presented here will deploy Palo Alto Networks VM-Series firewalls in GCP based on a centralized design with common VM-Series for all traffic; for a discussion of other options, please see the design guide from the reference architecture guides.

GitHub Logo Terraform Logo

Reference Architecture Design​

simple

This code implements:

  • a centralized design, a hub-and-spoke topology with a shared VPC containing VM-Series deployed in high availability to inspect all inbound, outbound, east-west, and enterprise traffic
  • the common option, which routes all traffic flows onto a single set of VM-Series

Detailed Architecture and Design​

Centralized Design​

This design uses a VPC Peering. Application functions are distributed across multiple projects that are connected in a logical hub-and-spoke topology. A security project acts as the hub, providing centralized connectivity and control for multiple application projects. You deploy all VM-Series firewalls within the security project. The spoke projects contain the workloads and necessary services to support the application deployment. This design model integrates multiple methods to interconnect and control your application project VPC networks with resources in the security project. VPC Peering enables the private VPC network in the security project to peer with, and share routing information to, each application project VPC network. Using Shared VPC, the security project administrators create and share VPC network resources from within the security project to the application projects. The application project administrators can select the network resources and deploy the application workloads.

Common Option with High Availabikity​

The common firewall option wiht High Availability leverages a single set of VM-Series firewalls that acts as a single entity. The sole set of firewalls operates as a shared resource and may present scale limitations with all traffic flowing through a single set of firewalls due to the performance degradation that occurs when traffic crosses virtual routers. This option is suitable for proof-of-concepts and smaller scale deployments because the number of firewalls is low. However, the technical integration complexity is high.

VM-Series-Common-Firewall-Option-with-HA

The scope of this code is to deploy an example of the VM-Series Common Firewall Option architecture with high availability configuration within a GCP project.

The example makes use of VM-Series full bootstrap process using XML templates to properly parametrize the initial Day 0 configuration.

With default variable values the topology consists of :

  • 6 VPC networks :
    • Management VPC
    • Untrust (outside) VPC
    • Trust (inside/security) VPC
    • HA2 (strictly used for High Availability) VPC
    • Spoke-1 VPC
    • Spoke-2 VPC
  • 2 VM-Series firewalls
  • 2 Linux Ubuntu VMs (inside Spoke VPCs - for testing purposes)
  • one internal network loadbalancer (for outbound/east-west traffic)
  • one external regional network loadbalancer (for inbound traffic)

Prerequisites​

The following steps should be followed before deploying the Terraform code presented here.

  1. Prepare VM-Series licenses
  2. Configure the terraform google provider

Usage​

  1. Access Google Cloud Shell or any other environment that has access to your GCP project

  2. Clone the repository:

git clone https://github.com/PaloAltoNetworks/terraform-google-vmseries-modules
cd terraform-google-vmseries-modules/examples/vmseries_ha
  1. Copy the example.tfvars to terraform.tfvars.

project, ssh_keys and allowed_sources should be modified for successful deployment and access to the instance.

There are also a few variables that have some default values but which should also be changed as per deployment requirements

  • region
  • vmseries.<fw-name>.bootstrap_options
  • linux_vms.<vm-name>.linux_disk_size
  1. Apply the terraform code:
terraform init
terraform apply

  1. Check the output plan and confirm the apply.

  2. Check the successful application and outputs of the resulting infrastructure:

Apply complete! Resources: 96 added, 0 changed, 0 destroyed. (Number of resources can vary based on how many instances you push through tfvars)

Outputs:

lbs_internal_ips = {
  "external-lb" = "<EXTERNAL_LB_PUBLIC_IP>"
}
lbs_internal_ips = {
  "internal-lb" = "10.10.12.5"
}
linux_vm_ips = {
  "spoke1-vm" = "192.168.1.2"
  "spoke2-vm" = "192.168.2.2"
}
vmseries_private_ips = {
  "fw-vmseries-01" = {
    "0" = "10.10.11.2"
    "1" = "10.10.10.2"
    "2" = "10.10.12.2"
    "3" = "10.10.13.2"
  }
  "fw-vmseries-02" = {
    "0" = "10.10.11.3"
    "1" = "10.10.10.3"
    "2" = "10.10.12.3"
    "3" = "10.10.13.3"
  }
}
vmseries_public_ips = {
  "fw-vmseries-01" = {
    "0" = "<UNTRUST_PUBLIC_IP>"
    "1" = "<MGMT_PUBLIC_IP>"
  }
  "fw-vmseries-02" = {
    "0" = "<UNTRUST_PUBLIC_IP>"
    "1" = "<MGMT_PUBLIC_IP>"
  }
}

Post build​

Connect to the VM-Series instance(s) via SSH using your associated private key and check if the bootstrap process if finished successfuly and then set a password :

  • Please allow for up to 10-15 minutes for the bootstrap process to finish
  • The key output you should check for is "Auto-commit Successful"
ssh admin@x.x.x.x -i /PATH/TO/YOUR/KEY/id_rsa
Welcome admin.
admin@PA-VM(active)> show system bootstrap status

Bootstrap Phase               Status         Details
===============               ======         =======
Media Detection               Success        Media detected successfully
Media Sanity Check            Success        Media sanity check successful
Parsing of Initial Config     Successful     
Auto-commit                   Successful

admin@PA-VM(active)> configure
Entering configuration mode
[edit]                                                                                                                                                                                  
admin@PA-VM(active)# set mgt-config users admin password
Enter password   : 
Confirm password : 

[edit]                                                                                                                                                                                  
admin@PA-VM(active)# commit
Configuration committed successfully

Check access via web UI​

Use a web browser to access https://<MGMT_PUBLIC_IP> and login with admin and your previously configured password.

Change the public Loopback public IP Address​

For the VM-Series that are backend instance group members of the public-facing loadbalancer - go to Network -> Interfaces -> Loopback and change the value of 1.1.1.1 with the value of object external-lb from the lbs_external_ips in Terraform outputs.

In order to successfuly access the web server hosted on spoke-1-vm - also reconfigure the two NAT policies that contain the destination address of 1.1.1.1 with the IP address from lbs_external_ips from Terraform outputs in Policies -> NAT -> no-nat-lb-healthchecks + inbound-app1

Check traffic from spoke VMs​

The firewalls are bootstrapped with a generic allow any policy just for demo purposes along with an outboud SNAT policy to allow Inernet access from spoke VMs.

SSH to one of the spoke VMs using GCP IAP and gcloud command and test connectivity :

gcloud compute ssh <NAME-PREFIX-VALUE-FROM-TFVARS>spoke1-vm --zone=<NAME-OF-THE-ZONE-FROM-TFVARS>
No zone specified. Using zone [us-east1-b] for instance: [spoke1-vm].
External IP address was not found; defaulting to using IAP tunneling.
WARNING: 

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

<USERNAME>@spoke1-vm:~$ping 8.8.8.8
<USERNAME>@spoke1-vm:~$ping 192.168.2.2

Check traffic towards the test HTTP web server​

From any browser access http://<EXTERNAL_LB_PUBLIC_IP>

Test fail-over​

Connect to the spoke VM via gcloud CLI and continously ping a destination on the internet :

gcloud compute ssh <NAME-PREFIX-VALUE-FROM-TFVARS>spoke1-vm --zone=<NAME-OF-THE-ZONE-FROM-TFVARS>
No zone specified. Using zone [us-east1-b] for instance: [spoke1-vm].
External IP address was not found; defaulting to using IAP tunneling.
WARNING: 

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

<USERNAME>@spoke1-vm:~$ping 8.8.8.8

Continously try to access the test HTTP web server - below is an example bash script that will continously try to access the web server :

while true; do curl -vvvv --connect-timeout 2 http://<EXTERNAL_LB_PUBLIC_IP>/; sleep 2; done

From the active VM-Series go to Device -> High Availability -> Operational Commands -> Suspend local device for high availability .

Check the succesful inbound and outbound traffic fail-over to and from the spoke VM.

Reference​

Requirements​

NameVersion
terraform>= 1.2, < 2.0

Providers​

NameVersion
googlen/a
localn/a

Modules​

NameSourceVersion
bootstrap../../modules/bootstrapn/a
iam_service_account../../modules/iam_service_accountn/a
lb_external../../modules/lb_externaln/a
lb_internal../../modules/lb_internaln/a
vmseries../../modules/vmseriesn/a
vpc../../modules/vpcn/a
vpc_peering../../modules/vpc-peeringn/a

Resources​

NameType
google_compute_instance.linux_vmresource
google_compute_route.thisresource
local_file.bootstrap_xmlresource
local_file.init_cfgresource
google_compute_image.my_imagedata source

Inputs​

NameDescriptionTypeDefaultRequired
bootstrap_bucketsA map containing each bootstrap bucket setting.

Example of variable deployment:
bootstrap_buckets = {
  vmseries-bootstrap-bucket-01 = {
    bucket_name_prefix  = "bucket-01-"
    location            = "us"
    service_account_key = "sa-vmseries-01"
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		map(any){}no
lbs_externalA map containing each external loadbalancer setting.

Example of variable deployment :
lbs_external = {
  "external-lb" = {
    name     = "external-lb"
    backends = ["fw-vmseries-01", "fw-vmseries-02"]
    rules = {
      "all-ports" = {
        ip_protocol = "L3_DEFAULT"
      }
    }
    http_health_check_port         = "80"
    http_health_check_request_path = "/php/login.php"
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		map(any){}no
lbs_internalA map containing each internal loadbalancer setting.

Example of variable deployment :
lbs_internal = {
  "internal-lb" = {
    name              = "internal-lb"
    health_check_port = "80"
    backends          = ["fw-vmseries-01", "fw-vmseries-02"]
    ip_address        = "10.10.12.5"
    subnetwork        = "fw-trust-sub"
    network           = "fw-trust-vpc"
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		map(any){}no
linux_vmsA map containing each Linux VM configuration that will be placed in SPOKE VPCs for testing purposes.

Example of varaible deployment:
linux_vms = {
  spoke1-vm = {
    linux_machine_type = "n2-standard-4"
    zone               = "us-east1-b"
    linux_disk_size    = "50" # Modify this value as per deployment requirements
    subnetwork         = "spoke1-sub"
    private_ip         = "192.168.1.2"
    scopes = [
      "https://www.googleapis.com/auth/compute.readonly",
      "https://www.googleapis.com/auth/cloud.useraccounts.readonly",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring.write",
    ]
    service_account_key = "sa-linux-01"
  }
}
any{}no
name_prefixA string to prefix resource namings.string"example-"no
networksA map containing each network setting.

Example of variable deployment :
networks = {
  "mgmt-network" = {
    create_network                  = true
    create_subnetwork               = true
    name                            = "fw-mgmt-vpc"
    subnetwork_name                 = "fw-mgmt-sub"
    ip_cidr_range                   = "10.10.10.0/28"
    allowed_sources                 = ["1.1.1.1/32"]
    delete_default_routes_on_create = false
    allowed_protocol                = "all"
    allowed_ports                   = []
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		anyn/ayes
projectThe project name to deploy the infrastructure in to.stringnullno
regionThe region into which to deploy the infrastructure in to.string"us-central1"no
routesA map containing each route setting. Note that you can only add routes using a next-hop type of internal load-balance rule.

Example of variable deployment :
routes = {
  "default-route-trust" = {
    name = "fw-default-trust"
    destination_range = "0.0.0.0/0"
    network = "fw-trust-vpc"
    lb_internal_name = "internal-lb"
  }
}
Multiple keys can be added and will be deployed by the code.		map(any){}no
service_accountsA map containing each service account setting.

Example of variable deployment :
service_accounts = {
  "sa-vmseries-01" = {
    service_account_id = "sa-vmseries-01"
    display_name       = "VM-Series SA"
    roles = [
      "roles/compute.networkViewer",
      "roles/logging.logWriter",
      "roles/monitoring.metricWriter",
      "roles/monitoring.viewer",
      "roles/viewer"
    ]
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		map(any){}no
vmseriesA map containing each individual vmseries setting.

Example of variable deployment :
vmseries = {
  "fw-vmseries-01" = {
    name             = "fw-vmseries-01"
    zone             = "us-east1-b"
    machine_type     = "n2-standard-4"
    min_cpu_platform = "Intel Cascade Lake"
    tags                 = ["vmseries"]
    service_account_key  = "sa-vmseries-01"
    scopes = [
      "https://www.googleapis.com/auth/compute.readonly",
      "https://www.googleapis.com/auth/cloud.useraccounts.readonly",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring.write",
    ]
    bootstrap_bucket_key = "vmseries-bootstrap-bucket-01"
    bootstrap_options = {
      panorama-server = "1.1.1.1"
      dns-primary     = "8.8.8.8"
      dns-secondary   = "8.8.4.4"
    }
    bootstrap_template_map = {
      trust_gcp_router_ip   = "10.10.12.1"
      untrust_gcp_router_ip = "10.10.11.1"
      private_network_cidr  = "192.168.0.0/16"
      untrust_loopback_ip   = "1.1.1.1/32" #This is placeholder IP - you must replace it on the vmseries config with the LB public IP address after the infrastructure is deployed
      trust_loopback_ip     = "10.10.12.5/32"
    }
    named_ports = [
      {
        name = "http"
        port = 80
      },
      {
        name = "https"
        port = 443
      }
    ]
    network_interfaces = [
      {
        subnetwork       = "fw-untrust-sub"
        private_ip       = "10.10.11.2"
        create_public_ip = true
      },
      {
        subnetwork       = "fw-mgmt-sub"
        private_ip       = "10.10.10.2"
        create_public_ip = true
      },
      {
        subnetwork = "fw-trust-sub"
        private_ip = "10.10.12.2"
      }
    ]
  }
}
For a full list of available configuration items - please refer to module documentation

The bootstrap_template_map contains variables that will be applied to the bootstrap template. Each firewall Day 0 bootstrap will be parametrised based on these inputs.
Multiple keys can be added and will be deployed by the code.		anyn/ayes
vmseries_commonA map containing common vmseries setting.

Example of variable deployment :
vmseries_common = {
  ssh_keys            = "admin:AAAABBBB..."
  vmseries_image      = "vmseries-flex-byol-1022h2"
  machine_type        = "n2-standard-4"
  min_cpu_platform    = "Intel Cascade Lake"
  service_account_key = "sa-vmseries-01"
  bootstrap_options = {
    type                = "dhcp-client"
    mgmt-interface-swap = "enable"
  }
}
Bootstrap options can be moved between vmseries individual instance variable (vmseries) and this common vmserie variable (vmseries_common).		anyn/ayes
vpc_peeringsA map containing each VPC peering setting.

Example of variable deployment :
vpc_peerings = {
  "trust-to-spoke1" = {
    local_network = "fw-trust-vpc"
    peer_network  = "spoke1-vpc"

    local_export_custom_routes                = true
    local_import_custom_routes                = true
    local_export_subnet_routes_with_public_ip = true
    local_import_subnet_routes_with_public_ip = true

    peer_export_custom_routes                = true
    peer_import_custom_routes                = true
    peer_export_subnet_routes_with_public_ip = true
    peer_import_subnet_routes_with_public_ip = true
  }
}
For a full list of available configuration items - please refer to module documentation

Multiple keys can be added and will be deployed by the code.		map(any){}no

Outputs​

NameDescription
lbs_external_ipsPublic IP addresses of external network loadbalancers.
lbs_internal_ipsPrivate IP addresses of internal network loadbalancers.
linux_vm_ipsPrivate IP addresses of Linux VMs.
vmseries_private_ipsPrivate IP addresses of the vmseries instances.
vmseries_public_ipsPublic IP addresses of the vmseries instances.