Reference Architecture with Terraform: VM-Series in GCP, Centralized Architecture, Common NGFW Option

Palo Alto Networks produces several validated reference architecture design and deployment documentation guides, which describe well-architected and tested deployments. When deploying VM-Series in a public cloud, the reference architectures guide users toward the best security outcomes, whilst reducing rollout time and avoiding common integration efforts. The Terraform code presented here will deploy Palo Alto Networks VM-Series firewalls in GCP based on a centralized design with common VM-Series for all traffic; for a discussion of other options, please see the design guide from the reference architecture guides.

Reference Architecture Design

simple

This code implements:

a centralized design, a hub-and-spoke topology with a shared VPC containing VM-Series deployed in high availability to inspect all inbound, outbound, east-west, and enterprise traffic
the common option, which routes all traffic flows onto a single set of VM-Series

Detailed Architecture and Design

Centralized Design

This design uses a VPC Peering. Application functions are distributed across multiple projects that are connected in a logical hub-and-spoke topology. A security project acts as the hub, providing centralized connectivity and control for multiple application projects. You deploy all VM-Series firewalls within the security project. The spoke projects contain the workloads and necessary services to support the application deployment. This design model integrates multiple methods to interconnect and control your application project VPC networks with resources in the security project. VPC Peering enables the private VPC network in the security project to peer with, and share routing information to, each application project VPC network. Using Shared VPC, the security project administrators create and share VPC network resources from within the security project to the application projects. The application project administrators can select the network resources and deploy the application workloads.

Common Option with High Availabikity

The common firewall option wiht High Availability leverages a single set of VM-Series firewalls that acts as a single entity. The sole set of firewalls operates as a shared resource and may present scale limitations with all traffic flowing through a single set of firewalls due to the performance degradation that occurs when traffic crosses virtual routers. This option is suitable for proof-of-concepts and smaller scale deployments because the number of firewalls is low. However, the technical integration complexity is high.

VM-Series-Common-Firewall-Option-with-HA

The scope of this code is to deploy an example of the VM-Series Common Firewall Option architecture with high availability configuration within a GCP project.

The example makes use of VM-Series full bootstrap process using XML templates to properly parametrize the initial Day 0 configuration.

With default variable values the topology consists of :

6 VPC networks :
- Management VPC
- Untrust (outside) VPC
- Trust (inside/security) VPC
- HA2 (strictly used for High Availability) VPC
- Spoke-1 VPC
- Spoke-2 VPC
2 VM-Series firewalls
2 Linux Ubuntu VMs (inside Spoke VPCs - for testing purposes)
one internal network loadbalancer (for outbound/east-west traffic)
one external regional network loadbalancer (for inbound traffic)

Prerequisites

The following steps should be followed before deploying the Terraform code presented here.

Prepare VM-Series licenses
Configure the terraform google provider

Usage

Access Google Cloud Shell or any other environment that has access to your GCP project
Clone the repository:

git clone https://github.com/PaloAltoNetworks/terraform-google-swfw-modules
cd terraform-google-swfw-modules/examples/vmseries_ha

Copy the example.tfvars to terraform.tfvars.

project, ssh_keys and source_ranges should be modified for successful deployment and access to the instance.

There are also a few variables that have some default values but which should also be changed as per deployment requirements

region
vmseries.<fw-name>.bootstrap_options
linux_vms.<vm-name>.linux_disk_size

Apply the terraform code:

terraform init
terraform apply

Check the output plan and confirm the apply.
Check the successful application and outputs of the resulting infrastructure:

Apply complete! Resources: 96 added, 0 changed, 0 destroyed. (Number of resources can vary based on how many instances you push through tfvars)

Outputs:

lbs_internal_ips = {
  "external-lb" = "<EXTERNAL_LB_PUBLIC_IP>"
}
lbs_internal_ips = {
  "internal-lb" = "10.10.12.5"
}
linux_vm_ips = {
  "spoke1-vm" = "192.168.1.2"
  "spoke2-vm" = "192.168.2.2"
}
vmseries_private_ips = {
  "fw-vmseries-01" = {
    "0" = "10.10.11.2"
    "1" = "10.10.10.2"
    "2" = "10.10.12.2"
    "3" = "10.10.13.2"
  }
  "fw-vmseries-02" = {
    "0" = "10.10.11.3"
    "1" = "10.10.10.3"
    "2" = "10.10.12.3"
    "3" = "10.10.13.3"
  }
}
vmseries_public_ips = {
  "fw-vmseries-01" = {
    "0" = "<UNTRUST_PUBLIC_IP>"
    "1" = "<MGMT_PUBLIC_IP>"
  }
  "fw-vmseries-02" = {
    "0" = "<UNTRUST_PUBLIC_IP>"
    "1" = "<MGMT_PUBLIC_IP>"
  }
}

Post build

Connect to the VM-Series instance(s) via SSH using your associated private key and check if the bootstrap process if finished successfuly and then set a password :

Please allow for up to 10-15 minutes for the bootstrap process to finish
The key output you should check for is "Auto-commit Successful"

ssh admin@x.x.x.x -i /PATH/TO/YOUR/KEY/id_rsa
Welcome admin.
admin@PA-VM(active)> show system bootstrap status

Bootstrap Phase               Status         Details
===============               ======         =======
Media Detection               Success        Media detected successfully
Media Sanity Check            Success        Media sanity check successful
Parsing of Initial Config     Successful     
Auto-commit                   Successful

admin@PA-VM(active)> configure
Entering configuration mode
[edit]                                                                                                                                                                                  
admin@PA-VM(active)# set mgt-config users admin password
Enter password   : 
Confirm password : 

[edit]                                                                                                                                                                                  
admin@PA-VM(active)# commit
Configuration committed successfully

Check access via web UI

Use a web browser to access https://<MGMT_PUBLIC_IP> and login with admin and your previously configured password.

Change the public Loopback public IP Address

For the VM-Series that are backend instance group members of the public-facing loadbalancer - go to Network -> Interfaces -> Loopback and change the value of 1.1.1.1 with the value of object external-lb from the lbs_external_ips in Terraform outputs.

In order to successfuly access the web server hosted on spoke-1-vm - also reconfigure the two NAT policies that contain the destination address of 1.1.1.1 with the IP address from lbs_external_ips from Terraform outputs in Policies -> NAT -> no-nat-lb-healthchecks + inbound-app1

Check traffic from spoke VMs

The firewalls are bootstrapped with a generic allow any policy just for demo purposes along with an outboud SNAT policy to allow Inernet access from spoke VMs.

SSH to one of the spoke VMs using GCP IAP and gcloud command and test connectivity :

gcloud compute ssh <NAME-PREFIX-VALUE-FROM-TFVARS>spoke1-vm --zone=<NAME-OF-THE-ZONE-FROM-TFVARS>
No zone specified. Using zone [us-east1-b] for instance: [spoke1-vm].
External IP address was not found; defaulting to using IAP tunneling.
WARNING: 

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

<USERNAME>@spoke1-vm:~$ping 8.8.8.8
<USERNAME>@spoke1-vm:~$ping 192.168.2.2

Check traffic towards the test HTTP web server

From any browser access http://<EXTERNAL_LB_PUBLIC_IP>

Test fail-over

Connect to the spoke VM via gcloud CLI and continously ping a destination on the internet :

gcloud compute ssh <NAME-PREFIX-VALUE-FROM-TFVARS>spoke1-vm --zone=<NAME-OF-THE-ZONE-FROM-TFVARS>
No zone specified. Using zone [us-east1-b] for instance: [spoke1-vm].
External IP address was not found; defaulting to using IAP tunneling.
WARNING: 

To increase the performance of the tunnel, consider installing NumPy. For instructions,
please see https://cloud.google.com/iap/docs/using-tcp-forwarding#increasing_the_tcp_upload_bandwidth

<USERNAME>@spoke1-vm:~$ping 8.8.8.8

Continously try to access the test HTTP web server - below is an example bash script that will continously try to access the web server :

while true; do curl -vvvv --connect-timeout 2 http://<EXTERNAL_LB_PUBLIC_IP>/; sleep 2; done

From the active VM-Series go to Device -> High Availability -> Operational Commands -> Suspend local device for high availability .

Check the succesful inbound and outbound traffic fail-over to and from the spoke VM.

Reference

Requirements

Name	Version
terraform	>= 1.3, < 2.0

Providers

Name	Version
google	n/a
local	n/a

Modules

Name	Source	Version
bootstrap	../../modules/bootstrap	n/a
iam_service_account	../../modules/iam_service_account	n/a
lb_external	../../modules/lb_external	n/a
lb_internal	../../modules/lb_internal	n/a
vmseries	../../modules/vmseries	n/a
vpc	../../modules/vpc	n/a
vpc_peering	../../modules/vpc-peering	n/a

Resources

Name	Type
google_compute_instance.linux_vm	resource
google_compute_route.this	resource
local_file.bootstrap_xml	resource
local_file.init_cfg	resource
google_compute_image.my_image	data source

Inputs

Name	Description	Type	Default	Required
bootstrap_buckets	A map containing each bootstrap bucket setting. Example of variable deployment:bootstrap_buckets = { vmseries-bootstrap-bucket-01 = { bucket_name_prefix = "bucket-01-" location = "us" service_account_key = "sa-vmseries-01" } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no
lbs_external	A map containing each external loadbalancer setting. Example of variable deployment :lbs_external = { "external-lb" = { name = "external-lb" backends = ["fw-vmseries-01", "fw-vmseries-02"] rules = { "all-ports" = { ip_protocol = "L3_DEFAULT" } } http_health_check_port = "80" http_health_check_request_path = "/php/login.php" } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no
lbs_internal	A map containing each internal loadbalancer setting. Example of variable deployment :lbs_internal = { "internal-lb" = { name = "internal-lb" health_check_port = "80" backends = ["fw-vmseries-01", "fw-vmseries-02"] ip_address = "10.10.12.5" subnetwork_key = "fw-trust-sub" vpc_network_key = "fw-trust-vpc" } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no
linux_vms	A map containing each Linux VM configuration that will be placed in SPOKE VPCs for testing purposes. Example of varaible deployment:linux_vms = { spoke1-vm = { linux_machine_type = "n2-standard-4" zone = "us-east1-b" linux_disk_size = "50" # Modify this value as per deployment requirements vpc_network_key = "fw-spoke1-vpc" subnetwork_key = "fw-spoke1-sub" private_ip = "192.168.1.2" scopes = [ "https://www.googleapis.com/auth/compute.readonly", "https://www.googleapis.com/auth/cloud.useraccounts.readonly", "https://www.googleapis.com/auth/devstorage.read_only", "https://www.googleapis.com/auth/logging.write", "https://www.googleapis.com/auth/monitoring.write", ] service_account_key = "sa-linux-01" } }	`any`	`{}`	no
name_prefix	A string to prefix resource namings.	`string`	`"example-"`	no
networks	A map containing each network setting. Example of variable deployment :networks = { fw-mgmt-vpc = { vpc_name = "fw-mgmt-vpc" create_network = true delete_default_routes_on_create = false mtu = "1460" routing_mode = "REGIONAL" subnetworks = { fw-mgmt-sub = { name = "fw-mgmt-sub" create_subnetwork = true ip_cidr_range = "10.10.10.0/28" region = "us-east1" } } firewall_rules = { allow-mgmt-ingress = { name = "allow-mgmt-vpc" source_ranges = ["10.10.10.0/24", "1.1.1.1/32"] # Replace 1.1.1.1/32 with your own souurce IP address for management purposes. priority = "1000" allowed_protocol = "all" allowed_ports = [] } } } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`any`	n/a	yes
project	The project name to deploy the infrastructure in to.	`string`	`null`	no
region	The region into which to deploy the infrastructure in to.	`string`	`"us-central1"`	no
routes	A map containing each route setting. Note that you can only add routes using a next-hop type of internal load-balance rule. Example of variable deployment :routes = { "default-route-trust" = { name = "fw-default-trust" destination_range = "0.0.0.0/0" vpc_network_key = "fw-trust-vpc" lb_internal_name = "internal-lb" } }Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no
service_accounts	A map containing each service account setting. Example of variable deployment :service_accounts = { "sa-vmseries-01" = { service_account_id = "sa-vmseries-01" display_name = "VM-Series SA" roles = [ "roles/compute.networkViewer", "roles/logging.logWriter", "roles/monitoring.metricWriter", "roles/monitoring.viewer", "roles/viewer" ] } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no
vmseries	A map containing each individual vmseries setting. Example of variable deployment :vmseries = { "fw-vmseries-01" = { name = "fw-vmseries-01" zone = "us-east1-b" machine_type = "n2-standard-4" min_cpu_platform = "Intel Cascade Lake" tags = ["vmseries"] service_account_key = "sa-vmseries-01" scopes = [ "https://www.googleapis.com/auth/compute.readonly", "https://www.googleapis.com/auth/cloud.useraccounts.readonly", "https://www.googleapis.com/auth/devstorage.read_only", "https://www.googleapis.com/auth/logging.write", "https://www.googleapis.com/auth/monitoring.write", ] bootstrap_bucket_key = "vmseries-bootstrap-bucket-01" bootstrap_options = { panorama-server = "1.1.1.1" dns-primary = "8.8.8.8" dns-secondary = "8.8.4.4" } bootstrap_template_map = { trust_gcp_router_ip = "10.10.12.1" untrust_gcp_router_ip = "10.10.11.1" private_network_cidr = "192.168.0.0/16" untrust_loopback_ip = "1.1.1.1/32" #This is placeholder IP - you must replace it on the vmseries config with the LB public IP address after the infrastructure is deployed trust_loopback_ip = "10.10.12.5/32" } named_ports = [ { name = "http" port = 80 }, { name = "https" port = 443 } ] network_interfaces = [ { vpc_network_key = "fw-untrust-vpc" subnetwork_key = "fw-untrust-sub" private_ip = "10.10.11.2" create_public_ip = true }, { vpc_network_key = "fw-mgmt-vpc" subnetwork_key = "fw-mgmt-sub" private_ip = "10.10.10.2" create_public_ip = true }, { vpc_network_key = "fw-trust-vpc" subnetwork_key = "fw-trust-sub" private_ip = "10.10.12.2" }, ] } }For a full list of available configuration items - please refer to module documentation The bootstrap_template_map contains variables that will be applied to the bootstrap template. Each firewall Day 0 bootstrap will be parametrised based on these inputs. Multiple keys can be added and will be deployed by the code.	`any`	n/a	yes
vmseries_common	A map containing common vmseries setting. Example of variable deployment :vmseries_common = { ssh_keys = "admin:AAAABBBB..." vmseries_image = "vmseries-flex-byol-1022h2" machine_type = "n2-standard-4" min_cpu_platform = "Intel Cascade Lake" service_account_key = "sa-vmseries-01" bootstrap_options = { type = "dhcp-client" mgmt-interface-swap = "enable" } }Bootstrap options can be moved between vmseries individual instance variable (`vmseries`) and this common vmserie variable (`vmseries_common`).	`any`	n/a	yes
vpc_peerings	A map containing each VPC peering setting. Example of variable deployment :vpc_peerings = { "trust-to-spoke1" = { local_network_key = "fw-trust-vpc" peer_network_key = "fw-spoke1-vpc" local_export_custom_routes = true local_import_custom_routes = true local_export_subnet_routes_with_public_ip = true local_import_subnet_routes_with_public_ip = true peer_export_custom_routes = true peer_import_custom_routes = true peer_export_subnet_routes_with_public_ip = true peer_import_subnet_routes_with_public_ip = true } }For a full list of available configuration items - please refer to module documentation Multiple keys can be added and will be deployed by the code.	`map(any)`	`{}`	no

Outputs

Name	Description
lbs_external_ips	Public IP addresses of external network loadbalancers.
lbs_internal_ips	Private IP addresses of internal network loadbalancers.
linux_vm_ips	Private IP addresses of Linux VMs.
vmseries_private_ips	Private IP addresses of the vmseries instances.
vmseries_public_ips	Public IP addresses of the vmseries instances.

Reference Architecture Design​

Detailed Architecture and Design​

Centralized Design​

Common Option with High Availabikity​

Prerequisites​

Usage​

Post build​

Check access via web UI​

Change the public Loopback public IP Address​

Check traffic from spoke VMs​

Check traffic towards the test HTTP web server​

Test fail-over​

Reference​

Requirements​

Providers​

Modules​

Resources​

Inputs​

Outputs​