Reference Architecture with Terraform: VM-Series in AWS, Combined Design Model, Common NGFW Option

Palo Alto Networks produces several validated reference architecture design and deployment documentation guides, which describe well-architected and tested deployments. When deploying VM-Series in a public cloud, the reference architectures guide users toward the best security outcomes, whilst reducing rollout time and avoiding common integration efforts. The Terraform code presented here will deploy Palo Alto Networks VM-Series firewalls in AWS based on the centralized design; for a discussion of other options, please see the design guide from the reference architecture guides.

Reference Architecture Design

This code implements:

• a centralized design, which secures outbound, inbound, and east-west traffic flows using an AWS transit gateway (TGW). Application resources are segmented across multiple VPCs that connect in a hub-and-spoke topology, with a dedicated VPC for security services where the VM-Series are deployed
• a combined model for inbound traffic, where an AWS gateway load balancer (GWLB) is used to forward inbound traffic to the VM-Series in the security services VPC, as well as outbound and east-west traffic

Detailed Architecture and Design

Centralized Design

This design supports interconnecting a large number of VPCs, with a scalable solution to secure outbound, inbound, and east-west traffic flows using a transit gateway to connect the VPCs. The centralized design model offers the benefits of a highly scalable design for multiple VPCs connecting to a central hub for inbound, outbound, and VPC-to-VPC traffic control and visibility. In the Centralized design model, you segment application resources across multiple VPCs that connect in a hub-and-spoke topology. The hub of the topology, or transit gateway, is the central point of connectivity between VPCs and Prisma Access or enterprise network resources attached through a VPN or AWS Direct Connect. This model has a dedicated VPC for security services where you deploy VM-Series firewalls for traffic inspection and control. The security VPC does not contain any application resources. The security VPC centralizes resources that multiple workloads can share. The TGW ensures that all spoke-to-spoke and spoke-to-enterprise traffic transits the VM-Series.

Combined Model for Inbound Traffic

Inbound traffic originates outside your VPCs and is destined to applications or services hosted within your VPCs, such as web or application servers. The combined model implements inbound security by using the VM-Series and Gateway Load Balancer (GWLB) in a Security VPC, with distributed GWLB endpoints in the application VPCs. Unlike with outbound traffic, this design option does not use the transit gateway for traffic forwarding between the security VPC and the application VPCs.

Prerequisites

The following steps should be followed before deploying the Terraform code presented here.

1. Deploy Panorama e.g. by using Panorama example
2. Prepare device group, template, template stack in Panorama
3. Download and install plugin sw_fw_license for managing licenses
4. Configure bootstrap definition and license manager
5. Configure license API key
6. Configure security rules and NAT rules for outbound traffic
7. Configure interface management profile to enable health checks from GWLB
8. Configure network interfaces and subinterfaces, zones and virtual router in template

In example VM-Series are licensed using Panorama-Based Software Firewall License Management sw_fw_license, from which after configuring license manager values of panorama-server, auth-key, dgname, tplname can be used in terraform.tfvars file. Another way to bootstrap and license VM-Series is using VM Auth Key. This approach requires preparing license (auth code) in file stored in S3 bucket or putting it in authcodes option. More information can be found in document describing how to choose a bootstrap method. Please note, that other bootstrapping methods may requires additional changes in example code (e.g. adding options vm-auth-key, authcodes) and/or creating additional resources (e.g. S3 buckets).

Usage

To run this Terraform example copy the example.tfvars to terraform.tfvars and adjust it to your needs.

All Firewall VMs will be set up with an SSH key. There are two ways to approach this:

• use an existing AWS Key Pair - in this case fill out the ssh_key_name property with existing Key Pair name
• create a Key Pair with Terraform - for this you will need to adjust the follwing properties:
  • create_ssh_key - set it to true to trigger Key Pair creation
  • ssh_key_name - a name of the newly created Key Pair
  • ssh_public_key_file - path to an SSH public key that will be used to create a Key Pair

A thing worth noticing is the Gateway Load Balancer (GWLB) configuration. AWS recommends that GWLB is set up in every Availability Zone available in a particular region. This example is set up for us-east-1 which has (at the time of writing) zones from a to f. When changing the region to one that has a different number of Availability Zones, make sure you adjust the GWLB set up accordingly. You can do it in the security_vpc_subnets property - add or remove subnets for the gwlb set.

When terraform.tfvars is ready, run the following commands:

terraform init
terraform apply

To cleanup the infrastructure run:

terraform destroy

Traffic Validation

If no errors occurred during deployment, configure the VM-Series machines as expected.

• Configure the data interface so that GWLB Health Checks work properly.
• All data interfaces should use DHCP
• Create subinterfaces for Inbound, Outbound and EastWest traffic
• Create appropriate zones that will be assigned to the correct subinterfaces
• Create a Deny All rule at the very end of the rule list to eliminate unwanted traffic to the environment (In the default configuration, due to the fact that we use subinterfaces, each traffic is seen as an intrazone)
• Create policies as needed
• Make sure GWLB sees all VM-Series in the target group as healthy
• Take the ALB address and see if we are able to get the welcome page from the test app
• Make sure all traffic is visible in the monitor tab in VM-Series (check if the traffic works as expected, if it goes to the right policies)

Reference

Requirements

Name	Version
terraform	>= 1.0.0, < 2.0.0
aws	~> 5.17

Providers

Name	Version
aws	~> 5.17

Modules

Name	Source	Version
gwlb	../../modules/gwlb	n/a
gwlbe_endpoint	../../modules/gwlb_endpoint_set	n/a
natgw_set	../../modules/nat_gateway_set	n/a
public_alb	../../modules/alb	n/a
public_nlb	../../modules/nlb	n/a
subnet_sets	../../modules/subnet_set	n/a
transit_gateway	../../modules/transit_gateway	n/a
transit_gateway_attachment	../../modules/transit_gateway_attachment	n/a
vmseries	../../modules/vmseries	n/a
vpc	../../modules/vpc	n/a
vpc_routes	../../modules/vpc_route	n/a

Resources

Name	Type
aws_ec2_transit_gateway_route.from_security_to_panorama	resource
aws_ec2_transit_gateway_route.from_spokes_to_security	resource
aws_iam_instance_profile.spoke_vm_iam_instance_profile	resource
aws_iam_instance_profile.vm_series_iam_instance_profile	resource
aws_iam_role.spoke_vm_ec2_iam_role	resource
aws_iam_role.vm_series_ec2_iam_role	resource
aws_iam_role_policy.vm_series_ec2_iam_policy	resource
aws_instance.spoke_vms	resource
aws_lb_target_group_attachment.this	resource
aws_ami.this	data source
aws_caller_identity.this	data source
aws_ebs_default_kms_key.current	data source
aws_kms_alias.current_arn	data source
aws_partition.this	data source

Inputs

Name	Description	Type	Default	Required
global_tags	Global tags configured for all provisioned resources	any	n/a	yes
gwlb_endpoints	A map defining GWLB endpoints. Following properties are available: - name: name of the GWLB endpoint - gwlb: key of GWLB - vpc: key of VPC - vpc_subnet: key of the VPC and subnet connected by '-' character - act_as_next_hop: set to true if endpoint is part of an IGW route table e.g. for inbound traffic - to_vpc_subnets: subnets to which traffic from IGW is routed to the GWLB endpoint Example: gwlb_endpoints = { security_gwlb_eastwest = { name = "eastwest-gwlb-endpoint" gwlb = "security_gwlb" vpc = "security_vpc" vpc_subnet = "security_vpc-gwlbe_eastwest" act_as_next_hop = false to_vpc_subnets = null } }	map(object({ name = string gwlb = string vpc = string vpc_subnet = string act_as_next_hop = bool to_vpc_subnets = string }))	{}	no
gwlbs	A map defining Gateway Load Balancers. Following properties are available: - name: name of the GWLB - vpc_subnet: key of the VPC and subnet connected by '-' character Example: gwlbs = { security_gwlb = { name = "security-gwlb" vpc_subnet = "security_vpc-gwlb" } }	map(object({ name = string vpc_subnet = string }))	{}	no
name_prefix	Prefix used in names for the resources (VPCs, EC2 instances, autoscaling groups etc.)	string	n/a	yes
natgws	A map defining NAT Gateways. Following properties are available: - name: name of NAT Gateway - vpc_subnet: key of the VPC and subnet connected by '-' character Example: natgws = { security_nat_gw = { name = "natgw" vpc_subnet = "security_vpc-natgw" } }	map(object({ name = string vpc_subnet = string }))	{}	no
panorama_attachment	An object defining TGW attachment and CIDR for Panorama. Following properties are available: - transit_gateway_attachment_id: ID of attachment for Panorama - vpc_cidr: CIDR of the VPC, where Panorama is deployed Example: panorama = { transit_gateway_attachment_id = "tgw-attach-123456789" vpc_cidr = "10.255.0.0/24" }	object({ transit_gateway_attachment_id = string vpc_cidr = string })	null	no
region	AWS region used to deploy whole infrastructure	string	n/a	yes
spoke_albs	A map defining Application Load Balancers deployed in spoke VPCs. Following properties are available: - rules: Rules defining the method of traffic balancing - vms: Instances to be the target group for ALB - vpc: The VPC in which the load balancer is to be run - vpc_subnet: The subnets in which the Load Balancer is to be run - security_gropus: Security Groups to be associated with the ALB	map(object({ rules = any vms = list(string) vpc = string vpc_subnet = string security_groups = string }))	n/a	yes
spoke_nlbs	A map defining Network Load Balancers deployed in spoke VPCs. Following properties are available: - vpc_subnet: key of the VPC and subnet connected by '-' character - vms: keys of spoke VMs Example: spoke_lbs = { "app1-nlb" = { vpc_subnet = "app1_vpc-app1_lb" vms = ["app1_vm01", "app1_vm02"] } }	map(object({ vpc_subnet = string vms = list(string) }))	{}	no
spoke_vms	A map defining VMs in spoke VPCs. Following properties are available: - az: name of the Availability Zone - vpc: name of the VPC (needs to be one of the keys in map vpcs) - vpc_subnet: key of the VPC and subnet connected by '-' character - security_group: security group assigned to ENI used by VM - type: EC2 type VM Example: spoke_vms = { "app1_vm01" = { az = "eu-central-1a" vpc = "app1_vpc" vpc_subnet = "app1_vpc-app1_vm" security_group = "app1_vm" type = "t2.micro" } }	map(object({ az = string vpc = string vpc_subnet = string security_group = string type = string }))	{}	no
ssh_key_name	Name of the SSH key pair existing in AWS key pairs and used to authenticate to VM-Series or test boxes	string	n/a	yes
tgw	A object defining Transit Gateway. Following properties are available: - create: set to false, if existing TGW needs to be reused - id: id of existing TGW or null - name: name of TGW to create or use - asn: ASN number - route_tables: map of route tables - attachments: map of TGW attachments Example: tgw = { create = true id = null name = "tgw" asn = "64512" route_tables = { "from_security_vpc" = { create = true name = "from_security" } } attachments = { security = { name = "vmseries" vpc_subnet = "security_vpc-tgw_attach" route_table = "from_security_vpc" propagate_routes_to = "from_spoke_vpc" } } }	object({ create = bool id = string name = string asn = string route_tables = map(object({ create = bool name = string })) attachments = map(object({ name = string vpc_subnet = string route_table = string propagate_routes_to = string })) })	null	no
vmseries	A map defining VM-Series instances Following properties are available: - instances: map of VM-Series instances - bootstrap_options: VM-Seriess bootstrap options used to connect to Panorama - panos_version: PAN-OS version used for VM-Series - ebs_kms_id: alias for AWS KMS used for EBS encryption in VM-Series - vpc: key of VPC - gwlb: key of GWLB - subinterfaces: configuration of network subinterfaces used to map with GWLB endpoints - system_services: map of system services - application_lb: ALB placed in front of the Firewalls' public interfaces - network_lb: NLB placed in front of the Firewalls' public interfaces Example: vmseries = { vmseries = { instances = { "01" = { az = "eu-central-1a" } "02" = { az = "eu-central-1b" } } # Value of panorama-server, auth-key, dgname, tplname can be taken from plugin sw\_fw\_license bootstrap_options = { mgmt-interface-swap = "enable" plugin-op-commands = "panorama-licensing-mode-on,aws-gwlb-inspect:enable,aws-gwlb-overlay-routing:enable" dhcp-send-hostname = "yes" dhcp-send-client-id = "yes" dhcp-accept-server-hostname = "yes" dhcp-accept-server-domain = "yes" } panos_version = "10.2.3" # TODO: update here ebs_kms_id = "alias/aws/ebs" # TODO: update here # Value of vpc must match key of objects stored in vpcs vpc = "security_vpc" # Value of gwlb must match key of objects stored in gwlbs gwlb = "security_gwlb" interfaces = { private = { device_index = 0 security_group = "vmseries_private" vpc_subnet = "security_vpc-private" create_public_ip = false source_dest_check = false } mgmt = { device_index = 1 security_group = "vmseries_mgmt" vpc_subnet = "security_vpc-mgmt" create_public_ip = true source_dest_check = true } public = { device_index = 2 security_group = "vmseries_public" vpc_subnet = "security_vpc-public" create_public_ip = true source_dest_check = false } } # Value of gwlb\_endpoint must match key of objects stored in gwlb\_endpoints subinterfaces = { inbound = { app1 = { gwlb_endpoint = "app1_inbound" subinterface = "ethernet1/1.11" } app2 = { gwlb_endpoint = "app2_inbound" subinterface = "ethernet1/1.12" } } outbound = { only_1_outbound = { gwlb_endpoint = "security_gwlb_outbound" subinterface = "ethernet1/1.20" } } eastwest = { only_1_eastwest = { gwlb_endpoint = "security_gwlb_eastwest" subinterface = "ethernet1/1.30" } } } system_services = { dns_primary = "4.2.2.2" # TODO: update here dns_secondy = null # TODO: update here ntp_primary = "pool.ntp.org" # TODO: update here ntp_secondy = null # TODO: update here } application_lb = null network_lb = null } }	map(object({ instances = map(object({ az = string })) bootstrap_options = object({ mgmt-interface-swap = string plugin-op-commands = string panorama-server = string auth-key = string dgname = string tplname = string dhcp-send-hostname = string dhcp-send-client-id = string dhcp-accept-server-hostname = string dhcp-accept-server-domain = string }) panos_version = string ebs_kms_id = string vpc = string gwlb = string interfaces = map(object({ device_index = number security_group = string vpc_subnet = string create_public_ip = bool source_dest_check = bool })) subinterfaces = map(map(object({ gwlb_endpoint = string subinterface = string }))) system_services = object({ dns_primary = string dns_secondy = string ntp_primary = string ntp_secondy = string }) application_lb = object({ name = string rules = any }) network_lb = object({ name = string rules = any }) }))	{}	no
vpcs	A map defining VPCs with security groups and subnets. Following properties are available: - name: VPC name - cidr: CIDR for VPC - nacls: map of network ACLs - security_groups: map of security groups - subnets: map of subnets with properties: - az: availability zone - set: internal identifier referenced by main.tf - nacl: key of NACL (can be null) - routes: map of routes with properties: - vpc_subnet - built from key of VPCs concatenate with - and key of subnet in format: VPCKEY-SUBNETKEY - next_hop_key - must match keys use to create TGW attachment, IGW, GWLB endpoint or other resources - next_hop_type - internet_gateway, nat_gateway, transit_gateway_attachment or gwlbe_endpoint Example: vpcs = { example_vpc = { name = "example-spoke-vpc" cidr = "10.104.0.0/16" nacls = { trusted_path_monitoring = { name = "trusted-path-monitoring" rules = { allow_inbound = { rule_number = 300 egress = false protocol = "-1" rule_action = "allow" cidr_block = "0.0.0.0/0" from_port = null to_port = null } } } } security_groups = { example_vm = { name = "example_vm" rules = { all_outbound = { description = "Permit All traffic outbound" type = "egress", from_port = "0", to_port = "0", protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } } } } subnets = { "10.104.0.0/24" = { az = "eu-central-1a", set = "vm", nacl = null } "10.104.128.0/24" = { az = "eu-central-1b", set = "vm", nacl = null } } routes = { vm_default = { vpc_subnet = "app1_vpc-app1_vm" to_cidr = "0.0.0.0/0" next_hop_key = "app1" next_hop_type = "transit_gateway_attachment" } } } }	map(object({ name = string cidr = string nacls = map(object({ name = string rules = map(object({ rule_number = number egress = bool protocol = string rule_action = string cidr_block = string from_port = string to_port = string })) })) security_groups = any subnets = map(object({ az = string set = string nacl = string })) routes = map(object({ vpc_subnet = string to_cidr = string next_hop_key = string next_hop_type = string })) }))	{}	no

Outputs

Name	Description
application_load_balancers	FQDNs of Application Load Balancers
network_load_balancers	FQDNs of Network Load Balancers.
vmseries_public_ips	Map of public IPs created within vmseries module instances.