Skip to main content

Expedition Automation Workflows

In this section we present some workflow examples to demonstrate how to consume the Expedition-APIs. The example scripts presented here can be found under the following URL:

https://conversionupdates.paloaltonetworks.com/expedition2/examples/workflows.zip

Converting a 3rd party vendor firewall configuration​

Step 1. Generating Expedition API Keys​

In general, all the API calls to Expedition require user authentication, in order to validate the level of user rights to perform specific API calls. This is done through the use of API keys. The first step is to log into Expedition and retrieve an API key that would offer us access to later API calls.

As shown in Snippet 1, defines the Expedition IP to connect (ip variable), credentials to be used for authentication (credentials) and the URL to access the login route (url).

Once those variables have been provided, creates and establishes an SSL connection (curl) to Expedition to make a request to the login URL with the specified credentials, and collects the response from the server into the $response variable.

Expedition API responses are in JSON format In the case of a login API call, in the content section we will obtain an API key and a csrfToken. The first can be used for API consumption in scripts, while the second is intended for HTTPS Web UI requests. While the API key has an expiration time of 1 month and extends its validity time on every login (it may change in the future), a csrfToken has a shorter validity and gets regenerated for each login call.

We collect the api_key by accessing the corresponding JSON element and remove the surrounding quotes to access the API key string and format it for future authenticated API calls.

info

This authentication credentials are later prepared in a $hed variable that we will attach into the headers of the API calls we send in the future.

API syntax for Login to Expedition:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/login{ "username" : "admin" , "password" : "paloalto" }
import json
import sys
import argparse
import requests
from time import sleep
import urllib3

urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

print('LOGIN')

ip="{YourExpeditionIP}"
credentials = {"username":"admin", "password":"paloalto"}

r = requests.post('https://'+ip+'/api/v1/login', data=credentials, verify=False)
response=r.json()
apiKey = json.dumps(response['Contents']['response']['data']['content']['api_key'])
auth_token = apiKey[1:-1]
print(auth_token)
print('')

hed = {'Authorization': 'Bearer ' + auth_token}

Step 2. Creating an Expedition project​

In the large amount of automation cases, we will require having an Expedition project. Making a POST call to the projects route, we can create a project with a desired name. By default, the creator of a project is as well one of the project administrators. Notice that this time we attach the credentials $hed in the curl headers to demonstrate we have permission to create a project.

API syntax for creating a new project:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/projects{ "project": "project1", "description": "Project for testing" }
print('CREATE NEW PROJECT')
data = {"project":projectName}
r = requests.post('https://'+ip+'/api/v1/projects', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    #print('New project created successfully')
    projectId = json.dumps(response['Contents']['response']['data']['content']['id'])
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Step 3. Upload the 3rd party vendor (source) configuration into Expedition​

The migration process would require to upload one of more configuration files to be migrated. A minimum one would be the original vendor configuration file, and we plan to support bringing also routing table files to support dynamic routing tables which values are not specified in the firewall configuration.

For example, you can upload the cisco configuration "cisco.asa.txt" within the folder:

/var/www/html/expedition-api/contents/configSamples/CiscoCase/

This folder should be reachable by the www-data user and its content should be readable by the user as well.

Step 4. Converting the source configuration to a PAN-OS configuration​

The migration process takes an original configuration folder ($originalConfigFolderPath) contains one or more files required for the conversion, a path for the resulting PANOS XML config ($panosConfigPath) and the vendor selection ($vendor) to determine the conversion logic to be applied.

API syntax for Converting 3rd party vendor's configuration:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/external-tools/convert{"file": "/var/www/html/expedition-api/contents/configSamples/CiscoCase/ciscoasa.txt",
"out": "/tmp/cisco.xml"
,"vendor": "ciscoasa" }
info

Replace "vendor" value with the corresponding supported vendor value:

  • ciscoasa Cisco ASA
  • pfsense Pfsense
  • sophos Sophos
  • sonicwall Sonicwall
  • netscreen Juniper Netscreen
  • fortinet Fortinet Fortigate
  • srx Juniper Junos
  • cp-r80 Checkpoint > R80
  • ciscoswitch Cisco Switch
  • ciscoisr Cisco ISR

In the case we have already imported a PAN-OS device into Expedition, it is also possible to provide that device’s identification to use the devices configuration as a base config on which to make the conversion.

print('CONVERT FORTINET CONFIGURATION TO PALO ALTO')
data = {"file":originalConfigPath,"out":panosConfigPath,"vendor":vendor, "serial":serial}
print(data)
r = requests.post('https://'+ip+'/api/v1/external-tools/convert', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    #print('Config converted successfully')
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Step 5. Importing the configuration into the Project​

Once the conversion has done, we can import the resulting XML config file into an existing project for later configuration manipulations, such as delete unused objects, rename zones, etc. Snipt below shows imporitng the converted configuration into the project.

API syntax for Importing the PAN-OS Configuration into the project:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/projects/\{projectId}/importConfig

With sample template value "project": "1"

The full API EndPoint will become:

https://<YourExpeditionIP>/api/v1/projects/1/importConfig		"project": "$projectID"{"config": "/tmp/cisco.xml" }
print('IMPORT CONFIGURATION')
data = {"config":panosConfigPath}
r = requests.post('https://'+ip+'/api/v1/projects/'+projectId+'importConfig', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    print('Config imported successfully')
    #print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Assign PAN-OS Device to the Expedition project​

If we want to interact with a device for being able to retrieve it’s configuration, push a new configuration or learn from the traffic logs that the device has generated (this feature will come in future releases), we need to declare a Panos Device in Expedition.

Step 1. Add a PAN-OS device (Firewall or Panorama)​

The first action is to generate an API call to Expedition to declare the new device. Some initial information regarding the device is necessary, including the serial number, device name, IP address or hostname to be able to reach the device and the device model.

API syntax for adding a PAN-OS device into the project:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/devices{"serial": "serial#ofyourFW",
"device_name": "myFW2",
"hostname": "myFWhostnameorIP",
"type":"pa220"}

The API response will provide us an internal Expedition identificator for the newly generated device.

print('CREATE NEW DEVICE')
data = {
	"device_name": device_name,
	"serial": serial,
	"hostname": hostname,
	"vsys": vsys,
	"type": type
}
r = requests.post('https://'+ip+'/api/v1/devices', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    deviceId = json.dumps(response['Contents']['response']['data']['content']['id'])

    #print('New device created successfully')
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message'][0]))
print('')

Step 2. Retrieve Device API Keys​

Once the device is created, if we need to retrieve its content or make future configuration pushes to the device, it will be necessary to have a device API Key.

API syntax for retrieve PAN-OS device API key:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/device/keys{"id": "deviceID",
"role": "admin",
"username": "YourPAN-OSdeviceLoginusername",
"password":"YourPAN-OSdeviceLoginPassword"}

Notice that to be able to retrieve the API keys, we need to provide the login and password that we use to connect with the PANOS device.

When creating and registering the API key, we should provide the role that can make use of it.

There are 4 different options: Admin, User, Viewer, Personal.

info

.admin (project related): all admin users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project administrators.

.user (project related): all users users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project users.

.viewer (project related): all viewer users in a given project will use this API key to interact with the device. In the device audit logs, there will be no unique identification of the user that has submitted the API calls to the device, as it could have been done by any of the project viewers.

.personal (user related): this API key is private to the user that has requested it, and it will be used when this user requires interactions with the Panos Device. A benefit of this key is that in the device audit logs, the user will be identified as having sent the API calls to the device.

The following snippet shows how to consume the Expedition API to retrieve PAN-OS Device API keys and register these API keys into the defined device.

print('ADD DEVICE KEYS')
data = {
	"id":deviceId,
	"role":device_roles,
	"auth_type":device_auth,
	"username":device_username,
	"password":device_pwd
}
r = requests.post('https://'+ip+'/api/v1/device/keys', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    #print('Device keys added')
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Step 3. Retrieve Device’s content​

We can retrieve the config from a device by specify the value either candidate or running in the {config} parameter when consuming API call.

API syntax for retrieve PAN-OS device contents:

MethodEndPointParametersExample Value
GEThttps://<YourExpeditionIP>/api/v1/device/\{deviceId}/retrieveContent/{config}

With {deviceID}:"1" {config}:"running"

API EndPoint Full path will become:

https://<YourExpeditionIP>/api/v1/device/1/retrieveContent/running		"deviceID"-> PAN-OS device ID,
"config"-> "running"or "candidate"

This is a task that, depending on the configuration size and the connection speed with the Pan-OS Device, may take a reasonable amount of time.

Therefore, this task (as several other tasks that require long processing time) is executed as a background task so it does not block the whole script execution.

So, the API call will respond with a job identified $jobId that can be checked to identify when the assigned task is completed.

In the second part of the code, we present one approach to monitor the execution state of the download config process, which is informed in the status element within the JSON content response.

print('RETRIEVE DEVICE CONTENT')
r = requests.get('https://'+ip+'/api/v1/device/'+deviceId+'/retrieveContent/candidate', verify=False, headers=hed)
response=r.json()
jobId = json.dumps(response['Contents']['response']['data']['content']['jobId'])
print('Job id: '+jobId)
#print(json.dumps(response['Contents']['response']['response-messages']['messages']))
print('')

print('CHECK IF DEVICE CONTENT IS DOWNLOADED')
r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False, headers=hed)
response=r.json()
jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])
percentage = float(jobState) * 100
print('Retrieving content from device: '+ str(percentage)+ '%')

#Wait until all content is retrieved from device
while(jobState != '1'):
    sleep(5)
    r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False,headers=hed)
    response = r.json()
    jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])
    percentage = float(jobState)*100
    print('Retrieving content from device: '+ str(round(percentage,2))+ '%')

print('')

Step 4. Attach device to the Expedition Project​

While a configuration is being downloaded, for instance, it would be possible to attach the device to an existing project, so we have the chance to import the device configuration into the project or so we can send the resulting project configuration back to the device.

API syntax for attach device to the Expedition project:

MethodEndPointParametersExample Value
PUThttps://<YourExpeditionIP>/api/v1/projects/{"devices":"deviceID",
"id":"projectID"}

In the snippet below, we show how the device created above which id we prior stored in the $deviceId variable, is attached to the newly created project. This call is making a modification on the project settings, therefore we are sending a PUT request to the project route.

print('ASSIGN DEVICE TO PROJECT')
data = {"devices":[deviceId],"id":projectId}
r = requests.put('https://'+ip+'/api/v1/projects', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    #print('Device assigned to project')
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Push configuration to a PAN-OS device​

Pushing a project configuration to a device requires having a project created, a device declared and having credentials (Device API keys) to interact with such device.
Once we have those, what would remain is the generation of the different API calls to a PAN-OS device that will receive the configuration, and executing those calls to apply the changes on the device’s candidate configuration.

Step 1. Generating PAN-OS API calls​

Once we have a project completed and we desire to submit all or part of the configuration to a PAN-OS device, we will have to generate the API calls. We can create mega, atomic, subatomic and clear API calls.

info

mega: generates one single API call containing the complete XML configuration.

atomic: would generate one API call for each section in the configuration, such as address objects section, service object section, etc. In case of multiple DGs, we may obtain one API call for each section in the DG.

subatomic: would generate an API call for every element in a configuration. For instance, one API call for each address object in a configuration. This may be required when only certain objects should be submitted to the device.

clear: would generate API calls to delete the content of the desired section

API syntax for generating PAN-OS API calls:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/projects/\{projectId}/apiCalls/{type}

With sample parameter value: {projectID}:"1", **{type} **:"atomic"

API EndPoint Full path will become:

https://<YourExpeditionIP>/api/v1/projects/1/apiCalls/atomic		"projectId"-> Expedition Project ID,

"type"-> "mega" or "atomic" or "subatomic" or "clear"		{"serial": serial#ofyourFW",
"role": "admin",
"auth_type": "UserPassword",
"username": "YourPAN-OSdeviceLoginusername",
"password": "YourPAN-OSdeviceLoginpassword",
"sourceId": 0,
"action": "set"
}

The response will provide us a list of the generated API calls with their corresponding ids and types and order of execution.

print('PUSH CONFIG TO DEVICE')
data = {"source":source}
r = requests.post('https://'+ip+'/api/v1/projects/'+projectId+'apiCalls/atomic', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    #print('Api calls created')
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')

Step 2. Sending API calls to the PAN-OS Device​

Once generated the API calls, we can submit them to a device, either a Firewall or a Panorama. We can provide the device identification via its serial number. In the case a device is managed by a Panorama, this will be used as a proxy to submit the API calls. The Device API key that we have assigned will be used for the submission of the calls. If a personal API key has been registered in Expedition, this will take precedence over an admin/user/viewer API key.

API syntax for sending APIcalls to the PAN-OS Device:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/projects/\{projectId}/device/push

With sample parameter value: {projectID}:"1"

API EndPoint Full path will become:

https://<YourExpeditionIP>/api/v1/projects/1/device/push		"projectId"-> Expedition Project ID
{"serial": serial#ofyourFW",
"source": PAN-OSconfigfile
}

When submitting the API calls, we can enumerate those ones that we want to submit or, if none is specified, all the generated API calls will be submitted to the device in the order of execution.

Submitting the API calls is executed as a set of tasks in a job. We can monitor the status of the tasks by checking the job status. Once the process is complete, we can request the result of the execution by checking the complete information of the job.

For instance, in the below snippet, we request information regarding possible errors faced during the submission of the API calls.

data = {"serial":serial,"source":source}
r = requests.post('https://'+ip+'/api/v1/projects/'+projectID+'/device/push', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    print('Sending api calls')
    print('')
jobId = json.dumps(response['Contents']['response']['data']['content']['jobId'])

print('API CALLS STATUS')
r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False, headers=hed)
response=r.json()
jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])
percentage = float(jobState)*100
print('Pushing config to device: '+ str(round(percentage,2))+ '%')

#Wait until all content is retrieved from device
while(jobState != '1'):
    sleep(5)
    r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId, verify=False,headers=hed)
    response = r.json()
    jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])
    percentage = float(jobState)*100
    print('Pushing config device: '+ str(round(percentage,2))+ '%')

r = requests.get('https://'+ip+'/api/v1/job/status/'+jobId+'/complete', verify=False, headers=hed)
response=r.json()
jobState = json.dumps(response['Contents']['response']['data']['content']['msg']['state'])
taskCode = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['statusCode'])
if (int(taskCode)==-1):
	taskStatus = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['statusMessage'])
	taskMessage = json.dumps(response['Contents']['response']['data']['content']['msg']['TasksMsg'][0]['resultCode'])
print('Config pushed successfully')

Export PAN-OS configuration from Expedition​

Instead of making API calls from Expedition as stated in previous section, you can export the PAN-OS configuration from Expedition and manually loaded the configuraiton back to PAN-OS device.

The output file path needs to be writable by the www-data user, therefore we recommend to place those files in the project directory.

API syntax for export PAN-OS configuration from Expedition:

MethodEndPointParametersExample Value
POSThttps://<YourExpeditionIP>/api/v1/projects/\{projectId}/export{"projectId":"$projectId"}

With sample parameter value: {projectID}:"1"

API EndPoint Full path will become:

https://<YourExpeditionIP>/api/v1/projects/1/export		{"out:"-> PAN-OS configuration file ,

"sourceID:"-> you can get the source id with the sources given in the previous api call
}

print('EXPORT API CALL AS XML') #you can use it to get the final xml and then upload it manually to the firewall
outPath = '/home/userSpace/projects/fortinetDemo'
sourceId = 1 #you can get the source id with the sources given in the previous api call
data = {"out":outPath,"sourceId":sourceId}
r = requests.post('https://'+ip+'/api/v1/'+projectId+'/export', data=data, verify=False, headers=hed)
response=r.json()
success = json.dumps(response['Contents']['success'])
if success=='true':
    print(json.dumps(response['Contents']['response']['response-messages']['messages'][0]['message']))
print('')