Apparently high temps are ok according to the support. The time fan kicks in when it is really hot but I don’t think it is healthy for the hardware:

Package id 0:  +96.0°C  (high = +105.0°C, crit = +105.0°C)  
Core 0:        +96.0°C  (high = +105.0°C, crit = +105.0°C)  
Core 1:        +96.0°C  (high = +105.0°C, crit = +105.0°C)  
Core 2:        +96.0°C  (high = +105.0°C, crit = +105.0°C)  
Core 3:        +94.0°C  (high = +105.0°C, crit = +105.0°C)

eth3-pci-0400  
Adapter: PCI adapter  
PHY Temperature:  +81.0°C  
MAC Temperature:  +81.0°C

acpitz-acpi-0  
Adapter: ACPI interface  
temp1:       +100.0°C  (crit = +110.0°C)

nct5585-isa-0a20  
Adapter: ISA adapter  
Vcore:               1.10 V  (min =  +0.00 V, max =  +1.74 V)  
\+5V:                 5.04 V  (min =  +0.00 V, max =  +0.00 V)  ALARM  
3.3V:                3.33 V  (min =  +0.00 V, max =  +0.00 V)  ALARM  
\+12V:               11.81 V  (min =  +0.00 V, max =  +0.00 V)  ALARM  
VCCIN\_AUX:           1.79 V  (min =  +0.00 V, max =  +0.00 V)  ALARM  
System Fan1 Speed:    0 RPM  (min =    0 RPM)  
CPU Temp:           +61.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor  
System Temp:        +63.0°C  (high = +80.0°C, hyst = +75.0°C)  sensor = thermistor

eth0-pci-0300  
Adapter: PCI adapter  
PHY Temperature:  +81.0°C  
MAC Temperature:  +81.0°C

So I designed a little cooling device that uses a thermistor to measure the bottom of the box and turn the large fan on, its speed depends on the temperature. I printed a housing for the fan and mounted it on the side of the box; it is controlled by Arduino. 

Now the temp is 32°C when not loaded and 45°C when I stress-test the Firewalla!

Cooling the Firewalla Gold Pro