multi wan multi vpn load balance and failover with iptables

Litle setup description:

intel sbc connected with vlans
managed switch connected with vlans
wan1 dhcp 100M cable internet
wan2 dhcp 100M+ mobile internet
wan3 dhcp wifi
wan4 dhcp pppoe cable internet
5 networks 10.x.x.x one nic for each interface
multiple dhcp openvpn clients

Wan 2-4 are hotplugged, and for now not available

As i like to see ads from a specific country i like to connect with vpn. To speed it up with multiple connections. For now i have managed to set up policy routing and route some stuff in some places, but what i want is, to have more speed and redundancy on the connection. I have seen a few options but non seem to be stable. For now i have a few routing tables

routing tables

255     local
254     main
253     default
0       unspec
101     wan1
102     wan2
103     wan3
104     wan4
1001    vpn1
1002    vpn2
1003    vpn3
1004    vpn4
4001    wan_failover
4002    wan_balance
5001    vpn_failover
5002    vpn_balance

openvpn route-up

#!/bin/sh
ip route flush table vpn1
ip route add ${trusted_ip} via ${route_net_gateway} table vpn1
ip route add 0.0.0.0/1 via ${route_vpn_gateway} dev tun0 table vpn1
ip route add 128.0.0.0/1 via ${route_vpn_gateway} dev tun0 table vpn1

openvpn route-pre-down

#!/bin/sh
ip route flush table vpn1

rules

0:      from all lookup local
0:      from all fwmark 0x3eb iif enp1s0.1003 lookup vpn3
0:      from all fwmark 0x3ea iif enp1s0.1003 lookup vpn2
0:      from all fwmark 0x3e9 iif enp1s0.1003 lookup vpn1
0:      from all fwmark 0x65 iif enp1s0.1003 lookup main
0:      from all fwmark 0x3ec iif enp1s0.1003 lookup vpn4
32766:  from all lookup main
32767:  from all lookup default

iptables

# Generated by iptables-save v1.8.7 on Mon Feb 15 09:31:24 2021
*mangle
:PREROUTING ACCEPT [25663:28806277]
:INPUT ACCEPT [9783:14660509]
:FORWARD ACCEPT [15880:14145768]
:OUTPUT ACCEPT [6423:771140]
:POSTROUTING ACCEPT [22303:14916908]
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_vpn1 dst -j MARK --set-xmark 0x3e9/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_vpn2 dst -j MARK --set-xmark 0x3ea/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_vpn3 dst -j MARK --set-xmark 0x3eb/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_wan1 dst -j MARK --set-xmark 0x65/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_wan2 dst -j MARK --set-xmark 0x66/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_wan3 dst -j MARK --set-xmark 0x67/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_vpn_failover dst -j MARK --set-xmark 0x1389/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_vpn4 dst -j MARK --set-xmark 0x3ec/0xffffffff
-A PREROUTING -s 10.0.0.0/16 -m set --match-set route_wan_failover dst -j MARK --set-xmark 0xfa1/0xffffffff
COMMIT
# Completed on Mon Feb 15 09:31:24 2021
# Generated by iptables-save v1.8.7 on Mon Feb 15 09:31:24 2021
*nat
:PREROUTING ACCEPT [40:8274]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.0.4.0/24 -j MASQUERADE
-A POSTROUTING -s 10.0.3.0/24 -j MASQUERADE
-A POSTROUTING -s 10.0.2.0/24 -j MASQUERADE
-A POSTROUTING -s 10.0.1.0/24 -j MASQUERADE
-A POSTROUTING -s 10.1.0.0/24 -j MASQUERADE
COMMIT
# Completed on Mon Feb 15 09:31:24 2021
# Generated by iptables-save v1.8.7 on Mon Feb 15 09:31:24 2021
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [8351:1019718]
:TCP - [0:0]
:UDP - [0:0]
:fw-interfaces - [0:0]
:fw-open - [0:0]
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m conntrack --ctstate INVALID -j DROP
-A INPUT -p icmp -m icmp --icmp-type 8 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -p udp -m conntrack --ctstate NEW -j UDP
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m conntrack --ctstate NEW -j TCP
-A INPUT -p udp -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p tcp -j REJECT --reject-with tcp-reset
-A INPUT -j REJECT --reject-with icmp-proto-unreachable
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j fw-interfaces
-A FORWARD -j fw-open
-A FORWARD -j REJECT --reject-with icmp-host-unreachable
-A TCP -p tcp -m tcp --dport 53 -j ACCEPT
-A UDP -p udp -m udp --dport 53 -j ACCEPT
-A UDP -p udp -m udp --dport 67 -j ACCEPT
-A fw-interfaces -i enp1s0.1001 -j ACCEPT
-A fw-interfaces -i enp1s0.1002 -j ACCEPT
-A fw-interfaces -i enp1s0.1003 -j ACCEPT
-A fw-interfaces -i enp1s0.1004 -j ACCEPT
-A fw-interfaces -i wlp0s20u3 -j ACCEPT
COMMIT
# Completed on Mon Feb 15 09:31:24 2021
# Generated by iptables-save v1.8.7 on Mon Feb 15 09:31:24 2021
*raw
:PREROUTING ACCEPT [33723:37728294]
:OUTPUT ACCEPT [8355:1019938]
COMMIT
# Completed on Mon Feb 15 09:31:24 2021

...

however, what would be the way to do it? i have tried the option with a routing table with all the vpn routes and nexthop

vpn_balance

0.0.0.0/1 table vpn_balance proto static\
                nexthop dev tun0 weight 1 \
                nexthop dev tun1 weight 1 \
                nexthop dev tun2 weight 1 \
                nexthop dev tun3 weight 1

but doesent seem to be stable. It always fall back to the default gateway. For now i have the table vpn_failover active with all the vpn with different metrics but it doesent seem to work well. At the end i would like to have a system with multiple wan connections running multiple vpns on top that scale to full bandwith. But yeah, there is a bit work left.