Well, may have problems with this host sorted out, though it’s hard to tell for sure. Chris spent a lot of time working to try and fix things, and went to the trouble of replacing my cheap-ass RTL8139 card with a far better Intel NIC. Unfortunately, was still experiencing the same problems as before.
To give some background, there are two interfaces on this machine. eth0 is configured as the exposed DMZ interface, where all the services, and subsequently all of the netfilter stuff would be set up. eth1 is available to (graciously) be able to mount things like portage trees and nfs backup mounts within Chris’ internal network. This also has the really nice advantage that if eth0 gets shut down due to a panic ruleset (which I don’t really expect, but is configured) or something similar, the machine is still indirectly accessible via the inside interface. eth1 is brought up via dhcpcd, which has the side effect of setting a secondary default route, which struck us as a bit weird back when it happened, but things still seemed to be working.
So, I said “what the hell?” and deleted the secondary route on eth1, and the machine suddenly was back on the Internet as if nothing ever happened. It’s still too early to say it wasn’t a coincidence, but I’m hopeful. Assuming this resolves things, this just means I need to add an additional
-G to my dhcpcd configuration for eth1, and all will be well. If that’s the case, a few more days of testing and I can actually switch to this thing for good.