A direct connection via parallel (LPT) is faster then that.
LPT is a line print terminal. I don't think that has anything to do with his connection?
You're using Vista, aren't you...?
You will have fluctuations with wireless. 54mb/s that is stated on the hardware represents the transfer speeds between two clients on the same network using 802.11a/b/g protocol.
On the other hand, having wireless could be a problem. Using a wired connection could increase connection reliability.
AOE uses 2 different protocols. TCP (Transmission Control Protocol)and UDP (User Datagram Protocol).
Ports 2300-2400 TCP or UDP
In your router, whatever your private address maybe (Click Start>Run>CMD>ipconfig /all) Whatever your default gateway is, is your private IP address, assuming you have a router. Chances are, your private IP is a class C address and will start with 192.168 and so on. You will need to locate your "port fowarding" menu and plug in either 47624 (to allow incoming traffic through that port) or any port number within 2300-2400.
Graphics cards can be strange. Is your card PCI express x16? What is your VMB (Video Memory Bandwidith)? Video memory could be 512mb or greater, but doesn't change the way the card processes images and streaming. Does AOE3 require DX10?
Paging of course isn't an issue if you're running 3gb. Make sure that there aren't any processes hanging up that could cause program interference.