EdgeRouter - How to capture packets on multiple EdgeRouter interfaces for debugging purposes

Kudos to our community member esprosenx for providing this article! 

I recently had an issue communicating from my laptop on a private network behind an EdgeRouter out to the Internet to a specific IP address.

Here are the steps I took in order to capture the packets that helped troubleshoot the issue:

In my case I was having trouble getting access to www.ups.com and so the first thing I did was a DNS lookup.  Note that I ran the "dig" command from my MacBook Pro command line since it is not (currently) included in the EdgeRouter code distribution.

Erics-MacBook-Pro-2:~ erosenbe$ dig www.ups.com @4.2.2.1
; <<>> DiG 9.8.3-P1 <<>> www.ups.com @4.2.2.1
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 14300
;; flags: qr rd ra; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;www.ups.com.INA
;; ANSWER SECTION:
www.ups.com.254INCNAMEwww.ups.com.akadns.net.
www.ups.com.akadns.net.254INCNAMEwww.upsprodcidr2.com.akadns.net.
www.upsprodcidr2.com.akadns.net. 30 INCNAMEorigin.www.ups.com.
origin.www.ups.com.2149INA153.2.224.50
origin.www.ups.com.2149INA153.2.228.50
;; Query time: 170 msec
;; SERVER: 4.2.2.1#53(4.2.2.1)
;; WHEN: Fri Apr 19 09:09:26 2013
;; MSG SIZE rcvd: 149
Erics-MacBook-Pro-2:~ erosenbe$ 

Note that in this case it resolved to two different A records, but for the purposes of my testing, both were broken, so I just chose one of the two.

Next I setup a capture on my MacBook to actually grab all IP packets going to or from the specific IP in question:

Erics-MacBook-Pro-2:~ erosenbe$ tcpdump -i en0 -s 0 -w UPSCapMac.cap host 153.2.224.50
tcpdump: listening on en0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C11 packets captured
1607 packets received by filter
0 packets dropped by kernel
Erics-MacBook-Pro-2:~ erosenbe$ 

The -i flag above identifies which interface on my mac I wanted to use (you can use the ifconfig) command to figure out which is which for you), and the "-s 0" setting is very important as it tells tcpdump to "snarf" the *entire* frame instead of only the first "x" number of bytes (why they made this the default, I do not know...).  The -w specifies an output file name (by default it drops the file in the current directory you execute it from).  And finally, the "host x.x.x.x" command is a filter definition that only captures traffic to that single IP.  This is great as it avoids all the other packets that are flying through my interface during the capture.

Similarly, I setup another capture on the EdgeRouter itself on the *outside* interface (eth0 in my case).  This was in a SSH session to the device that I executed it.

ubnt@plunger:~$ sudo tcpdump -i eth0 -s 0 -w UPSCapPlungerETH0.cap host 153.2.224.50
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
^C29 packets captured
29 packets received by filter
0 packets dropped by kernel
ubnt@plunger:~$ 

I furthermore created another SSH session to the EdgeRouter and setup a capture on the *inside* interface (eth1) so that I could see what NAT modifications, etc... the EdgeRouter had made to the packet.

ubnt@plunger:~$ sudo tcpdump -i eth1 -s 0 -w UPSCapPlungerETH1.cap host 153.2.224.50
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
^C26 packets captured
26 packets received by filter
0 packets dropped by kernel
ubnt@plunger:~$ 

I should note that in order to simplify this down to the most minimal test case possible, I just wanted to open a TCP session from my mac to this www.ups.comIP address on port 80 (the web port).  The best way to accomplish this was with the "telnet" application.

Erics-MacBook-Pro-2:~ erosenbe$ telnet 153.2.224.50 80
Trying 153.2.224.50...
telnet: connect to address 153.2.224.50: Operation timed out
telnet: Unable to connect to remote host
Erics-MacBook-Pro-2:~ erosenbe$  

Once I had generated the packets I wanted to capture, I canceled all three packet captures (usinc Ctrl-C) and then I copied the ones from the EdgeRouter down to my laptop:

Erics-MacBook-Pro-2:~ erosenbe$ scp ubnt@192.168.0.1:/home/ubnt/UPSCapPlungerETH0.cap .
Welcome to EdgeOS
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
ubnt@192.168.0.1's password:
UPSCapPlungerETH0.cap 100% 2100 2.1KB/s 00:00
Erics-MacBook-Pro-2:~ erosenbe$
Erics-MacBook-Pro-2:~ erosenbe$ scp ubnt@192.168.0.1:/home/ubnt/UPSCapPlungerETH1.cap .
Welcome to EdgeOS
By logging in, accessing, or using the Ubiquiti product, you
acknowledge that you have read and understood the Ubiquiti
License Agreement (available in the Web UI at, by default,
http://192.168.1.1) and agree to be bound by its terms.
ubnt@192.168.0.1's password:
UPSCapPlungerETH1.cap 100% 2100 2.1KB/s 00:00
Erics-MacBook-Pro-2:~ erosenbe$ 

Once these files are down on my computer then I was able to go through them with WireShark and see where the issue was.

I thought this info might be of use for a KnowledgeBase article.

-Eric

Related Articles