I was wondering, since the sequence number in a TCP header field is randomly chosen during the handshake and is gradually incremented as packets are exchanged, what happens after 2^32  initial_seq_no transmissions? Does the sequence number wrap around and become 0 or is the initial value reused (or is a new connection initialized from where the previous one stopped)?
It wraps around going to 0. According to RFC 793:
It is essential to remember that the actual sequence number space is finite, though very large. This space ranges from 0 to 2**32  1. Since the space is finite, all arithmetic dealing with sequence numbers must be performed modulo 2**32. This unsigned arithmetic preserves the relationship of sequence numbers as they cycle from 2**32  1 to 0 again. There are some subtleties to computer modulo arithmetic, so great care should be taken in programming the comparison of such values. The symbol "=<" means "less than or equal" (modulo 2**32).

3Every number is less than or equal to every other number, modulo 2**32... Sep 2 '15 at 14:52

2@user20574 That's why the TCP window size is not allowed to grow larger than 1GB, and comparison of sequence numbers need to take the shortest way around (i.e. the difference must be in the range 2^31 to 2^31).– kasperdSep 2 '15 at 19:37
Does the sequence number wrap around and become 0?
Yes. All the details can be found in the TCP Specification RFC 793  Transmission Control Protocol.
Sequence Numbers
It is essential to remember that the actual sequence number space is finite, though very large. This space ranges from 0 to 2^{32}  1.
Since the space is finite, all arithmetic dealing with sequence numbers must be performed modulo 2^{32}. This unsigned arithmetic preserves the relationship of sequence numbers as they cycle from 2^{32}  1 to 0 again.
There are some subtleties to computer modulo arithmetic, so great care should be taken in programming the comparison of such values. The symbol "=<" means "less than or equal" (modulo 2^{32}).

1I don't mean to shoot the messenger, but "less than or equal (modulo N)"? Clearly the RFC author missed out on the "subtleties to computer module arithmetic". Sep 2 '15 at 14:44

In cases where the maximum window will be less than 2^31, and if
x
andy
are typeuint32_t
it's practical to definex<=y
to mean(uint32_t)(yx) < 0x80000000
.– supercatSep 2 '15 at 18:16 
@BenVoigt, mor likely they took for granted what was later described in an RFC tools.ietf.org/html/rfc1982 Sep 2 '15 at 22:10

@Carsten that is a useful arithmetic but it is not "arithmetic modulo N" Sep 2 '15 at 22:14

1@BenVoigt, yeah, whatever. Btw, I am well aware that the groups Z/(n) are not ordered, but I am also capable of interpreting statements in context. Sep 2 '15 at 22:23
Yes, it does wrap around. You can read it on Wikipedia or on RFC1323, which shows how to protect against wrapped sequence numbers.
Let me quote:
TCP timestamps are used in an algorithm known as Protection Against Wrapped Sequence numbers, or PAWS (see RFC 1323 for details). PAWS is used when the receive window crosses the sequence number wraparound boundary. In the case where a packet was potentially retransmitted it answers the question: "Is this sequence number in the first 4 GB or the second?" And the timestamp is used to break the tie.
And:
PAWS uses the same TCP Timestamps option as the RTTM mechanism described earlier, and assumes that every received TCP segment (including data and ACK segments) contains a timestamp SEG.TSval whose values are monotone nondecreasing in time. The basic idea is that a segment can be discarded as an old duplicate if it is received with a timestamp SEG.TSval less than some timestamp recently received on this connection.
In both the PAWS and the RTTM mechanism, the "timestamps" are 32bit unsigned integers in a modular 32bit space. Thus, "less than" is defined the same way it is for TCP sequence numbers, and the same implementation techniques apply. If s and t are timestamp values, s < t if 0 < (t  s) < 2**31, computed in unsigned 32bit arithmetic.