This thesis describes an investigation of the performance of binary differential phase shift keying (DPSK) modulation in an optical heterodyne communication system when both laser phase noise and shot noise are present. The laser phase noise is modelled as a Brownian motion process while the shot noise is modelled by additive white Gaussian noise. Four different receiver structures are considered: (i) the standard delay-and-multiply form, which is optimal when only the shot noise is present but is severely degraded by quite small amounts of phase noise; (ii) the matched receiver, which tolerates more phase noise by combating the most likely phase noise induced error-event; (iii) the innovations receiver that is derived to be optimal for small amounts of phase noise; and (iv) the weighted receiver, which is the innovations receiver optimised for minimum bit error rate (BER) at specific levels of phase and shot noise. A new and accurate analysis of the standard receiver is presented, based on a perturbation solution for the probability density function (pdf) of filtered phase noise. This approach is used to verify the accuracy of a commonly applied approximation: that of neglecting the effects of narrow-band filtering on the magnitude of the phase noise corrupted signal. The well known BER floor effect of standard DPSK systems is observed. A second analysis method based on the moments of the receiver decision statistic is also presented. This method is found to be inaccurate for small BER's, and illustrates the difficulty in applying moment based methods to the extreme tail of a pdf. To attempt to improve the phase noise tolerance of a DPSK receiver, the calculus of variations is used to determine the most-likely phase noise path that causes a detection error in the standard receiver. The matched receiver is then formulated, such that these phase noise paths are less likely to cause an error. An analysis of this receiver in terms of likelihood functions provides a lower bound on its BER. This result is then shown to yield an optimal rule for combining the various branches of the receiver, such that the performance bound can be achieved. Simulated results for the matched receiver show that it tolerates larger amounts of phase noise than the standard receiver, and that the BER floor has been lowered. The application of the innovations approach allows a log-likelihood ratio to be formulated for the DPSK detection problem in terms of minimum mean-square-error estimates of the phase noise path. In order to evaluate these estimates, an approximation is used that is valid for small amounts of phase noise. This leads to a simple but useful form for the log-likelihood ratio, which is implemented by the innovations receiver. An expression for the BER floor of the innovations receiver is developed, which provides a comparison with the BER floor of the the standard receiver. Simulated results are also obtained, and combined with the BER floor results show that the innovations receiver does not offer much improvement over the standard receiver. Investigation of the innovations receiver indicates that its relatively poor performance can be attributed to the small-phase-noise approximation used in its derivation. This results in the weighting factor of its band-pass filters being non-optimal for other than very small amounts of phase noise. Optimisation of this weighting factor as a function of the amount of phase and shot noise leads to the weighted receiver. This optimisation is achieved through the use of moments of the receiver decision statistic, for which a formulation is presented. Computed BER floors in conjunction with simulated results show that the weighted receiver performs significantly better than the standard receiver, and also better than the matched receiver. In particular, the BER floor of the weighted receiver can be lowered by trading off signal power, and can theoretically be reduced to an arbitrarily low level given enough received power.