Substantial areas of alpine tall tussock grasslands are being retired from grazing as part of Crown pastoral lease tenure review because of the perceived negative impact of grazing livestock. However, relatively little is known about the effect of sheep exclusion on these grasslands. We analysed data from five grazing exclosure plots over a 6-year period to examine the effect merino sheep have relative to hares and rabbits in alpine tall tussock grasslands used for summer grazing. Unfortunately, because of snow damage to fences, we were unable to detect any significant hare and rabbit effects. Over the time of this study, there was no evidence for significant vegetation recovery after exclusion of sheep grazing. This may be because of other grazing animals in the system, or the low stocking rates and non-random grazing behaviour of merino ewes. There was, however, a significant increase in the cover of exotic herbs Pilosella officinarum and P. praealta and a significant decrease in the cover of native tussocks Festuca novae-zelandiae and Poa colensoi regardless of grazing exclusion. While this pattern has been previously documented in studies at lower elevations and usually with a history of burning, our results demonstrate that alpine grasslands with no burning history can also be invaded by Pilosella spp. over relatively short time frames. Replicated grazing exclosure trials such as the one described in this paper are important for providing objective information on both long-term trends in vegetation composition and the impacts of grazing animals in alpine grasslands as a basis for making informed decisions on their future management.