We present a new approach to the problem of stationary viscoplastic duct flow as modelled by the Herschel-Bulkley model, with Bingham fluids included as a special case. While the mathematical formulation of this problem is conventionally based on a variational inequality, or equivalently, on a nonsmooth minimisation problem for the flow velocity, we suggest an alternative approach. Considering the Lagrangian dual in terms of the stress, rather than the velocity, turns out to be advantageous in numerous ways. The objective functional possesses higher regularity, which ensures applicability of second order methods. Our numerical experiments with a trust-region SQP algorithm also demonstrate clearly superior performance compared to the widely used augmented Lagrangian method, although no artificial regularisation is introduced into the problem. Hence, besides providing a new theoretical angle to a classical problem, our results also pave the way for an entirely new class of numerical approaches to simulating flows of viscoplastic fluids.