There is considerable evidence for computationally complex behavior, that is, behavior that appears to require the equivalent of mathematical calculation by the organism. Spatial navigation by path integration is perhaps the best example. The most influential account of such behavior has been Gallistel's (1990) computational-representational theory, which assumes that organisms represent key environmental variables such as direction and distance traveled as real numbers stored in engrams and are able to perform arithmetic computations on those representations. But how are these computations accomplished? A novel perspective is gained from the historical development of algebra. We propose that computationally complex behavior suggests that the perceptual system represents an algebraic field, which is a mathematical concept that expresses the structure underlying arithmetic. Our field representation hypothesis predicts that the perceptual system computes 2 operations on represented magnitudes, not 1. We review recent research in which human observers were trained to estimate differences and ratios of stimulus pairs in a nonsymbolic task without explicit instruction (Grace, Morton, Ward, Wilson, & Kemp, 2018). Results show that the perceptual system automatically computes two operations when comparing stimulus magnitudes. A field representation offers a resolution to longstanding controversies in psychophysics about which of 2 algebraic operations is fundamental (e.g., the Fechner-Stevens debate), overlooking the possibility that both might be. In terms of neural processes that might support computationally complex behavior, our hypothesis suggests that we should look for evidence of 2 operations and for symmetries corresponding to the additive and multiplicative groups.