The human brain is a complex organ containing about 100 billion neurons, connecting to each other by as many as 1000 trillion synaptic connections. In neuroscience and computer science, spiking neural network models are a conventional model that allow us to simulate particular regions of the brain. In this thesis we look at these spiking neural networks, and how we can benefit future works and develop new models. We have two main focuses, our first focus is on the development of a modular framework for devising new models and unifying terminology, when describing artificial spiking neural networks. We investigate models and algorithms proposed in the literature and offer insight on designing spiking neural networks. We propose the Spiking State Machine as a standard experimental framework, which is a subset of Liquid State Machines. The design of the Spiking State Machine is to focus on the modularity of the liquid component in respect to spiking neural networks. We also develop an ontology for describing the liquid component to distinctly differentiate the terminology describing our simulated spiking neural networks from its anatomical counterpart. We then evaluate the framework looking for consequences of the design and restrictions of the modular liquid component. For this, we use nonlinear problems which have no biological relevance; that is an issue that is irrelevant for the brain to solve, as it is not likely to encounter it in real world circumstances. Our results show that a Spiking State Machine could be used to solve our simple nonlinear problems which only needed small networks. For more complex problems the liquid required a substantial growth in order to find a solution. Growing the network may be unfeasible for more challenging problems compared to manually constructed networks, which may provide better clarification of the situation and the mechanics involved. Our second focus is the modelling of spontaneous neurotransmission in the brain, for which we propose a novel model called the imperfect synaptic model. Imperfect synapses allow for communication without spiking, by potential differences in the membrane potentials of neurons. We evaluated this method against the standard synapse model on the classical control system problem of balancing a pole on a cart. We found that networks with imperfect synapses responded quicker to external actions. Otherwise, there was no significant difference to networks of standard synapses. We discovered that by doping a network with imperfect synapses, the network learnt to balance the cart-pole system in half the time compared to networks with standard synapses and networks with only imperfect synapses. We suspect that doping imperfect synapses increase the separation potential of the network, by allowing some neurons to inhibit or excite other neurons without action potentials. This decreases the chance of repeated structures occurring within the network. Our proposed method does not cover all mechanisms of neurotransmission, and in general, neurons will require at least some prior stimulation. Computationally, the imperfect synaptic model requires more computation per propagation cycle.