A neural network is a stack of simple, tunable functions (layers of weighted sums followed by non-linearities) that together approximate very complex mappings from input to output. “Learning” means adjusting the weights so the output matches training data, via gradient descent.
Given enough layers and data, networks discover their own useful features rather than being hand-programmed. The transformer is a particular network design tuned for sequences of tokens, which is what makes large language models possible.