How many strong neural connections a baby's brain will have is highly dependent upon the experiences and interactions they have with their caregivers and the environment.
Birth through age five is a time of rapid brain development. After the age of five the connections of neurons start to slow down and those connections that are weaker or not being used will be lost in a process known as pruning.
Here are a few great links on early brain development:
Baby Brain Map - interactive brain development map from prenatal to three.
Nature ... Nurture ... and Early Brain Development - how nature and nurture work together to shape the human brain.
Early Brain Development - what parents and caregivers need to know.
Importance of The Early Years - provides the on track guide.