Your car only starts reliably because two electrical workhorses share the load. The battery delivers instant power on demand, while the alternator keeps that energy flowing once the engine runs.
Yes, in most cases you should replace your car's serpentine belt when replacing the alternator. It's just much simpler to do ...