We show, using a Bogoliubov-de Gennes (BdG) mean field theory, that the local pairing amplitude Δ(r) becomes highly inhomogeneous with increasing disorder in an s-wave superconductor. The probability distribution P(Δ) is peaked about the BCS value at low disorder, but with increasing disorder, progressively develops into a broad distribution with significant build up of weight near Δ≈0. At high disorder, the system is found to form superconducting "islands" separated by a non-superconducting sea. Surprisingly, a finite energy gap persists into the highly disordered state in spite of many sites having negligible pairing amplitude and is understood in detail within the BdG framework. Once the pairing amplitude becomes inhomogeneous, the role of quantum phase fluctuations becomes crucial in driving a superconductor-insulator transition at a critical disorder. The insulator is unusual as it has a finite gap for all disorder strengths in marked contrast to the Anderson insulator in non-interacting systems. We treat the phase fluctuations within a self consistent harmonic approximation and obtain the superfluid stiffness as a function of disorder, which agrees well with our earlier quantum Monte Carlo studies.