A series of Ti-B-C-N thin films were deposited on Si (100) at 500 °C by incorporation of different amounts of N into Ti-B-C using reactive unbalanced dc magnetron sputtering in an Ar-N2 gas mixture. The effect of N content on phase configuration, nanostructure evolution, and mechanical behaviors was studied by x-ray diffraction, x-ray photoelectron spectroscopy, Raman spectroscopy, high-resolution transmission electron microscopy, and microindentation. It was found that the pure Ti-B-C was two-phased quasi-amorphous thin films comprising TiCx and TiB2. Incorporation of a small amount of N not only dissolved into TiCx but also promoted growth of TiCx nano-grains. As a result, nanocomposite thin films of nanocrystalline (nc-) TiCx(Ny) (x + y < 1) embedded into amorphous (a-) TiB2 were observed until nitrogen fully filled all carbon vacancy lattice (at that time x + y = 1). Additional increase of N content promoted formation of a-BN at the cost of TiB2, which produced nanocomposite thin films of nc-Ti(Cx,N1-x) embedded into a-(TiB2, BN). Formation of BN also decreased nanocrystalline size. Both microhardness and elastic modulus values were increased with an increase of N content and got their maximums at nanocomposite thin films consisting of nc-Ti(Cx,N1-x) and a-TiB2. Both values were decreased after formation of BN. Residual compressive stress value was successively decreased with an increase of N content. Enhancement of hardness was attributed to formation of nanocomposite structure and solid solution hardening.