His father was a lawyer of undistinguished abilities but of cultured and literary tastes. His mother was devotedly religious, and reared her family in the strict Presbyterian faith. His grandfather's hotel, the Lanier House, was the centre of a cordial, hospitable social life. The city of Macon, a prosperous commercial centre, counted among its citizens many wealthy plantation owners but few who aspired to higher education or intellectual achievement. Even his father's literary interests seem to have been confined to Shakespeare and Addison and Sir Walter Scott—to the items of that self-sufficient culture which reigned everywhere in the South before the Civil War. Although Scott and Froissart fired Lanier's young mind with ideals of chivalry, the thing which set him apart from the Macon school boys was his remarkable musical ability. At seven he had made himself a reed flageolet, and on receiving a flute at Christmas he soon organized quartets and bands among his playfellows. Indeed, it was because of his leadership in serenading parties at Oglethorpe, which he had entered shortly before his fifteenth birthday, that his father brought him home to spend a year in the Macon post office. When he returned to Oglethorpe as a junior he began to play the violin with such effect that he would at times lose consciousness for hours. His father, fearing this stimulation, induced him to return to the flute and discouraged him as much as possible from devotion to music. The result is seen in the boy's journal:
The prime inclination—that is, natural bent (which I have checked, though) of my nature is to music, and for that I have the greatest talent; indeed, not boasting, for God gave it me, I have an extraordinary musical talent, and feel it within me plainly that I could rise as high as any composer. But I cannot bring myself to believe that I was intended for a musician, because it seems so small a business in comparison with other things which, it seems to me, I might do.His later life seems to bear out the assumption that America, by his father's solicitude and the social pressure of Southern opinion at the time, was deprived of another distinguished name in music.