When Henry's grandfather had died in 1832, he had been the second richest man in New York state, eclipsed only by John Jacob Astor. This wealth allowed Henry James Sr. to lavish a great deal of attention on the education of his children. He felt that the best education available was a European one, and the James family traveled extensively beginning when Henry was two.

The family returned to the United States in 1845, and remained there for the next ten years: interesting in that it was the longest consecutive stretch of time that Henry would ever spend in the United States. A quiet boy with few friends, Henry read voraciously but disliked formal study.

In 1855, the James family went abroad again, traveling to Paris, Geneva and Boulogne. Henry and his brothers attended a series of schools and had a series of tutors, none of whom met Henry Sr's ideals. After 3 years of a veritably nomadic lifestyle in Europe, Henry Sr. announced the superiority of American schools and the whole family moved to Newport, Rhode Island.

It only took a year, however, before Henry Sr. decided that education in America was lacking, and so the family traveled to Germany and Switzerland. Henry was enrolled in a local technical school, since his father had decided that he should learn science, where he was hopelessly out of place amongst students preparing "to be engineers, architects, machinists, and the like." Henry was eventually allowed to drop out, and the family moved to Bonn. William's decision to become an artist inspired the family to return to Newport in 1860, to allow the eldest James to study under William Morris Hunt.

When the Civil War broke out the next year, the James brothers, with their sensitive European educations, did not enlist. William, who had by then decided that he was not a painter, enrolled in Harvard while Henry remained at home. In October of 1861, a serious accident occurred: while helping to put out a fire, he experiences a "horrid even if obscure hurt." There has been a great deal of contention about what this hurt might have been: some have dramatically claimed that it was castration, but it was most likely a back injury. This injury was to pain Henry for the rest of his life.

In 1862, Henry began to study law at Harvard, but spent most of the year in the library reading books that interested him. He dropped out after a year, but remained in Cambridge as his family moved there in 1864. That year, while living with his parents at 20 Quincy Street, Henry began his writing career by having his first short story published, anonymously, in the Continental Monthly.