Born in Pennsylvania in April of 1791, James enjoyed life as the wealthiest boy in town. His father was an immigrant who came to America just a few years earlier and soon made his wealth in agriculture and real estate. The young heir went off to Dickinson College and, perhaps due to his arrogance or mere young mischief, was nearly expelled twice.