Avery's publicist said he died from complications following open-heart surgery in Glendale, California, on Tuesday afternoon.
The actor is survived by his wife, Barbara, pictured here with him in 2006
Fellow actor Alfonso Ribeiro, who played Avery's son, Carlton Banks, in the classic 1990s comedy series, paid tribute on his Facebook page.
"The world has lost a truly special man. I am very saddened to say that James Avery has passed," he wrote.
"Even though he played my father on TV, he was a wonderful father figure to me in life. He will be deeply missed."
Tatyana Ali, who played Avery's daughter, Ashley, in The Fresh Prince of Bel-Air, tweeted: "James was my teacher my protector & the most magnificent actor. We're feeling his loss very deeply. He'll always be a part of me #JamesAvery."
Having served in Vietnam during 1968 and 1969, Avery wrote poetry and television scripts before enjoying a host of acting roles in a large number of films and TV shows.
These included parts in Greys Anatomy, The Closer and Star Trek: Enterprise, and a voice role in the 1996 movie Happy Gilmore.
Avery with fellow Fresh Prince cast members, including Will Smith, in 2005
But he is best known for his portrayal of Philip Banks in The Fresh Prince of Bel-Air, in which he appeared in 141 episodes alongside on-screen nephew Will Smith.A statement issued to Sky News by his publicist, Tony Benson, said Avery was survived by his wife of 26 years, Barbara, his mother, Florence Avery, and his step-son, Kevin Waters.
The statement said he had been working until September last year on the Zach Braff film Wish I Was Here, which will premiere at the Sundance Film Festival.
She added that plans were being made for a memorial service.