The rapper was let out of a prison in Ray Brook, New York, and will be under house arrest until 28 July.
The 37-year-old pleaded guilty to not filing tax returns over a five-year period in 2011 and attempted criminal possession of a weapon the year before.
He was sentenced to 28 months in prison after failing to pay $1.1m (£710,319) in taxes between 2004 and 2008.
The rapper was also sentenced to two years in prison for possessing a semi-automatic handgun, which police found hidden in his car after a concert in New York in 2007.
He served both sentences back-to-back.
Ja Rule scored number one albums with 2000’s Rule 3:36 and 2001’s Pain Is Love, which also got him a Grammy nomination.
The rapper, whose real name is Jeffrey Atkins, is best known for his collaborations with Jennifer Lopez on I’m Real and Ain’t It Funny, which both topped the US Billboard song chart in 2001.