The little boy was playing with his two year old little sister when the accident happened.
Police investigating the accident say the gun was left with a single shot in the chamber but no one in the house knew that.
Two year old Caroline Starks was taken to Cumberland County hospital after being shot but died there.
The coroner in Lexington, Kentucky was nearly at a loss for words when he said it was "just one of those crazy accidents."
He also told the Lexington Herald-Leader "it's a little rifle for a kid, the little boy's used to shooting the little gun."
The boy was given the .22 caliber gun called “The Cricket” as a gift last year. The model is marketed as “My first rifle.”
The children’s mother was home at the time of the accident.